Skip to content

Model

Model Examples

Examples of using the Model Class are in the Examples section at the bottom of this page. AWS Model setup and deployment are quite complicated to do manually but the Workbench Model Class makes it a breeze!

Model: Manages AWS Model Package/Group creation and management.

Models are automatically set up and provisioned for deployment into AWS. Models can be viewed in the AWS Sagemaker interfaces or in the Workbench Dashboard UI, which provides additional model details and performance metrics

Model

Bases: ModelCore

Model: Workbench Model API Class.

Common Usage 
my_model = Model(name)
my_model.details()
my_model.to_endpoint()
Source code in src/workbench/api/model.py 
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
class Model(ModelCore):
    """Model: Workbench Model API Class.

    Common Usage:
        ```python
        my_model = Model(name)
        my_model.details()
        my_model.to_endpoint()
        ```
    """

    def details(self, **kwargs) -> dict:
        """Retrieve the Model Details.

        Returns:
            dict: A dictionary of details about the Model
        """
        return super().details(**kwargs)

    def to_endpoint(
        self,
        name: str = None,
        tags: list = None,
        serverless: bool = True,
        mem_size: int = 2048,
        max_concurrency: int = 2,
        instance: str = None,
        data_capture: bool = False,
        async_endpoint: bool = False,
        min_instances: int = 0,
        max_instances: int = None,
        auto_scaling_mode: str = None,
        scale_in_idle_minutes: int = None,
        async_max_concurrent: int = 1,
    ) -> Endpoint:
        """Create an Endpoint from the Model.

        Args:
            name (str): Set the name for the endpoint. If not specified, an automatic name will be generated
            tags (list): Set the tags for the endpoint. If not specified automatic tags will be generated.
            serverless (bool): Set the endpoint to be serverless (default: True)
            mem_size (int): The memory size for the Endpoint in MB (default: 2048)
            max_concurrency (int): The maximum concurrency for the Endpoint (default: 2)
            instance (str): The instance type for Realtime Endpoints (default: None = auto-select based on model)
            data_capture (bool): Enable data capture for the Endpoint (default: False)
            async_endpoint (bool): Deploy as an async endpoint (default: False). Async
                endpoints are conceptually *batch* endpoints — support up to 60-minute
                per-invocation timeouts, use S3 for I/O, and scale 0 → max_instances →
                0 as traffic arrives and drains.
            min_instances (int): Autoscaler floor (default: 0 — scale to zero).
            max_instances (int): Autoscaler upper bound for async endpoints (default:
                None = use the 8-instance default in register_autoscaling).
            auto_scaling_mode (str): Autoscaling strategy for async endpoints.
                Defaults to "batch" (step scaling — instant jump 0 → max on traffic,
                fast drain to min when idle). Other modes may be added in the future.
            scale_in_idle_minutes (int): Batch-mode only — minutes of empty queue
                before scaling in to min_instances (default: None = register_autoscaling's
                default of 15).
            async_max_concurrent (int): MaxConcurrentInvocationsPerInstance for async
                endpoints (default: 1, suits CPU-bound predict_fn). Bump for I/O-bound
                workloads where multiple invocations can share a single instance.

        All deploy-time sizing/autoscaling args are persisted to the endpoint's
        workbench_meta so later reconstructions can see what it was deployed with.

        Returns:
            Endpoint: The Endpoint (auto-routes to the async transport when
            ``async_endpoint=True``).
        """

        # Ensure the endpoint_name is valid
        if name:
            Artifact.is_name_valid(name, delimiter="-", lower_case=False)

        # If the endpoint_name wasn't given generate it
        else:
            name = self.name.replace("_features", "") + ""
            name = Artifact.generate_valid_name(name, delimiter="-")

        # Create the Endpoint Tags
        tags = [name] if tags is None else tags

        # Create an Endpoint from the Model
        model_to_endpoint = ModelToEndpoint(
            self.name,
            name,
            serverless=serverless,
            instance=instance,
            async_endpoint=async_endpoint,
            min_instances=min_instances,
            max_instances=max_instances,
            auto_scaling_mode=auto_scaling_mode,
            scale_in_idle_minutes=scale_in_idle_minutes,
            async_max_concurrent=async_max_concurrent,
        )
        model_to_endpoint.set_output_tags(tags)
        model_to_endpoint.transform(
            mem_size=mem_size,
            max_concurrency=max_concurrency,
            data_capture=data_capture,
        )

        end = Endpoint(name)
        end.set_owner(self.get_owner())
        return end

    def prox(self, space: str) -> "Optional[Union[FingerprintProximity, FeatureSpaceProximity]]":  # noqa: F821
        """Return a proximity model for this Model — precomputed if it has one, else fresh.

        Precomputed-first: returns the proximity the model already carries for ``space``
        (frozen at training time). If it has none, builds one fresh over the model's
        FeatureSet — using the model's own features/target — and logs that it did so.
        Cached per ``space`` on this instance.

        Returns ``None`` for ``space="features"`` when the model's features include
        ``smiles`` or ``fingerprint`` — that's a structure model, so feature-space
        proximity is meaningless; use ``prox("fingerprint")`` instead.

        Args:
            space: ``"fingerprint"`` or ``"features"``.

        Returns:
            A FingerprintProximity or FeatureSpaceProximity (or None — see above).
        """
        if space not in ("fingerprint", "features"):
            raise ValueError(f"space must be 'fingerprint' or 'features', got {space!r}")

        structure_cols = {"smiles", "fingerprint"}.intersection(f.lower() for f in (self.features() or []))
        if space == "features" and structure_cols:
            self.log.important(
                f"{self.name}: features are structural ({structure_cols}) — "
                "use prox('fingerprint') for structural neighbors."
            )
            return None

        if not hasattr(self, "_prox_cache"):
            self._prox_cache = {}
        if space not in self._prox_cache:
            prox = precomputed_model_proximity(self, space)
            if prox is None:
                from workbench.api import FeatureSet

                self.log.important(
                    f"No precomputed {space} proximity for {self.name}; building fresh from its FeatureSet..."
                )
                fs = FeatureSet(self.get_input())
                if space == "features":
                    prox = fs.prox("features", feature_list=self.features(), target=self.target())
                else:
                    prox = fs.prox("fingerprint", target=self.target())
            self._prox_cache[space] = prox
        return self._prox_cache[space]

    def uq_model(
        self,
        version: str = None,
        refresh_proximity: bool = False,
        radius: int = 2,
        n_bits: int = 4096,
    ) -> "UQModelV0 | UQModelV1":  # noqa: F821
        """Load this model's fitted UQ model (V0 or V1) for uncertainty-quantified inference.

        Usage:
            model = Model("my-model")
            uq = model.uq_model()                    # bundle-default version
            uq_v0 = model.uq_model(version="v0")     # original isotonic calibrator
            uq_v1 = model.uq_model(version="v1")     # proximity-augmented RF
            out = uq.predict(test_df[["smiles"]], predictions, prediction_std)

        Args:
            version (str | None): Which UQ version to load — ``"v0"`` (original
                isotonic-on-(prediction, std) calibrator) or ``"v1"`` (proximity-
                augmented RandomForest error model). If ``None``, reads
                ``hyperparameters["uq_version"]`` from the bundle and falls back
                to ``"v0"`` if not set.
            refresh_proximity (bool): V1-only. If False (default), use the
                proximity backend embedded in the model artifact at training
                time. If True, build a fresh FingerprintProximity from the
                current source FeatureSet. Ignored for V0.
            radius (int): Morgan fingerprint radius (V1 + refresh_proximity only).
            n_bits (int): Fingerprint bit width (V1 + refresh_proximity only).

        Returns:
            A ready-to-use ``UQModelV0`` or ``UQModelV1`` for inference (``.predict``).

        Raises:
            FileNotFoundError: If the requested version's artifact is not in the bundle.
        """
        return uq_model_local(
            self,
            version=version,
            refresh_proximity=refresh_proximity,
            radius=radius,
            n_bits=n_bits,
        )

    def noise_model(self) -> NoiseModel:
        """Create a local Noise Model for this Model

        Returns:
            NoiseModel: A local Noise Model
        """
        return noise_model_local(self)

    def cleanlab_model(self) -> CleanlabModels:
        """Create a CleanlabModels instance for this Model's training data.

        Returns:
            CleanlabModels: Factory providing access to CleanLearning and Datalab models. Use get_label_issues() to get
                a DataFrame with id_column, label_quality, predicted_label, given_label, is_label_issue.
        """
        return cleanlab_model_local(self)

cleanlab_model()

Create a CleanlabModels instance for this Model's training data.

Returns:

Name Type Description
CleanlabModels CleanlabModels

Factory providing access to CleanLearning and Datalab models. Use get_label_issues() to get a DataFrame with id_column, label_quality, predicted_label, given_label, is_label_issue.
Source code in src/workbench/api/model.py 
239
240
241
242
243
244
245
246
def cleanlab_model(self) -> CleanlabModels:
    """Create a CleanlabModels instance for this Model's training data.

    Returns:
        CleanlabModels: Factory providing access to CleanLearning and Datalab models. Use get_label_issues() to get
            a DataFrame with id_column, label_quality, predicted_label, given_label, is_label_issue.
    """
    return cleanlab_model_local(self)

details(**kwargs)

Retrieve the Model Details.

Returns:

Name Type Description
dict dict

A dictionary of details about the Model
Source code in src/workbench/api/model.py 
46
47
48
49
50
51
52
def details(self, **kwargs) -> dict:
    """Retrieve the Model Details.

    Returns:
        dict: A dictionary of details about the Model
    """
    return super().details(**kwargs)

noise_model()

Create a local Noise Model for this Model

Returns:

Name Type Description
NoiseModel NoiseModel

A local Noise Model
Source code in src/workbench/api/model.py 
231
232
233
234
235
236
237
def noise_model(self) -> NoiseModel:
    """Create a local Noise Model for this Model

    Returns:
        NoiseModel: A local Noise Model
    """
    return noise_model_local(self)

prox(space)

Return a proximity model for this Model — precomputed if it has one, else fresh.

Precomputed-first: returns the proximity the model already carries for space (frozen at training time). If it has none, builds one fresh over the model's FeatureSet — using the model's own features/target — and logs that it did so. Cached per space on this instance.

Returns None for space="features" when the model's features include smiles or fingerprint — that's a structure model, so feature-space proximity is meaningless; use prox("fingerprint") instead.

Parameters:

Name Type Description Default
space str

"fingerprint" or "features".

 required

Returns:

Type Description
'Optional[Union[FingerprintProximity, FeatureSpaceProximity]]'

A FingerprintProximity or FeatureSpaceProximity (or None — see above).
Source code in src/workbench/api/model.py 
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
def prox(self, space: str) -> "Optional[Union[FingerprintProximity, FeatureSpaceProximity]]":  # noqa: F821
    """Return a proximity model for this Model — precomputed if it has one, else fresh.

    Precomputed-first: returns the proximity the model already carries for ``space``
    (frozen at training time). If it has none, builds one fresh over the model's
    FeatureSet — using the model's own features/target — and logs that it did so.
    Cached per ``space`` on this instance.

    Returns ``None`` for ``space="features"`` when the model's features include
    ``smiles`` or ``fingerprint`` — that's a structure model, so feature-space
    proximity is meaningless; use ``prox("fingerprint")`` instead.

    Args:
        space: ``"fingerprint"`` or ``"features"``.

    Returns:
        A FingerprintProximity or FeatureSpaceProximity (or None — see above).
    """
    if space not in ("fingerprint", "features"):
        raise ValueError(f"space must be 'fingerprint' or 'features', got {space!r}")

    structure_cols = {"smiles", "fingerprint"}.intersection(f.lower() for f in (self.features() or []))
    if space == "features" and structure_cols:
        self.log.important(
            f"{self.name}: features are structural ({structure_cols}) — "
            "use prox('fingerprint') for structural neighbors."
        )
        return None

    if not hasattr(self, "_prox_cache"):
        self._prox_cache = {}
    if space not in self._prox_cache:
        prox = precomputed_model_proximity(self, space)
        if prox is None:
            from workbench.api import FeatureSet

            self.log.important(
                f"No precomputed {space} proximity for {self.name}; building fresh from its FeatureSet..."
            )
            fs = FeatureSet(self.get_input())
            if space == "features":
                prox = fs.prox("features", feature_list=self.features(), target=self.target())
            else:
                prox = fs.prox("fingerprint", target=self.target())
        self._prox_cache[space] = prox
    return self._prox_cache[space]

to_endpoint(name=None, tags=None, serverless=True, mem_size=2048, max_concurrency=2, instance=None, data_capture=False, async_endpoint=False, min_instances=0, max_instances=None, auto_scaling_mode=None, scale_in_idle_minutes=None, async_max_concurrent=1)

Create an Endpoint from the Model.

Parameters:

Name Type Description Default
name str

Set the name for the endpoint. If not specified, an automatic name will be generated

 None
tags list

Set the tags for the endpoint. If not specified automatic tags will be generated.

 None
serverless bool

Set the endpoint to be serverless (default: True)

 True
mem_size int

The memory size for the Endpoint in MB (default: 2048)

 2048
max_concurrency int

The maximum concurrency for the Endpoint (default: 2)

 2
instance str

The instance type for Realtime Endpoints (default: None = auto-select based on model)

 None
data_capture bool

Enable data capture for the Endpoint (default: False)

 False
async_endpoint bool

Deploy as an async endpoint (default: False). Async endpoints are conceptually batch endpoints — support up to 60-minute per-invocation timeouts, use S3 for I/O, and scale 0 → max_instances → 0 as traffic arrives and drains.

 False
min_instances int

Autoscaler floor (default: 0 — scale to zero).

 0
max_instances int

Autoscaler upper bound for async endpoints (default: None = use the 8-instance default in register_autoscaling).

 None
auto_scaling_mode str

Autoscaling strategy for async endpoints. Defaults to "batch" (step scaling — instant jump 0 → max on traffic, fast drain to min when idle). Other modes may be added in the future.

 None
scale_in_idle_minutes int

Batch-mode only — minutes of empty queue before scaling in to min_instances (default: None = register_autoscaling's default of 15).

 None
async_max_concurrent int

MaxConcurrentInvocationsPerInstance for async endpoints (default: 1, suits CPU-bound predict_fn). Bump for I/O-bound workloads where multiple invocations can share a single instance.

 1

All deploy-time sizing/autoscaling args are persisted to the endpoint's workbench_meta so later reconstructions can see what it was deployed with.

Returns:

Name Type Description
Endpoint Endpoint

The Endpoint (auto-routes to the async transport when
Endpoint

async_endpoint=True).
Source code in src/workbench/api/model.py 
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
def to_endpoint(
    self,
    name: str = None,
    tags: list = None,
    serverless: bool = True,
    mem_size: int = 2048,
    max_concurrency: int = 2,
    instance: str = None,
    data_capture: bool = False,
    async_endpoint: bool = False,
    min_instances: int = 0,
    max_instances: int = None,
    auto_scaling_mode: str = None,
    scale_in_idle_minutes: int = None,
    async_max_concurrent: int = 1,
) -> Endpoint:
    """Create an Endpoint from the Model.

    Args:
        name (str): Set the name for the endpoint. If not specified, an automatic name will be generated
        tags (list): Set the tags for the endpoint. If not specified automatic tags will be generated.
        serverless (bool): Set the endpoint to be serverless (default: True)
        mem_size (int): The memory size for the Endpoint in MB (default: 2048)
        max_concurrency (int): The maximum concurrency for the Endpoint (default: 2)
        instance (str): The instance type for Realtime Endpoints (default: None = auto-select based on model)
        data_capture (bool): Enable data capture for the Endpoint (default: False)
        async_endpoint (bool): Deploy as an async endpoint (default: False). Async
            endpoints are conceptually *batch* endpoints — support up to 60-minute
            per-invocation timeouts, use S3 for I/O, and scale 0 → max_instances →
            0 as traffic arrives and drains.
        min_instances (int): Autoscaler floor (default: 0 — scale to zero).
        max_instances (int): Autoscaler upper bound for async endpoints (default:
            None = use the 8-instance default in register_autoscaling).
        auto_scaling_mode (str): Autoscaling strategy for async endpoints.
            Defaults to "batch" (step scaling — instant jump 0 → max on traffic,
            fast drain to min when idle). Other modes may be added in the future.
        scale_in_idle_minutes (int): Batch-mode only — minutes of empty queue
            before scaling in to min_instances (default: None = register_autoscaling's
            default of 15).
        async_max_concurrent (int): MaxConcurrentInvocationsPerInstance for async
            endpoints (default: 1, suits CPU-bound predict_fn). Bump for I/O-bound
            workloads where multiple invocations can share a single instance.

    All deploy-time sizing/autoscaling args are persisted to the endpoint's
    workbench_meta so later reconstructions can see what it was deployed with.

    Returns:
        Endpoint: The Endpoint (auto-routes to the async transport when
        ``async_endpoint=True``).
    """

    # Ensure the endpoint_name is valid
    if name:
        Artifact.is_name_valid(name, delimiter="-", lower_case=False)

    # If the endpoint_name wasn't given generate it
    else:
        name = self.name.replace("_features", "") + ""
        name = Artifact.generate_valid_name(name, delimiter="-")

    # Create the Endpoint Tags
    tags = [name] if tags is None else tags

    # Create an Endpoint from the Model
    model_to_endpoint = ModelToEndpoint(
        self.name,
        name,
        serverless=serverless,
        instance=instance,
        async_endpoint=async_endpoint,
        min_instances=min_instances,
        max_instances=max_instances,
        auto_scaling_mode=auto_scaling_mode,
        scale_in_idle_minutes=scale_in_idle_minutes,
        async_max_concurrent=async_max_concurrent,
    )
    model_to_endpoint.set_output_tags(tags)
    model_to_endpoint.transform(
        mem_size=mem_size,
        max_concurrency=max_concurrency,
        data_capture=data_capture,
    )

    end = Endpoint(name)
    end.set_owner(self.get_owner())
    return end

uq_model(version=None, refresh_proximity=False, radius=2, n_bits=4096)

Load this model's fitted UQ model (V0 or V1) for uncertainty-quantified inference.

Usage

model = Model("my-model") uq = model.uq_model() # bundle-default version uq_v0 = model.uq_model(version="v0") # original isotonic calibrator uq_v1 = model.uq_model(version="v1") # proximity-augmented RF out = uq.predict(test_df[["smiles"]], predictions, prediction_std)

Parameters:

Name Type Description Default
version str | None

Which UQ version to load — "v0" (original isotonic-on-(prediction, std) calibrator) or "v1" (proximity- augmented RandomForest error model). If None, reads hyperparameters["uq_version"] from the bundle and falls back to "v0" if not set.

 None
refresh_proximity bool

V1-only. If False (default), use the proximity backend embedded in the model artifact at training time. If True, build a fresh FingerprintProximity from the current source FeatureSet. Ignored for V0.

 False
radius int

Morgan fingerprint radius (V1 + refresh_proximity only).

 2
n_bits int

Fingerprint bit width (V1 + refresh_proximity only).

 4096

Returns:

Type Description
'UQModelV0 | UQModelV1'

A ready-to-use UQModelV0 or UQModelV1 for inference (.predict).

Raises:

Type Description
FileNotFoundError

If the requested version's artifact is not in the bundle.
Source code in src/workbench/api/model.py 
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
def uq_model(
    self,
    version: str = None,
    refresh_proximity: bool = False,
    radius: int = 2,
    n_bits: int = 4096,
) -> "UQModelV0 | UQModelV1":  # noqa: F821
    """Load this model's fitted UQ model (V0 or V1) for uncertainty-quantified inference.

    Usage:
        model = Model("my-model")
        uq = model.uq_model()                    # bundle-default version
        uq_v0 = model.uq_model(version="v0")     # original isotonic calibrator
        uq_v1 = model.uq_model(version="v1")     # proximity-augmented RF
        out = uq.predict(test_df[["smiles"]], predictions, prediction_std)

    Args:
        version (str | None): Which UQ version to load — ``"v0"`` (original
            isotonic-on-(prediction, std) calibrator) or ``"v1"`` (proximity-
            augmented RandomForest error model). If ``None``, reads
            ``hyperparameters["uq_version"]`` from the bundle and falls back
            to ``"v0"`` if not set.
        refresh_proximity (bool): V1-only. If False (default), use the
            proximity backend embedded in the model artifact at training
            time. If True, build a fresh FingerprintProximity from the
            current source FeatureSet. Ignored for V0.
        radius (int): Morgan fingerprint radius (V1 + refresh_proximity only).
        n_bits (int): Fingerprint bit width (V1 + refresh_proximity only).

    Returns:
        A ready-to-use ``UQModelV0`` or ``UQModelV1`` for inference (``.predict``).

    Raises:
        FileNotFoundError: If the requested version's artifact is not in the bundle.
    """
    return uq_model_local(
        self,
        version=version,
        refresh_proximity=refresh_proximity,
        radius=radius,
        n_bits=n_bits,
    )

ModelFramework

Bases: Enum

Enumerated Types for Workbench Model Frameworks

Source code in src/workbench/core/artifacts/model_core.py 
50
51
52
53
54
55
56
57
58
59
class ModelFramework(Enum):
    """Enumerated Types for Workbench Model Frameworks"""

    SKLEARN = "sklearn"
    XGBOOST = "xgboost"
    PYTORCH = "pytorch"
    CHEMPROP = "chemprop"
    TRANSFORMER = "transformer"
    META = "meta"
    UNKNOWN = "unknown"

ModelType

Bases: Enum

Enumerated Types for Workbench Model Types

Source code in src/workbench/core/artifacts/model_core.py 
36
37
38
39
40
41
42
43
44
45
46
47
class ModelType(Enum):
    """Enumerated Types for Workbench Model Types"""

    CLASSIFIER = "classifier"
    REGRESSOR = "regressor"
    CLUSTERER = "clusterer"
    PROXIMITY = "proximity"
    PROJECTION = "projection"
    UQ_REGRESSOR = "uq_regressor"
    ENSEMBLE_REGRESSOR = "ensemble_regressor"
    TRANSFORMER = "transformer"
    UNKNOWN = "unknown"

Examples

All of the Workbench Examples are in the Workbench Repository under the examples/ directory. For a full code listing of any example please visit our Workbench Examples

Create a Model from a FeatureSet

featureset_to_model.py
from workbench.api.feature_set import FeatureSet
from workbench.api.model import ModelType
from pprint import pprint

# Grab a FeatureSet
my_features = FeatureSet("test_features")

# Create a Model from the FeatureSet
# Note: ModelTypes can be CLASSIFIER, REGRESSOR (XGBoost is default)
my_model = my_features.to_model(name="test-model",
                                model_type=ModelType.REGRESSOR, 
                                target_column="iq_score")
pprint(my_model.details())

Output

{'approval_status': 'Approved',
 'content_types': ['text/csv'],
 ...
 'inference_types': ['ml.t2.medium'],
 'input': 'test_features',
 'model_metrics':   metric_name  value
                0        RMSE  7.924
                1         MAE  6.554,
                2          R2  0.604,
 'regression_predictions':       iq_score  prediction
                            0   136.519012  139.964460
                            1   133.616974  130.819950
                            2   122.495415  124.967834
                            3   133.279510  121.010284
                            4   127.881073  113.825005
    ...
 'response_types': ['text/csv'],
 'workbench_tags': ['test-model'],
 'shapley_values': None,
 'size': 0.0,
 'status': 'Completed',
 'transform_types': ['ml.m5.large'],
 'name': 'test-model',
 'version': 1}

Use a specific Scikit-Learn Model

featureset_to_knn.py
from workbench.api.feature_set import FeatureSet
from pprint import pprint

# Grab a FeatureSet
my_features = FeatureSet("abalone_features")

# Transform FeatureSet into KNN Regression Model
# Note: model_class can be any sckit-learn model 
#  "KNeighborsRegressor", "BayesianRidge",
#  "GaussianNB", "AdaBoostClassifier", etc
my_model = my_features.to_model(
    model_class="KNeighborsRegressor",
    model_import_str="from sklearn.neighbors import KNeighborsRegressor",
    target_column="class_number_of_rings",
    name="abalone-knn-reg",
    description="Abalone KNN Regression",
    tags=["abalone", "knn"],
)
pprint(my_model.details())
Another Scikit-Learn Example

featureset_to_rfc.py
from workbench.api.feature_set import FeatureSet
from pprint import pprint

# Grab a FeatureSet
my_features = FeatureSet("wine_features")

# Using a Scikit-Learn Model
# Note: model_class can be any sckit-learn model ("KMeans", "BayesianRidge",
#       "GaussianNB", "AdaBoostClassifier", "Ridge, "Lasso", "SVC", "SVR", etc...)
my_model = my_features.to_model(
    model_class="RandomForestClassifier",
    model_import_str="from sklearn.ensemble import RandomForestClassifier",
    target_column="wine_class",
    name="wine-rfc-class",
    description="Wine RandomForest Classification",
    tags=["wine", "rfc"]
)
pprint(my_model.details())

Create an Endpoint from a Model

Endpoint Costs

Serverless endpoints are a great option, they have no AWS charges when not running. A realtime endpoint has less latency (no cold start) but AWS charges an hourly fee which can add up quickly!

model_to_endpoint.py
from workbench.api.model import Model

# Grab the abalone regression Model
model = Model("abalone-regression")

# By default, an Endpoint is serverless, you can
# make a realtime endpoint with serverless=False
model.to_endpoint(name="abalone-regression-end",
                  tags=["abalone", "regression"],
                  serverless=True)

Model Health Check and Metrics

model_metrics.py
from workbench.api.model import Model

# Grab the abalone-regression Model
model = Model("abalone-regression")

# Perform a health check on the model
# Note: The health_check() method returns 'issues' if there are any
#       problems, so if there are no issues, the model is healthy
health_issues = model.health_check()
if not health_issues:
    print("Model is Healthy")
else:
    print("Model has issues")
    print(health_issues)

# Get the model metrics and regression predictions
print(model.model_metrics())
print(model.regression_predictions())

Output

Model is Healthy
  metric_name  value
0        RMSE  2.190
1         MAE  1.544
2          R2  0.504

     class_number_of_rings  prediction
0                        9    8.648378
1                       11    9.717787
2                       11   10.933070
3                       10    9.899738
4                        9   10.014504
..                     ...         ...
495                     10   10.261657
496                      9   10.788254
497                     13    7.779886
498                     12   14.718514
499                     13   10.637320

Workbench UI

Running these few lines of code creates an AWS Model Package Group and an AWS Model Package. These model artifacts can be viewed in the Sagemaker Console/Notebook interfaces or in the Workbench Dashboard UI.

workbench_new_light
Workbench Dashboard: Models

Not Finding a particular method?

The Workbench API Classes use the 'Core' Classes Internally, so for an extensive listing of all the methods available please take a deep dive into: Workbench Core Classes