Endpoint

Endpoint Examples

Examples of using the Endpoint class are listed at the bottom of this page Examples.

Endpoint: Manages AWS Endpoint creation and deployment. Endpoints are automatically set up and provisioned for deployment into AWS. Endpoints can be viewed in the AWS Sagemaker interfaces or in the Workbench Dashboard UI, which provides additional model details and performance metrics

`Endpoint`

Bases: EndpointCore

Endpoint: Workbench Endpoint API Class

Common Usage

my_endpoint = Endpoint(name)
my_endpoint.details()
my_endpoint.inference(eval_df)

Source code in src/workbench/api/endpoint.py

class Endpoint(EndpointCore):
    """Endpoint: Workbench Endpoint API Class

    Common Usage:
        ```python
        my_endpoint = Endpoint(name)
        my_endpoint.details()
        my_endpoint.inference(eval_df)
        ```
    """

    def details(self, **kwargs) -> dict:
        """Endpoint Details

        Returns:
            dict: A dictionary of details about the Endpoint
        """
        return super().details(**kwargs)

    def inference(self, eval_df: pd.DataFrame, capture_uuid: str = None, id_column: str = None) -> pd.DataFrame:
        """Run inference on the Endpoint using the provided DataFrame

        Args:
            eval_df (pd.DataFrame): The DataFrame to run predictions on
            capture_uuid (str, optional): The UUID of the capture to use (default: None)
            id_column (str, optional): The name of the column to use as the ID (default: None)

        Returns:
            pd.DataFrame: The DataFrame with predictions
        """
        return super().inference(eval_df, capture_uuid, id_column)

    def auto_inference(self, capture: bool = False) -> pd.DataFrame:
        """Run inference on the Endpoint using the FeatureSet evaluation data

        Args:
            capture (bool): Capture the inference results

        Returns:
            pd.DataFrame: The DataFrame with predictions
        """
        return super().auto_inference(capture)

    def fast_inference(self, eval_df: pd.DataFrame) -> pd.DataFrame:
        """Run inference on the Endpoint using the provided DataFrame

        Args:
            eval_df (pd.DataFrame): The DataFrame to run predictions on

        Returns:
            pd.DataFrame: The DataFrame with predictions

        Note:
            There's no sanity checks or error handling... just FAST Inference!
        """
        return super().fast_inference(eval_df)

`auto_inference(capture=False)`

Run inference on the Endpoint using the FeatureSet evaluation data

Parameters:

Name	Type	Description	Default
`capture`	`bool`	Capture the inference results	`False`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: The DataFrame with predictions

Source code in src/workbench/api/endpoint.py

def auto_inference(self, capture: bool = False) -> pd.DataFrame:
    """Run inference on the Endpoint using the FeatureSet evaluation data

    Args:
        capture (bool): Capture the inference results

    Returns:
        pd.DataFrame: The DataFrame with predictions
    """
    return super().auto_inference(capture)

`details(**kwargs)`

Endpoint Details

Returns:

Name	Type	Description
`dict`	`dict`	A dictionary of details about the Endpoint

Source code in src/workbench/api/endpoint.py

def details(self, **kwargs) -> dict:
    """Endpoint Details

    Returns:
        dict: A dictionary of details about the Endpoint
    """
    return super().details(**kwargs)

`fast_inference(eval_df)`

Run inference on the Endpoint using the provided DataFrame

Parameters:

Name	Type	Description	Default
`eval_df`	`DataFrame`	The DataFrame to run predictions on	required

Returns:

Type	Description
`DataFrame`	pd.DataFrame: The DataFrame with predictions

Note

There's no sanity checks or error handling... just FAST Inference!

Source code in src/workbench/api/endpoint.py

def fast_inference(self, eval_df: pd.DataFrame) -> pd.DataFrame:
    """Run inference on the Endpoint using the provided DataFrame

    Args:
        eval_df (pd.DataFrame): The DataFrame to run predictions on

    Returns:
        pd.DataFrame: The DataFrame with predictions

    Note:
        There's no sanity checks or error handling... just FAST Inference!
    """
    return super().fast_inference(eval_df)

`inference(eval_df, capture_uuid=None, id_column=None)`

Run inference on the Endpoint using the provided DataFrame

Parameters:

Name	Type	Description	Default
`eval_df`	`DataFrame`	The DataFrame to run predictions on	required
`capture_uuid`	`str`	The UUID of the capture to use (default: None)	`None`
`id_column`	`str`	The name of the column to use as the ID (default: None)	`None`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: The DataFrame with predictions

Source code in src/workbench/api/endpoint.py

def inference(self, eval_df: pd.DataFrame, capture_uuid: str = None, id_column: str = None) -> pd.DataFrame:
    """Run inference on the Endpoint using the provided DataFrame

    Args:
        eval_df (pd.DataFrame): The DataFrame to run predictions on
        capture_uuid (str, optional): The UUID of the capture to use (default: None)
        id_column (str, optional): The name of the column to use as the ID (default: None)

    Returns:
        pd.DataFrame: The DataFrame with predictions
    """
    return super().inference(eval_df, capture_uuid, id_column)

Examples

Run Inference on an Endpoint

endpoint_inference.py

from workbench.api.feature_set import FeatureSet
from workbench.api.model import Model
from workbench.api.endpoint import Endpoint

# Grab an existing Endpoint
endpoint = Endpoint("abalone-regression-end")

# Workbench has full ML Pipeline provenance, so we can backtrack the inputs,
# get a DataFrame of data (not used for training) and run inference
model = Model(endpoint.get_input())
fs = FeatureSet(model.get_input())
athena_table = fs.view("training").table
df = fs.query(f"SELECT * FROM {athena_table} where training = FALSE")

# Run inference/predictions on the Endpoint
results_df = endpoint.inference(df)

# Run inference/predictions and capture the results
results_df = endpoint.inference(df, capture=True)

# Run inference/predictions using the FeatureSet evaluation data
results_df = endpoint.auto_inference(capture=True)

Output

Processing...
     class_number_of_rings  prediction
0                       13   11.477922
1                       12   12.316887
2                        8    7.612847
3                        8    9.663341
4                        9    9.075263
..                     ...         ...
839                      8    8.069856
840                     15   14.915502
841                     11   10.977605
842                     10   10.173433
843                      7    7.297976

Endpoint Details

The details() method

The detail() method on the Endpoint class provides a lot of useful information. All of the Workbench classes have a details() method try it out!

endpoint_details.py

from workbench.api.endpoint import Endpoint
from pprint import pprint

# Get Endpoint and print out it's details
endpoint = Endpoint("abalone-regression-end")
pprint(endpoint.details())

Output

{
 'input': 'abalone-regression',
 'instance': 'Serverless (2GB/5)',
 'model_metrics':   metric_name  value
            0        RMSE  2.190
            1         MAE  1.544
            2          R2  0.504,
 'model_name': 'abalone-regression',
 'model_type': 'regressor',
 'modified': datetime.datetime(2023, 12, 29, 17, 48, 35, 115000, tzinfo=datetime.timezone.utc),
     class_number_of_rings  prediction
0                        9    8.648378
1                       11    9.717787
2                       11   10.933070
3                       10    9.899738
4                        9   10.014504
..                     ...         ...
495                     10   10.261657
496                      9   10.788254
497                     13    7.779886
498                     12   14.718514
499                     13   10.637320
 'workbench_tags': ['abalone', 'regression'],
 'status': 'InService',
 'uuid': 'abalone-regression-end',
 'variant': 'AllTraffic'}

Endpoint Metrics

endpoint_metrics.py

from workbench.api.endpoint import Endpoint

# Grab an existing Endpoint
endpoint = Endpoint("abalone-regression-end")

# Workbench tracks both Model performance and Endpoint Metrics
model_metrics = endpoint.details()["model_metrics"]
endpoint_metrics = endpoint.endpoint_metrics()
print(model_metrics)
print(endpoint_metrics)

Output

  metric_name  value
0        RMSE  2.190
1         MAE  1.544
2          R2  0.504

    Invocations  ModelLatency  OverheadLatency  ModelSetupTime  Invocation5XXErrors
29          0.0          0.00             0.00            0.00                  0.0
30          1.0          1.11            23.73           23.34                  0.0
31          0.0          0.00             0.00            0.00                  0.0
48          0.0          0.00             0.00            0.00                  0.0
49          5.0          0.45             9.64           23.57                  0.0
50          2.0          0.57             0.08            0.00                  0.0
51          0.0          0.00             0.00            0.00                  0.0
60          4.0          0.33             5.80           22.65                  0.0
61          1.0          1.11            23.35           23.10                  0.0
62          0.0          0.00             0.00            0.00                  0.0
...

Workbench UI

Running these few lines of code creates and deploys an AWS Endpoint. The Endpoint artifacts can be viewed in the Sagemaker Console/Notebook interfaces or in the Workbench Dashboard UI. Workbench will monitor the endpoint, plot invocations, latencies, and tracks error metrics.

workbench_endpoints — Workbench Dashboard: Endpoints

Not Finding a particular method?

The Workbench API Classes use the 'Core' Classes Internally, so for an extensive listing of all the methods available please take a deep dive into: Workbench Core Classes