Skip to content

Release 0.8.8

Need Help?

The SuperCowPowers team is happy to give any assistance needed when setting up AWS and Workbench. So please contact us at workbench@supercowpowers.com or on chat us up on Discord

The Workbench framework continues to flex to support different real world use cases when operating a set of production machine learning pipelines.

Additional Functionality

API Changes

  • Auto Inference name change

    When auto_inference is run on an endpoint the name of that inference run is currently training_holdout. That is too close to model_training and is confusing. So we're going to change the name to auto_inference which is way more explanatory and intuitive.

    Porting plugins: There should really not be any hard coding for training_holdout, plugins should just call list_inference_runs() (see below) and use the first one on the list.

  • list_inference_runs()

    The list_inference_runs() method on Models has been improved. It now handles error states better (no model, no model training data) and will return 'model_training' LAST on the list, this should improve UX for plugin components.

Examples

 model = Model("abalone-regression")
 model.list_inference_runs()
 Out[1]: ['auto_inference', 'model_training']

 model = Model("wine-classification")
 model.list_inference_runs()
 Out[2]: ['auto_inference', 'training_holdout', 'model_training']

 model = Model("aqsol-mol-regression")
 model.list_inference_runs()
 Out[3]: ['training_holdout', 'model_training']

 model = Model("empty-model-group")
 model.list_inference_runs()
 Out[4]: []

Glue Job Changes

We're spinning up the CloudWatch Handler much earlier now, so if you're setting config like this:

from workbench.utils.config_manager import ConfigManager

# Set the Workbench Config
cm = ConfigManager()
cm.set_config("WORKBENCH_BUCKET", args_dict["workbench-bucket"])
cm.set_config("REDIS_HOST", args_dict["redis-host"])

Just switch out that code for this code. Note: these need to be set before importing workbench

# Set these ENV vars for Workbench 
os.environ['WORKBENCH_BUCKET'] = args_dict["workbench-bucket"]
os.environ["REDIS_HOST"] = args_dict["redis-host"]

Misc

Confusion Matrix support for 'ordinal' labels

Pandas has an ‘ordinal’ type, so the confusion matrix method endpoint.confusion_matrix() now checks the label column to see if it’s ordinal and uses that order, if not just it will alphabetically sort.

Note: This change may not affect your UI experience. Confusion matricies are saved in the Workbench/S3 meta data storage, so a bunch of stuff upstream will also need to happen. FeatureSet object/api for setting the label order, recreation of the model/endpoint and confustion matrix, etc. In general this is a forwarding looking change that will be useful later. :)

Questions?

The SuperCowPowers team is happy to answer any questions you may have about AWS and Workbench. Please contact us at workbench@supercowpowers.com or on chat us up on Discord