Skip to content

Release 0.8.11

Need Help?

The SuperCowPowers team is happy to give any assistance needed when setting up AWS and SageWorks. So please contact us at sageworks@supercowpowers.com or on chat us up on Discord

The SageWorks framework continues to flex to support different real world use cases when operating a set of production machine learning pipelines.

Note: These release notes cover all the changes from 0.8.8 to 0.8.11

General

The AWSAccountClamp had too many responsibilities so that class has been split up into two classes and a set of utilities:

  • AWSAccountClamp
  • AWSSession
  • utils/execution_environment.py

API Changes

For all/most of these API changes they include both DataSources and FeatureSets. We're using a FeatureSet (fs) in the examples below but also applies to DataSoources.

  • Column Names/Table Names

    fs.column_names() -> fs.columns
fs.get_table_name() -> fs.table_name

  • Display/Training/Computation Views

    In general methods for FS/DS are now part of the View API, here's a change list:

    fs.get_display_view() -> fs.view("display")
fs.get_training_view() -> fs.view("training")
fs.get_display_columns() -> fs.view("display").columns
fs.get_computation_columns() -> fs.view("computation").columns
fs.get_training_view_table() -> fs.view("training").table_name
fs.get_training_data(self) -> fs.view("training").pull_dataframe()

    Some FS/DS methods have also been removed

    num_display_columns() -> gone num_computation_columns() -> gone

  • Views: Methods that we're Keeping

    We're keeping the methods below since they handle some underlying mechanics and serve as nice convenience methods. 

    ds/fs.set_display_columns()
ds/fs.set_computation_columns()

  • AWSAccountClamp

    AWSAccountClamp().boto_session() --> AWSAccountClamp().boto3_session

  • All Classes

    If the class previously had a boto_session attribute that has been renamed to boto3_session

Glue Job Fixes

For sageworks==0.8.8 you needed to be careful about when/where you set your config/ENV vars. With >=0.8.9 you can now use the typical setup like this:

```
from sageworks.utils.config_manager import ConfigManager

# Set the SageWorks Config
cm = ConfigManager()
cm.set_config("SAGEWORKS_BUCKET", args_dict["sageworks-bucket"])
cm.set_config("REDIS_HOST", args_dict["redis-host"])
```

Robust ModelNotReadyException Handling

AWS will 'deep freeze' Serverless Endpoints and if that endpoint hasn't been used for a while it can sometimes take a long time to come up and be ready for inference. SageWorks now properly manages this AWS error condition, it will report the issue, wait 60 seconds, and try again 5 times before raising the exception.

(endpoint_core.py:502) ERROR Endpoint model not ready
(endpoint_core.py:503) ERROR Waiting and Retrying...
...
After a while, inference will run successfully :)

Questions?

The SuperCowPowers team is happy to answer any questions you may have about AWS and SageWorks. Please contact us at sageworks@supercowpowers.com or on chat us up on Discord