Release 0.8.11
Need Help?
The SuperCowPowers team is happy to give any assistance needed when setting up AWS and SageWorks. So please contact us at sageworks@supercowpowers.com or on chat us up on Discord
The SageWorks framework continues to 'flex' to support different real world use cases when operating a set of production machine learning pipelines.
Note: These release notes cover all the changes from 0.8.8
to 0.8.11
General
The AWSAccountClamp had too many responsibilities so that class has been split up into two classes and a set of utilities:
- AWSAccountClamp
- AWSSession
- utils/execution_environment.py
API Changes
For all/most of these API changes they include both DataSources and FeatureSets. We're using a FeatureSet (fs) in the examples below but also applies to DataSoources.
-
Column Names/Table Names
-
Display/Training/Computation Views
In general methods for FS/DS are now part of the View API, here's a change list:
fs.get_display_view() -> fs.view("display") fs.get_training_view() -> fs.view("training") fs.get_display_columns() -> fs.view("display").columns fs.get_computation_columns() -> fs.view("computation").columns fs.get_training_view_table() -> fs.view("training").table_name fs.get_training_data(self) -> fs.view("training").pull_dataframe()
Some FS/DS methods have also been removed
num_display_columns() -> gone num_computation_columns() -> gone
-
Views: Methods that we're Keeping
We're keeping the methods below since they handle some underlying mechanics and serve as nice convenience methods.
-
AWSAccountClamp
-
All Classes
If the class previously had a
boto_session
attribute that has been renamed toboto3_session
Glue Job Fixes
For sageworks==0.8.8
you needed to be careful about when/where you set your config/ENV vars. With >=0.8.9
you can now use the typical setup like this:
```
from sageworks.utils.config_manager import ConfigManager
# Set the SageWorks Config
cm = ConfigManager()
cm.set_config("SAGEWORKS_BUCKET", args_dict["sageworks-bucket"])
cm.set_config("REDIS_HOST", args_dict["redis-host"])
```
Robust ModelNotReadyException Handling
AWS will 'deep freeze' Serverless Endpoints and if that endpoint hasn't been used for a while it can sometimes take a long time to come up and be ready for inference. SageWorks now properly manages this AWS error condition, it will report the issue, wait 60 seconds, and try again 5 times before raising the exception.
(endpoint_core.py:502) ERROR Endpoint model not ready
(endpoint_core.py:503) ERROR Waiting and Retrying...
...
After a while, inference will run successfully :)
Questions?
The SuperCowPowers team is happy to answer any questions you may have about AWS and SageWorks. Please contact us at sageworks@supercowpowers.com or on chat us up on Discord