fix: fix kmeans test deletion sequence, increment lineage statics #2815

mufaddal-rohawala · 2021-12-30T23:47:15Z

Issue #, if available:

Description of changes:

Current failure scenario sequence:

Deploy KMeans Model, deploy, predict.
Delete Endpoint.
Predictor delete_model.
- Checks for endpoint config name by checking endpoint description.
- Endpoint does not exist as already deleted.

>       predictor.delete_model()

tests/integ/test_kmeans.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py38/lib/python3.8/site-packages/sagemaker/predictor.py:339: in delete_model
    current_model_names = self._get_model_names()
.tox/py38/lib/python3.8/site-packages/sagemaker/predictor.py:529: in _get_model_names
    current_endpoint_config_name = self._get_endpoint_config_name()
.tox/py38/lib/python3.8/site-packages/sagemaker/predictor.py:519: in _get_endpoint_config_name
    endpoint_desc = self.sagemaker_session.sagemaker_client.describe_endpoint(
.tox/py38/lib/python3.8/site-packages/botocore/client.py:391: in _api_call
    return self._make_api_call(operation_name, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <botocore.client.SageMaker object at 0x7f69df67e670>
operation_name = 'DescribeEndpoint'
api_params = {'EndpointName': 'kmeans-1640902848-6c75'}

>           raise error_class(parsed_response, operation_name)
E           botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the DescribeEndpoint operation: Could not find endpoint "arn:aws:sagemaker:us-west-2:142577830533:endpoint/kmeans-1640902848-6c75".

Change the sequence to:

Deploy KMeans Model, deploy, predict.
Predictor delete_model.
Delete Endpoint.

Testing done:
Tested in personal account:

mufi@3c22fbe5d54b sagemaker-python-sdk % export IGNORE_COVERAGE=- ; tox -e py38 -- -s -vv tests/integ/test_kmeans.py::test_kmeans ; unset IGNORE_COVERAGE
GLOB sdist-make: /Users/mufi/workspace/sagemaker-python-sdk/setup.py
py38 inst-nodeps: /Users/mufi/workspace/sagemaker-python-sdk/.tox/.tmp/package/7/sagemaker-2.72.2.dev0.zip
py38 installed: alembic==1.7.5,apache-airflow==1.10.11,apispec==1.3.3,argcomplete==1.12.3,attrs==19.3.0,awslogs==0.14.0,Babel==2.9.1,backports.entry-points-selectable==1.1.1,bcrypt==3.2.0,black==21.12b0,boto3==1.20.24,botocore==1.23.24,cached-property==1.5.2,cattrs==1.0.0,certifi==2021.10.8,cffi==1.15.0,charset-normalizer==2.0.9,click==7.1.2,colorama==0.4.4,colorlog==4.0.2,configparser==3.5.3,contextlib2==21.6.0,coverage==6.1.2,croniter==0.3.37,cryptography==36.0.1,defusedxml==0.7.1,dill==0.3.4,distlib==0.3.4,distro==1.6.0,dnspython==2.1.0,docker==5.0.0,docker-compose==1.29.2,dockerpty==0.4.1,docopt==0.6.2,docutils==0.18.1,email-validator==1.1.3,execnet==1.9.0,fabric==2.6.0,filelock==3.4.0,flake8==4.0.1,Flask==1.1.4,Flask-Admin==1.5.4,Flask-AppBuilder==2.3.4,Flask-Babel==1.0.0,Flask-Caching==1.3.3,Flask-JWT-Extended==3.25.1,Flask-Login==0.4.1,Flask-OpenID==1.3.0,Flask-SQLAlchemy==2.5.1,flask-swagger==0.2.13,Flask-WTF==0.14.3,funcsigs==1.0.2,future==0.18.2,google-pasta==0.2.0,graphviz==0.19.1,greenlet==1.1.2,gunicorn==19.10.0,idna==3.3,importlib-metadata==4.8.2,importlib-resources==5.4.0,iniconfig==1.1.1,invoke==1.6.0,iso8601==1.0.2,itsdangerous==1.1.0,Jinja2==2.10.3,jmespath==0.10.0,json-merge-patch==0.2,jsonschema==3.2.0,lazy-object-proxy==1.7.1,lockfile==0.12.2,Mako==1.1.6,Markdown==2.6.11,MarkupSafe==2.0.1,marshmallow==2.21.0,marshmallow-enum==1.5.1,marshmallow-sqlalchemy==0.23.1,mccabe==0.6.1,mock==4.0.3,more-itertools==8.12.0,multiprocess==0.70.12.2,mypy-extensions==0.4.3,natsort==8.0.2,numpy==1.21.4,packaging==21.3,pandas==1.3.5,paramiko==2.8.1,pathlib2==2.3.6,pathos==0.2.8,pathspec==0.9.0,pendulum==1.4.4,platformdirs==2.4.0,pluggy==0.13.1,pox==0.3.0,ppft==1.6.6.4,prison==0.2.1,protobuf==3.19.1,protobuf3-to-dict==0.1.5,psutil==5.8.0,py==1.11.0,pycodestyle==2.8.0,pycparser==2.21,pyflakes==2.4.0,Pygments==2.10.0,PyJWT==1.7.1,PyNaCl==1.4.0,pyparsing==3.0.6,pyrsistent==0.18.0,pytest==6.0.2,pytest-cov==3.0.0,pytest-forked==1.4.0,pytest-rerunfailures==10.2,pytest-timeout==2.0.2,pytest-xdist==2.4.0,python-daemon==2.3.0,python-dateutil==2.8.2,python-dotenv==0.19.2,python-nvd3==0.15.0,python-slugify==4.0.1,python3-openid==3.2.0,pytz==2021.3,pytzdata==2020.1,PyYAML==5.4.1,requests==2.26.0,s3transfer==0.5.0,sagemaker @ file:///Users/mufi/workspace/sagemaker-python-sdk/.tox/.tmp/package/7/sagemaker-2.72.2.dev0.zip,sagemaker-experiments==0.1.35,scipy==1.7.3,setproctitle==1.2.2,six==1.16.0,smdebug-rulesconfig==1.0.1,SQLAlchemy==1.4.28,SQLAlchemy-JSONField==0.9.0,SQLAlchemy-Utils==0.37.9,stopit==1.1.2,tabulate==0.8.9,tenacity==4.12.0,termcolor==1.1.0,text-unidecode==1.3,texttable==1.6.4,thrift==0.15.0,toml==0.10.2,tomli==1.2.3,tox==3.24.4,typing_extensions==4.0.1,tzlocal==1.5.1,unicodecsv==0.14.1,urllib3==1.26.7,virtualenv==20.10.0,websocket-client==0.59.0,Werkzeug==0.16.1,WTForms==3.0.0,zipp==3.6.0,zope.deprecation==4.4.0
py38 run-test-pre: PYTHONHASHSEED='2335144008'
py38 run-test: commands[0] | python -c 'import os; os.system('"'"'install-custom-pkgs --install-boto-wheels'"'"')'
sh: install-custom-pkgs: command not found
py38 run-test: commands[1] | pytest --cov=sagemaker --cov-append -s -vv tests/integ/test_kmeans.py::test_kmeans
================================================================================================= test session starts ==================================================================================================
platform darwin -- Python 3.8.8, pytest-6.0.2, py-1.11.0, pluggy-0.13.1 -- /Users/mufi/workspace/sagemaker-python-sdk/.tox/py38/bin/python
cachedir: .tox/py38/.pytest_cache
rootdir: /Users/mufi/workspace/sagemaker-python-sdk, configfile: tox.ini
plugins: forked-1.4.0, timeout-2.0.2, rerunfailures-10.2, xdist-2.4.0, cov-3.0.0
collected 1 item                                                                                                                                                                                                       

tests/integ/test_kmeans.py::test_kmeans 2021-12-30 22:26:34 Starting - Starting the training job...
2021-12-30 22:26:57 Starting - Launching requested ML instancesProfilerReport-1640903192: InProgress
......
2021-12-30 22:27:58 Starting - Preparing the instances for training.........
2021-12-30 22:29:43 Downloading - Downloading input data
2021-12-30 22:29:43 Training - Downloading the training image...
2021-12-30 22:30:18 Uploading - Uploading generated training model
2021-12-30 22:30:18 Completed - Training job completed
Docker entrypoint called with argument(s): train
Running default environment configuration script
[12/30/2021 22:30:07 INFO 140709267486528] Reading default configuration from /opt/amazon/lib/python3.7/site-packages/algorithm/resources/default-input.json: {'init_method': 'random', 'mini_batch_size': '5000', 'epochs': '1', 'extra_center_factor': 'auto', 'local_lloyd_max_iter': '300', 'local_lloyd_tol': '0.0001', 'local_lloyd_init_method': 'kmeans++', 'local_lloyd_num_trials': 'auto', 'half_life_time_size': '0', 'eval_metrics': '["msd"]', 'force_dense': 'true', '_disable_wait_to_read': 'false', '_enable_profiler': 'false', '_kvstore': 'auto', '_log_level': 'info', '_num_gpus': 'auto', '_num_kv_servers': '1', '_num_slices': '1', '_tuning_objective_metric': ''}
[12/30/2021 22:30:07 INFO 140709267486528] Merging with provided configuration from /opt/ml/input/config/hyperparameters.json: {'local_lloyd_init_method': 'kmeans++', 'feature_dim': '784', 'local_lloyd_max_iter': '1', 'local_lloyd_num_trials': '1', 'local_lloyd_tol': '1.0', 'extra_center_factor': '1', 'eval_metrics': '["ssd", "msd"]', 'init_method': 'random', 'k': '10', 'force_dense': 'True', 'epochs': '1', 'half_life_time_size': '1'}
[12/30/2021 22:30:07 INFO 140709267486528] Final configuration: {'init_method': 'random', 'mini_batch_size': '5000', 'epochs': '1', 'extra_center_factor': '1', 'local_lloyd_max_iter': '1', 'local_lloyd_tol': '1.0', 'local_lloyd_init_method': 'kmeans++', 'local_lloyd_num_trials': '1', 'half_life_time_size': '1', 'eval_metrics': '["ssd", "msd"]', 'force_dense': 'True', '_disable_wait_to_read': 'false', '_enable_profiler': 'false', '_kvstore': 'auto', '_log_level': 'info', '_num_gpus': 'auto', '_num_kv_servers': '1', '_num_slices': '1', '_tuning_objective_metric': '', 'feature_dim': '784', 'k': '10'}
[12/30/2021 22:30:07 WARNING 140709267486528] Loggers have already been setup.
Process 1 is a worker.
[12/30/2021 22:30:07 INFO 140709267486528] Using default worker.
[12/30/2021 22:30:07 INFO 140709267486528] Loaded iterator creator application/x-recordio-protobuf for content type ('application/x-recordio-protobuf', '1.0')
[12/30/2021 22:30:07 INFO 140709267486528] Create Store: local
[12/30/2021 22:30:07 INFO 140709267486528] nvidia-smi: took 0.031 seconds to run.
[12/30/2021 22:30:07 INFO 140709267486528] nvidia-smi identified 0 GPUs.
[12/30/2021 22:30:07 INFO 140709267486528] Number of GPUs being used: 0
[12/30/2021 22:30:07 INFO 140709267486528] Checkpoint loading and saving are disabled.
[12/30/2021 22:30:07 INFO 140709267486528] Setting up with params: {'init_method': 'random', 'mini_batch_size': '5000', 'epochs': '1', 'extra_center_factor': '1', 'local_lloyd_max_iter': '1', 'local_lloyd_tol': '1.0', 'local_lloyd_init_method': 'kmeans++', 'local_lloyd_num_trials': '1', 'half_life_time_size': '1', 'eval_metrics': '["ssd", "msd"]', 'force_dense': 'True', '_disable_wait_to_read': 'false', '_enable_profiler': 'false', '_kvstore': 'auto', '_log_level': 'info', '_num_gpus': 'auto', '_num_kv_servers': '1', '_num_slices': '1', '_tuning_objective_metric': '', 'feature_dim': '784', 'k': '10'}
[12/30/2021 22:30:07 INFO 140709267486528] Number of GPUs being used: 0
[12/30/2021 22:30:07 INFO 140709267486528] number of center slices 1
[12/30/2021 22:30:07 WARNING 140709267486528] Batch size 5000 is bigger than the first batch data. Effective batch size used to initialize is 100
#metrics {"StartTime": 1640903407.9306726, "EndTime": 1640903407.930727, "Dimensions": {"Algorithm": "AWS/KMeansWebscale", "Host": "algo-1", "Operation": "training", "Meta": "init_train_data_iter"}, "Metrics": {"Total Records Seen": {"sum": 100.0, "count": 1, "min": 100, "max": 100}, "Total Batches Seen": {"sum": 1.0, "count": 1, "min": 1, "max": 1}, "Max Records Seen Between Resets": {"sum": 100.0, "count": 1, "min": 100, "max": 100}, "Max Batches Seen Between Resets": {"sum": 1.0, "count": 1, "min": 1, "max": 1}, "Reset Count": {"sum": 0.0, "count": 1, "min": 0, "max": 0}, "Number of Records Since Last Reset": {"sum": 100.0, "count": 1, "min": 100, "max": 100}, "Number of Batches Since Last Reset": {"sum": 1.0, "count": 1, "min": 1, "max": 1}}}
[2021-12-30 22:30:07.931] [tensorio] [info] epoch_stats={"data_pipeline": "/opt/ml/input/data/train", "epoch": 0, "duration": 23, "num_examples": 1, "num_bytes": 316400}
[2021-12-30 22:30:07.991] [tensorio] [info] epoch_stats={"data_pipeline": "/opt/ml/input/data/train", "epoch": 1, "duration": 56, "num_examples": 1, "num_bytes": 316400}
[12/30/2021 22:30:07 INFO 140709267486528] processed a total of 100 examples
[12/30/2021 22:30:07 INFO 140709267486528] #progress_metric: host=algo-1, completed 100.0 % of epochs
#metrics {"StartTime": 1640903407.931212, "EndTime": 1640903407.992887, "Dimensions": {"Algorithm": "AWS/KMeansWebscale", "Host": "algo-1", "Operation": "training", "epoch": 0, "Meta": "training_data_iter"}, "Metrics": {"Total Records Seen": {"sum": 200.0, "count": 1, "min": 200, "max": 200}, "Total Batches Seen": {"sum": 2.0, "count": 1, "min": 2, "max": 2}, "Max Records Seen Between Resets": {"sum": 100.0, "count": 1, "min": 100, "max": 100}, "Max Batches Seen Between Resets": {"sum": 1.0, "count": 1, "min": 1, "max": 1}, "Reset Count": {"sum": 1.0, "count": 1, "min": 1, "max": 1}, "Number of Records Since Last Reset": {"sum": 100.0, "count": 1, "min": 100, "max": 100}, "Number of Batches Since Last Reset": {"sum": 1.0, "count": 1, "min": 1, "max": 1}}}
[12/30/2021 22:30:07 INFO 140709267486528] #throughput_metric: host=algo-1, train throughput=1616.8254849353934 records/second
[12/30/2021 22:30:07 WARNING 140709267486528] wait_for_all_workers will not sync workers since the kv store is not running distributed
[12/30/2021 22:30:07 INFO 140709267486528] No shrinking is required as extra_center_factor is 1.
[12/30/2021 22:30:07 INFO 140709267486528] batch data loading with context took: 37.4359%, (0.025392 secs)
[12/30/2021 22:30:07 INFO 140709267486528] compute all data-center distances: inner product took: 20.8187%, (0.014121 secs)
[12/30/2021 22:30:07 INFO 140709267486528] collect from kv store took: 12.8873%, (0.008741 secs)
[12/30/2021 22:30:07 INFO 140709267486528] splitting centers key-value pair took: 12.4817%, (0.008466 secs)
[12/30/2021 22:30:07 INFO 140709267486528] compute all data-center distances: point norm took: 4.3654%, (0.002961 secs)
[12/30/2021 22:30:07 INFO 140709267486528] gradient: one_hot took: 3.8184%, (0.002590 secs)
[12/30/2021 22:30:07 INFO 140709267486528] update state and report convergance took: 2.8100%, (0.001906 secs)
[12/30/2021 22:30:07 INFO 140709267486528] predict compute msd took: 1.8802%, (0.001275 secs)
[12/30/2021 22:30:07 INFO 140709267486528] gradient: cluster center took: 1.6862%, (0.001144 secs)
[12/30/2021 22:30:07 INFO 140709267486528] update set-up time took: 1.0306%, (0.000699 secs)
[12/30/2021 22:30:07 INFO 140709267486528] compute all data-center distances: center norm took: 0.4520%, (0.000307 secs)
[12/30/2021 22:30:07 INFO 140709267486528] gradient: cluster size  took: 0.2907%, (0.000197 secs)
[12/30/2021 22:30:07 INFO 140709267486528] predict minus dist took: 0.0429%, (0.000029 secs)
[12/30/2021 22:30:07 INFO 140709267486528] TOTAL took: 0.06782746315002441
[12/30/2021 22:30:07 INFO 140709267486528] Number of GPUs being used: 0
#metrics {"StartTime": 1640903407.9068732, "EndTime": 1640903407.9979143, "Dimensions": {"Algorithm": "AWS/KMeansWebscale", "Host": "algo-1", "Operation": "training"}, "Metrics": {"initialize.time": {"sum": 10.697126388549805, "count": 1, "min": 10.697126388549805, "max": 10.697126388549805}, "epochs": {"sum": 1.0, "count": 1, "min": 1, "max": 1}, "update.time": {"sum": 61.42926216125488, "count": 1, "min": 61.42926216125488, "max": 61.42926216125488}, "finalize.time": {"sum": 3.4265518188476562, "count": 1, "min": 3.4265518188476562, "max": 3.4265518188476562}, "model.serialize.time": {"sum": 1.0650157928466797, "count": 1, "min": 1.0650157928466797, "max": 1.0650157928466797}}}
[12/30/2021 22:30:07 INFO 140709267486528] Test data is not provided.
#metrics {"StartTime": 1640903407.9980702, "EndTime": 1640903407.9999442, "Dimensions": {"Algorithm": "AWS/KMeansWebscale", "Host": "algo-1", "Operation": "training"}, "Metrics": {"setuptime": {"sum": 14.035940170288086, "count": 1, "min": 14.035940170288086, "max": 14.035940170288086}, "totaltime": {"sum": 162.3985767364502, "count": 1, "min": 162.3985767364502, "max": 162.3985767364502}}}
Training seconds: 48
Billable seconds: 48
-------!No monitors found for endpoint. endpoint: kmeans-1640903185-ecd3
Docker entrypoint called with argument(s): serve
Running default environment configuration script
[12/30/2021 22:33:18 INFO 139822761547584 integration.py:636] worker started
[12/30/2021 22:33:18 INFO 139822761547584] loading entry points
[12/30/2021 22:33:18 INFO 139822761547584] Loaded iterator creator application/x-recordio-protobuf for content type ('application/x-recordio-protobuf', '1.0')
[12/30/2021 22:33:18 INFO 139822761547584] loaded request iterator application/json
[12/30/2021 22:33:18 INFO 139822761547584] loaded request iterator application/jsonlines
[12/30/2021 22:33:18 INFO 139822761547584] loaded request iterator application/x-recordio-protobuf
[12/30/2021 22:33:18 INFO 139822761547584] loaded request iterator text/csv
[12/30/2021 22:33:18 INFO 139822761547584] loaded response encoder application/json
[12/30/2021 22:33:18 INFO 139822761547584] loaded response encoder application/jsonlines
[12/30/2021 22:33:18 INFO 139822761547584] loaded response encoder application/x-recordio-protobuf
[12/30/2021 22:33:18 INFO 139822761547584] loaded response encoder text/csv
[12/30/2021 22:33:18 INFO 139822761547584] loaded entry point class algorithm:model
[12/30/2021 22:33:18 INFO 139822761547584] Loaded iterator creator application/x-recordio-protobuf for content type ('application/x-recordio-protobuf', '1.0')
[12/30/2021 22:33:18 INFO 139822761547584] Number of server workers: 4
[2021-12-30 22:33:18 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2021-12-30 22:33:18 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2021-12-30 22:33:18 +0000] [1] [INFO] Using worker: sync
[2021-12-30 22:33:18 +0000] [44] [INFO] Booting worker with pid: 44
[12/30/2021 22:33:18 INFO 139822761547584] loading model...
[12/30/2021 22:33:18 WARNING 139822761547584] Requesting context without setting the requested num of gpus. Using 'auto'
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi: took 0.056 seconds to run.
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi identified 0 GPUs.
[12/30/2021 22:33:19 INFO 139822761547584] Number of GPUs being used: 0
[12/30/2021 22:33:19 INFO 139822761547584] ...model loaded.
[2021-12-30 22:33:19 +0000] [65] [INFO] Booting worker with pid: 65
[12/30/2021 22:33:19 INFO 139822761547584] loading model...
[12/30/2021 22:33:19 WARNING 139822761547584] Requesting context without setting the requested num of gpus. Using 'auto'
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi: took 0.030 seconds to run.
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi identified 0 GPUs.
[12/30/2021 22:33:19 INFO 139822761547584] Number of GPUs being used: 0
[12/30/2021 22:33:19 INFO 139822761547584] ...model loaded.
[2021-12-30 22:33:19 +0000] [86] [INFO] Booting worker with pid: 86
[12/30/2021 22:33:19 INFO 139822761547584] loading model...
[12/30/2021 22:33:19 WARNING 139822761547584] Requesting context without setting the requested num of gpus. Using 'auto'
[2021-12-30 22:33:19 +0000] [95] [INFO] Booting worker with pid: 95
[12/30/2021 22:33:19 INFO 139822761547584] loading model...
[12/30/2021 22:33:19 WARNING 139822761547584] Requesting context without setting the requested num of gpus. Using 'auto'
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi: took 0.033 seconds to run.
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi identified 0 GPUs.
[12/30/2021 22:33:19 INFO 139822761547584] Number of GPUs being used: 0
[12/30/2021 22:33:19 INFO 139822761547584] ...model loaded.
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi: took 0.032 seconds to run.
[12/30/2021 22:33:19 INFO 139822761547584] nvidia-smi identified 0 GPUs.
[12/30/2021 22:33:19 INFO 139822761547584] Number of GPUs being used: 0
[12/30/2021 22:33:19 INFO 139822761547584] ...model loaded.
[2021-12-30 22:34:12.980] [tensorio] [info] epoch_stats={"data_pipeline": "/tmp/tmpu963yvgf/tmp94nehr_3", "epoch": 0, "duration": 43, "num_examples": 1, "num_bytes": 31640}
PASSED

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2021-12-30T23:57:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 940ff02
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2021-12-31T00:15:24Z

Codecov Report

Merging #2815 (d29665a) into dev (1312575) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##              dev    #2815   +/-   ##
=======================================
  Coverage   88.67%   88.67%           
=======================================
  Files         173      173           
  Lines       15428    15429    +1     
=======================================
+ Hits        13681    13682    +1     
  Misses       1747     1747

Impacted Files	Coverage Δ
src/sagemaker/session.py	`70.55% <100.00%> (+0.02%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1312575...d29665a. Read the comment docs.

sagemaker-bot · 2021-12-31T00:18:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d29665a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

)

…s#2815)

* feature: allow conditional parellel builds (#2727) * fix endpoint bug (#2772) Co-authored-by: Basil Beirouti <[email protected]> * fix: local mode - support relative file structure (#2768) * prepare release v2.72.0 * update development version to v2.72.1.dev0 * fix: Set ProcessingStep upload locations deterministically to avoid c… (#2790) * fix: Prevent repack_model script from referencing nonexistent directories (#2755) Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> * fix: S3Input - add support for instance attributes (#2754) * fix: typos and broken link (#2765) Co-authored-by: Shreya Pandit <[email protected]> * prepare release v2.72.1 * update development version to v2.72.2.dev0 * fix: Model Registration with BYO scripts (#2797) Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> * fix: Add ContentType in test_auto_ml_describe * fix: Re-deploy static integ test endpoint if it is not found * documentation :SageMaker model parallel library 1.6.0 API doc (#2814) * update smdmp change log, archive api doc for 1.4.0 and 1.5.0 * add no-index flags * finish api doc archive * fix: Set ProcessingStep upload locations deterministically to avoid c… (#2790) * fix: Prevent repack_model script from referencing nonexistent directories (#2755) Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> * fix: S3Input - add support for instance attributes (#2754) * fix: typos and broken link (#2765) Co-authored-by: Shreya Pandit <[email protected]> * add all api docs * add appendix, fix links * structural changes, fix links * incorporate feedback * prepare release v2.72.1 * update development version to v2.72.2.dev0 Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Mohamed Ali Jamaoui <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Jeniya Tabassum <[email protected]> * fix: fix kmeans test deletion sequence, increment lineage statics (#2815) * fix: Increment static lineage pipeline (#2817) * fix: Update CHANGELOG.md (#2832) * prepare release v2.72.2 * update development version to v2.72.3.dev0 * change: update master from dev (#2836) Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: Mohamed Ali Jamaoui <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: sreedes <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Ameen Khan <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Xiaoguang Chen <[email protected]> Co-authored-by: Jonathan Guinegagne <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Yifei Zhu <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> * prepare release v2.72.3 * update development version to v2.72.4.dev0 * fix: fixes unnecessary session call while generating pipeline definition for lambda step (#2824) * feature: Add models_v2 under lineage context (#2800) * feature: enable python 3.9 (#2802) Co-authored-by: Ahsan Khan <[email protected]> * change: Update CHANGELOG.md (#2842) * fix: update pricing link (#2805) Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Mohamed Ali Jamaoui <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: sreedes <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Ameen Khan <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Xiaoguang Chen <[email protected]> Co-authored-by: Jonathan Guinegagne <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Yifei Zhu <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> * doc: Document the available ExecutionVariables (#2807) * fix: Remove duplicate vertex/edge in query lineage (#2784) * feature: Support model pipelines in CreateModelStep (#2845) Co-authored-by: Payton Staub <[email protected]> * feature: support JsonGet/Join parameterization in tuning step Hyperparameters (#2833) * doc: Enhance smddp 1.2.2 doc (#2852) * feature: support checkpoint to be passed from estimator (#2849) Co-authored-by: marckarp <[email protected]> * fix: allow kms_key to be passed for processing step (#2779) * feature: Adds support for Serverless inference (#2831) * feature: Add support for SageMaker lineage queries in action (#2853) * feature: Adds Lineage queries in artifact, context and trial components (#2838) * feature: Add EMRStep support in Sagemaker pipeline (#2848) Co-authored-by: chenxy <[email protected]> * prepare release v2.73.0 * update development version to v2.73.1.dev0 * feature: Add support for SageMaker lineage queries context (#2830) * fix: support specifying a facet by its column index Currently the Clarify BiasConfig only accepts facet name. Actually Clarify analysis configuration supports both name and index. This commit adds the same support to BiasConfig. * doc: more documentation for serverless inference (#2859) * prepare release v2.74.0 * update development version to v2.74.1.dev0 * Add deprecation warning in Clarify DataConfig (#2847) * feature: Update instance types for integ test (#2881) * feature: Adds support for async inference (#2846) * fix: update to incorporate black v22, pin tox versions (#2889) Co-authored-by: Mufaddal Rohawala <[email protected]> * make black happy Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: Basil Beirouti <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Payton Staub <[email protected]> Co-authored-by: Ahsan Khan <[email protected]> Co-authored-by: Mohamed Ali Jamaoui <[email protected]> Co-authored-by: Shreya Pandit <[email protected]> Co-authored-by: sreedes <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Jeniya Tabassum <[email protected]> Co-authored-by: Ameen Khan <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Xiaoguang Chen <[email protected]> Co-authored-by: Jonathan Guinegagne <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Yifei Zhu <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> Co-authored-by: Xinghan Chen <[email protected]> Co-authored-by: Navin Soni <[email protected]> Co-authored-by: Tulio Casagrande <[email protected]> Co-authored-by: jerrypeng7773 <[email protected]> Co-authored-by: marckarp <[email protected]> Co-authored-by: marckarp <[email protected]> Co-authored-by: jayatalr <[email protected]> Co-authored-by: bhaoz <[email protected]> Co-authored-by: Ethan Cheng <[email protected]> Co-authored-by: chenxy <[email protected]> Co-authored-by: Xiaoguang Chen <[email protected]> Co-authored-by: keerthanvasist <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Shreya Pandit <[email protected]>

navinsoni approved these changes Dec 30, 2021

View reviewed changes

navinsoni self-requested a review December 30, 2021 23:52

fix: fix kmeans test deletion sequence, increment lineage statics

d29665a

mufaddal-rohawala force-pushed the fix-kmeans branch from 940ff02 to d29665a Compare December 31, 2021 00:02

mufaddal-rohawala requested a review from shreyapandit December 31, 2021 00:46

ahsan-z-khan approved these changes Dec 31, 2021

View reviewed changes

mufaddal-rohawala merged commit 6319bce into aws:dev Dec 31, 2021

mufaddal-rohawala deleted the fix-kmeans branch December 31, 2021 21:00

mufaddal-rohawala added a commit that referenced this pull request Jan 4, 2022

fix: fix kmeans test deletion sequence, increment lineage statics (#2815

bbe6284

)

mufaddal-rohawala added a commit that referenced this pull request Jan 4, 2022

fix: fix kmeans test deletion sequence, increment lineage statics (#2815

42dc98e

)

shreyapandit pushed a commit that referenced this pull request Jan 11, 2022

fix: fix kmeans test deletion sequence, increment lineage statics (#2815

9ebfaf1

)

EthanShouhanCheng pushed a commit to SissiChenxy/sagemaker-python-sdk that referenced this pull request Jan 11, 2022

fix: fix kmeans test deletion sequence, increment lineage statics (aw…

8892b36

…s#2815)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: fix kmeans test deletion sequence, increment lineage statics #2815

fix: fix kmeans test deletion sequence, increment lineage statics #2815

mufaddal-rohawala commented Dec 30, 2021 •

edited

Loading

sagemaker-bot commented Dec 30, 2021

codecov-commenter commented Dec 31, 2021

sagemaker-bot commented Dec 31, 2021

fix: fix kmeans test deletion sequence, increment lineage statics #2815

fix: fix kmeans test deletion sequence, increment lineage statics #2815

Conversation

mufaddal-rohawala commented Dec 30, 2021 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Dec 30, 2021

AWS CodeBuild CI Report

codecov-commenter commented Dec 31, 2021

Codecov Report

sagemaker-bot commented Dec 31, 2021

AWS CodeBuild CI Report

mufaddal-rohawala commented Dec 30, 2021 •

edited

Loading