doc: add more details about PyTorch eia #1357

chuyang-deng · 2020-03-14T06:05:44Z

Issue #, if available:
#1360

Description of changes:
Adding more details about PyTorch EIA in readthedocs

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the commit message format described in CONTRIBUTING
I have used the regional endpoint when creating S3 and/or STS clients (if appropriate)
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2020-03-14T06:06:32Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-14T06:06:54Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-14T06:06:56Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-14T06:11:15Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

dfan · 2020-03-14T06:15:30Z

doc/using_pytorch.rst

@@ -303,7 +314,8 @@ It loads the model parameters from a ``model.pth`` file in the SageMaker model d



What do you think about changing the existing examples to use model.pt as well? Or are you against it because people have to implement their own model_fn without EI? I'm worried people will get confused

As discussed offline, changing all .pth to .pt will make us change all previous test scripts, docs and notebook examples.

sagemaker-bot · 2020-03-14T06:37:33Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-14T06:37:46Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-14T06:37:48Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-14T06:45:23Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

davidthomas426 · 2020-03-16T15:36:39Z

doc/using_pytorch.rst

@@ -303,7 +314,8 @@ It loads the model parameters from a ``model.pth`` file in the SageMaker model d

 However, if you are using PyTorch Elastic Inference, you do not have to provide a ``model_fn`` since the PyTorch serving
 container has a default one for you. But please note that if you are utilizing the default ``model_fn``, please save
-yor parameter file as ``model.pt`` instead of ``model.pth``. For more information on inference script, please refer to:
+your ScriptModule as ``model.pt``. If you are implementing your own ``model_fn``, please use TorchScript and ``torch.jit.save``
+to save your ScriptModule. For more information on inference script, please refer to:


Add "then load it in your model_fn with torch.jit.load"

davidthomas426 · 2020-03-16T15:38:54Z

doc/using_pytorch.rst

+    import torch
+
+    model_path = os.path.join(model_dir, "model.pt")
+    model = torch.jit.load(model_path)


Is this supposed to be an example of saving or loading? I'd think this would be where the example would go that traces the model on an example input to get a scriptmodule and then calls torch.jit.save to save it.

This is meant to be a snippet for saving the model, modified.

sagemaker-bot · 2020-03-18T17:20:14Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-18T17:20:27Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-18T17:20:29Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-18T17:28:25Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

davidthomas426 · 2020-03-18T17:46:26Z

doc/using_pytorch.rst

@@ -461,6 +474,23 @@ If you implement your own prediction function, you should take care to ensure th
   first argument to ``output_fn``. If you use the default
   ``output_fn``, this should be a torch.Tensor.

+The default Elastic Inference ``predict_fn`` is similar but using TorchScript and ``torch.jit.optimized_execution``
+to load the output. If you are implementing your own ``predict_fn``, please also use the ``torch.jit.optimized_execution``


nit: "to load the output" => "to run the model".

How about "The default Elastic Inference predict_fn is similar but runs the TorchScript model using torch.jit.optimized_execution

sagemaker-bot · 2020-03-18T22:33:47Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-18T22:34:03Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-18T22:34:05Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-03-18T22:41:49Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Co-authored-by: Raymond Liu <[email protected]> Co-authored-by: Ruilian Gao <[email protected]> Co-authored-by: John Barboza <[email protected]> Co-authored-by: Gary Wang <[email protected]> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Zuoyuan Huang <[email protected]> Co-authored-by: Ao Guo <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Mike Schneider <[email protected]> Co-authored-by: Bhupendra Singh <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: evakravi <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Alexander Pivovarov <[email protected]> Co-authored-by: qidewenwhen <[email protected]> Co-authored-by: mariumof <[email protected]> Co-authored-by: matherit <[email protected]> Co-authored-by: amzn-choeric <[email protected]> Co-authored-by: Ao Guo <[email protected]> Co-authored-by: Sally Seok <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: Qingzi-Lan <[email protected]> Co-authored-by: Sally Seok <[email protected]> Co-authored-by: Manu Seth <[email protected]> Co-authored-by: Miyoung <[email protected]> Co-authored-by: Sarah Castillo <[email protected]> Co-authored-by: EC2 Default User <[email protected]> Co-authored-by: EC2 Default User <[email protected]> Co-authored-by: EC2 Default User <[email protected]> Co-authored-by: Xin Wang <[email protected]> Co-authored-by: stacicho <[email protected]> Co-authored-by: martinRenou <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Akash Goel <[email protected]> Co-authored-by: Joseph Zhang <[email protected]> Co-authored-by: Harsha Reddy <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Xin Wang <[email protected]> Co-authored-by: Gili Nachum <[email protected]> Co-authored-by: Jose Pena <[email protected]> Co-authored-by: cansun <[email protected]> Co-authored-by: AWS-pratab <[email protected]> Co-authored-by: shenlongtang <[email protected]> Co-authored-by: Zach Kimberg <[email protected]> Co-authored-by: chrivtho-github <[email protected]> Co-authored-by: Justin <[email protected]> Co-authored-by: Duc Trung Le <[email protected]> Co-authored-by: HappyAmazonian <[email protected]> Co-authored-by: cj-zhang <[email protected]> Co-authored-by: Matthew <[email protected]> Co-authored-by: Zach Kimberg <[email protected]> Co-authored-by: Rohith Nadimpally <[email protected]> Co-authored-by: rohithn1 <[email protected]> Co-authored-by: Victor Zhu <[email protected]> Co-authored-by: Gary Wang <[email protected]> Co-authored-by: SSRraymond <[email protected]> Co-authored-by: jbarz1 <[email protected]> Co-authored-by: Mohan Gandhi <[email protected]> Co-authored-by: Mohan Gandhi <[email protected]> Co-authored-by: Barboza <[email protected]> Co-authored-by: ruiliann666 <[email protected]> Co-authored-by: Rohan Gujarathi <[email protected]> Co-authored-by: svia3 <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Dewen Qi <[email protected]> Co-authored-by: Edward Sun <[email protected]> Co-authored-by: Stephen Via <[email protected]> Co-authored-by: Namrata Madan <[email protected]> Co-authored-by: Stacia Choe <[email protected]> Co-authored-by: Edward Sun <[email protected]> Co-authored-by: Edward Sun <[email protected]> Co-authored-by: Rohan Gujarathi <[email protected]> Co-authored-by: JohnaAtAWS <[email protected]> Co-authored-by: Vera Yu <[email protected]> Co-authored-by: bhaoz <[email protected]> Co-authored-by: Qing Lan <[email protected]> Co-authored-by: Namrata Madan <[email protected]> Co-authored-by: Sirut Buasai <[email protected]> Co-authored-by: wayneyao <[email protected]> Co-authored-by: Jacky Lee <[email protected]> Co-authored-by: haNa-meister <[email protected]> Co-authored-by: Shailav <[email protected]> Fix unit tests (#1018) Fix happy hf test (#1026) fix logic setup (#1034) fixes (#1045) Fix flake error in init (#1050) fix (#1053) fix: skip tensorflow local mode notebook test (#4060) fix: tags for jumpstart model package models (#4061) fix: pipeline variable kms key (#4065) fix: jumpstart cache using sagemaker session s3 client (#4051) fix: gated models unsupported region (#4069) fix: pipeline upsert failed to pass parallelism_config to update (#4066) fix: temporarily skip kmeans notebook (#4092) fixes (#1051) Fix missing absolute import error (#1057) Fix flake8 error in unit test (#1058) fixes (#1056) Fix flake8 error in integ test (#1060) Fix black format error in test_pickle_dependencies (#1062) Fix docstyle error under serve (#1065) Fix docstyle error in builder failure (#1066) fix black and flake8 formatting (#1069) Fix format error (#1070) Fix integ test (#1074) fix: HuggingFaceProcessor parameterized instance_type when image_uri is absent (#4072) fix: log message when sdk defaults not applied (#4104) fix: handle bad jumpstart default session (#4109) Fix the version information, whl and flake8 (#1085) Fix JSON serializer error (#1088) Fix unit test (#1091) fix format (#1103) Fix local mode predictor (#1107) Fix DJLPredictor (#1108) Fix modelbuilder unit tests (#1118) fixes (#1136) fixes (#1165) fixes (#1166) fix: auto ml integ tests and add flaky test markers (#4136) fix model data for JumpStartModel (#4135) fix: transform step unit test (#4151) fix: Update pipeline.py and selective_execution_config.py with small fixes (#1099) fix: Fixed bug in _create_training_details (#4141) fix: use correct line endings and s3 uris on windows (#4118) fix: js tagging s3 prefix (#4167) fix: Update Ec2 instance type to g5.4xlarge in test_huggingface_torch_distributed.py (#4181) fix: import error in unsupported js regions (#4188) fix: update local mode schema (#4185) fix: fix flaky Inference Recommender integration tests (#4156) fix: clone distribution in validate_distribution (#4205) Fix hyperlinks in feature_processor.scheduler parameter descriptions (#4208) Fix master merge formatting (#1186) Fix master unit tests (#1203) Fix djl unit tests (#1204) Fix merge conflicts (#1217) fix: fix URL links (#4217) fix: bump urllib3 version (#4223) fix: relax upper bound on urllib in local mode requirements (#4219) fixes (#1224) fix formatting (#1233) fix byoc unit tests (#1235) fix byoc unit tests (#1236) Fixed Modelpackage's deploy calling model's deploy (#1155) fix: jumpstart unit-test (#1265) fixes (#963) Fix TorchTensorSer/Deser (#969) fix (#971) fix local container mode (#972) Fix auto detect (#979) Fix routing fn (#981) fix local container serialization (#989) fix custom serialiazation with local container. Also remove a lot of unused code (#994) Fix custom serialization for local container mode (#1000) fix pytorch version (#1001) Fix unit test (#990) fix: Multiple bug fixes including removing unsupported feature. (#1105) Fix some problems with pipeline compilation (#1125) fix: Refactor JsonGet s3 URI and add serialize_output_to_json flag (#1164) fix: invoke_function circular import (#1262) fix: pylint (#1264) fix: Add logging for docker build failures (#1267) Fix session bug when provided in ModelBuilder (#1288) fixes (#1313) fix: Gated content bucket env var override (#1280) fix: Change the library used in pytorch test causing cloudpickle version conflict (#1287) fix: HMAC signing for ModelBuilder Triton python backend (#1282) fix: do not delete temp folder generated by sdist (#1291) fix: Do not require model_server if provided image_uri is a 1p image. (#1303) fix: check image type vs instance type (#1307) fix: unit test (#1315) fix: Fixed model builder's register unable to deploy (#1323) fix: missing `self._framework` in `InferenceSpec` path (#1325) fix: enable xgboost integ test in our own pipeline (#1326) fix: skip py310 (#1328) fix: Update autodetect dlc logic (#1329) Fix secret key in the Model object (#1334) fix: improve error message (#1333) Fix unit testing (#1340) fix: Typing and formatting (#1341) fix: WaiterError on failed pipeline execution. results() (#1337) Fix tox identified errors (#1344) Fix issue when the user runs in Python 3.11 (#1345) fixes (#1346) fix: use copy instead of move in bootstrap script (#1339) Resolve keynote3 conflicts (#1351) Resolve keynote3 conflicts v2 (#1353) Fix conflicts (#1354) Fix conflicts v3 (#1355) fix: get whl from local to run integ tests (#1357) fix: enable triton pt tests (#1358) fix: integ test (#1362) Fix Python 3.11 issue with dataclass decorator (#1345) fix: remote function include_local_workdir default value (#1342) fix: error message (#1373) fixes (#1372) fix: Remvoe PickleSerializer (#1378)

Deng added 2 commits March 13, 2020 23:02

doc: add more details about PyTorch eia

b81c923

fix a space

2756493

dfan reviewed Mar 14, 2020

View reviewed changes

fix sphinx error

efba0e1

davidthomas426 suggested changes Mar 16, 2020

View reviewed changes

address comments

88bafc0

davidthomas426 reviewed Mar 18, 2020

View reviewed changes

davidthomas426 previously approved these changes Mar 18, 2020

View reviewed changes

address comments

5d48b5e

chuyang-deng dismissed davidthomas426’s stale review via 5d48b5e March 18, 2020 22:32

dfan approved these changes Mar 18, 2020

View reviewed changes

laurenyu approved these changes Mar 19, 2020

View reviewed changes

chuyang-deng merged commit d8d64da into aws:master Mar 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

doc: add more details about PyTorch eia #1357

doc: add more details about PyTorch eia #1357

chuyang-deng commented Mar 14, 2020 •

edited

Loading

sagemaker-bot commented Mar 14, 2020

sagemaker-bot commented Mar 14, 2020

sagemaker-bot commented Mar 14, 2020

sagemaker-bot commented Mar 14, 2020

dfan Mar 14, 2020

chuyang-deng Mar 18, 2020

sagemaker-bot commented Mar 14, 2020

sagemaker-bot commented Mar 14, 2020

sagemaker-bot commented Mar 14, 2020

sagemaker-bot commented Mar 14, 2020

davidthomas426 Mar 16, 2020

chuyang-deng Mar 18, 2020

davidthomas426 Mar 16, 2020

chuyang-deng Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

davidthomas426 Mar 18, 2020

dfan Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

sagemaker-bot commented Mar 18, 2020

		@@ -303,7 +314,8 @@ It loads the model parameters from a ``model.pth`` file in the SageMaker model d

doc: add more details about PyTorch eia #1357

doc: add more details about PyTorch eia #1357

Conversation

chuyang-deng commented Mar 14, 2020 • edited Loading

Merge Checklist

General

Tests

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 14, 2020

AWS CodeBuild CI Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Mar 18, 2020

AWS CodeBuild CI Report

chuyang-deng commented Mar 14, 2020 •

edited

Loading