doc: add some clarification to Processing docs #1600

laurenyu · 2020-06-16T22:19:30Z

Issue #, if available:
#1569

Description of changes:
Added a line to hopefully help explain what makes ScriptProcessor different from Processor. Also cleaned up some formatting things in the Processing docs.

Testing done:

built the docs locally
tox -e sphinx,doc8

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the commit message format described in CONTRIBUTING
I have passed the region in to any/all clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2020-06-16T22:32:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2af4794
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-16T22:45:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 2af4794
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-16T23:15:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2af4794
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-16T23:47:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 2af4794
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T02:54:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 7cd8c1e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T03:10:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 7cd8c1e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T03:14:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 7cd8c1e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T03:39:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 7cd8c1e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T16:53:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 6d16f34
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

metrizable

a few nits is all

metrizable · 2020-06-17T16:54:42Z

doc/amazon_sagemaker_processing.rst

-You can run a Scikit-Learn script to do data processing on SageMaker using the `SKLearnProcessor`_ class.
-
-.. _SKLearnProcessor: https://sagemaker.readthedocs.io/en/stable/sagemaker.sklearn.html#sagemaker.sklearn.processing.SKLearnProcessor
+You can run a Scikit-Learn script to do data processing on SageMaker using the :class:`sagemaker.sklearn.processing.SKLearnProcessor` class.


@ignorable small doc nit: you typically see in scikit-learn.org docs, the L is never uppercase. The S is typically only uppercase at the beginning of sentences.

metrizable · 2020-06-17T16:55:58Z

doc/amazon_sagemaker_processing.rst

+    sklearn_processor.run(
+        code='preprocessing.py',
+        inputs=[
+            ProcessingInput(source='s3://your-bucket/path/to/your/data, destination='/opt/ml/processing/input'),


@ignorable nit-doc: should probably match our preferred formatter black with double-quotes, etc.

metrizable · 2020-06-17T16:58:27Z

src/sagemaker/processing.py

@@ -269,7 +269,9 @@ def _normalize_outputs(self, outputs=None):


 class ScriptProcessor(Processor):
-    """Handles Amazon SageMaker processing tasks for jobs using a machine learning framework."""
+    """Handles Amazon SageMaker processing tasks for jobs using a machine learning framework,
+    which allows for providing a script to be run as part of the Processing Job.


nit pep257: one line first line, details can be added in body

sagemaker-bot · 2020-06-17T17:08:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 6d16f34
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T17:13:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 6d16f34
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T17:37:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 6d16f34
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T18:04:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5aa0ff8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T18:20:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5aa0ff8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T18:29:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5aa0ff8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T18:48:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0cea402
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T18:49:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5aa0ff8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T19:04:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0cea402
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T19:34:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 0cea402
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2020-06-17T19:35:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0cea402
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

metrizable

ship it!

* prepare release v2.232.2 * update development version to v2.232.3.dev0 * fix: Use Miniforge to replace MambaForge (#4884) * Use Miniforge to replace MambaForge * Fix download url * tests: Implement integration tests covering JumpStart PrivateHub workflows (#4883) * tests: Implement integration tests covering JumpStart PrivateHub workflows * linting * formating * Only delete the pytest session specific test * change scope to session * address nits * Address test failures * address typo * address comments * resolve flake8 errors * implement throttle handling * flake8 * flake8 * Adding more assertions --------- Co-authored-by: malavhs <[email protected]> * chore: add lmi image config in me-central-1 (#4887) * changes for PT 2.4 currency upgrade (#4890) Co-authored-by: Andrew Tian <[email protected]> * chore(deps): bump pyspark from 3.3.1 to 3.3.2 in /requirements/extras (#4894) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update cloudpickle version to >=2.2.1 (#4891) * update cloudpickle version to >=2.2.1 in pyproject.toml (#4899) * Revert "update cloudpickle version to >=2.2.1 in pyproject.toml (#4899)" (#4902) * release: huggingface tgi neuronx 0.0.25 image (#4893) * release: huggingface tgi neuronx 0.0.25 image * tests * add tgi 2.3.1 * update default version * update py version * fix tests * test * Revert "fix tests" This reverts commit 9374c75. * Revert "Revert "fix tests"" This reverts commit 20d46d1. * Revert "test" This reverts commit 90f6e0b. * fix: fixing typo in dependecy setup (#4905) charset-normalizer is misplet in the requirements.txt files * Fix: Returning ModelPackage object on register of PipelineModel (#4878) Co-authored-by: Keshav Chandak <[email protected]> * fix: modified pull request template (#4906) Added warning to spell check dependencies added and ensure they exist in PyPi * Fix: image_uris graviton image uri (#4909) * change: update image_uri_configs 09-27-2024 07:18:01 PST * change: update image_uri_configs 10-03-2024 07:17:59 PST * change: update image_uri_configs 10-17-2024 07:17:55 PST * change: update image_uri_configs 10-23-2024 11:26:03 PST * change: adding eu-central-2 bucket info to JS constants (#4907) * change: adding eu-central-2 bucket info to JS constants * change: adding zrh image uris for dji neuronx --------- Co-authored-by: shaernev <[email protected]> * fix: Skip pytorch tests incompatible with latest version 2.4.0 (#4910) * change: update image_uri_configs 10-29-2024 07:17:56 PST * prepare release v2.232.3 * update development version to v2.232.4.dev0 * change: Updates for DJL 0.30.0 release (#4892) Co-authored-by: pintaoz-aws <[email protected]> * bumping smp version from 2.6.0 to 2.6.1 (#4913) Co-authored-by: Andrew Tian <[email protected]> Co-authored-by: nileshvd <[email protected]> * feat: Marketplace model support in HubService (#4916) * feat: Marketplace model support in HubService * fix: removing field * fix: Reverting name change for code coverage * fix: Adding more code coverage * fix: linting * fix: Fixing coverage tests * fix: Fixing integration tests * fix: Minor fixes * feat: triton v24.09 (#4908) * fix: Fixing JumpStart Tests (#4917) * fix: Fixing tests * fix: fixing test name * fix: dummy commit * fix: reverting dummy commit * fix: Removing flakey tests --------- Co-authored-by: nileshvd <[email protected]> * fix: merge * fix: Commenting out marketplace test * fix: Linting --------- Co-authored-by: ci <ci> Co-authored-by: pintaoz-aws <[email protected]> Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: malavhs <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: adtian2 <[email protected]> Co-authored-by: Andrew Tian <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: liujiaor <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: Ernev Sharma <[email protected]> Co-authored-by: shaernev <[email protected]> Co-authored-by: Tyler Osterberg <[email protected]> Co-authored-by: nileshvd <[email protected]> Co-authored-by: Aaqib <[email protected]>

doc: add some clarification to Processing docs

2af4794

laurenyu mentioned this pull request Jun 16, 2020

Difference between ScriptProcessor vs Processor #1569

Closed

Merge branch 'master' into processor-docs

7cd8c1e

Merge branch 'master' into processor-docs

6d16f34

ajaykarpur previously approved these changes Jun 17, 2020

View reviewed changes

metrizable previously approved these changes Jun 17, 2020

View reviewed changes

address PR comments

5aa0ff8

laurenyu dismissed stale reviews from metrizable and ajaykarpur via 5aa0ff8 June 17, 2020 17:51

laurenyu requested a review from metrizable June 17, 2020 17:52

Merge branch 'master' into processor-docs

0cea402

metrizable approved these changes Jun 17, 2020

View reviewed changes

laurenyu merged commit 4d948ac into aws:master Jun 17, 2020

laurenyu deleted the processor-docs branch June 17, 2020 19:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

doc: add some clarification to Processing docs #1600

doc: add some clarification to Processing docs #1600

laurenyu commented Jun 16, 2020

sagemaker-bot commented Jun 16, 2020

sagemaker-bot commented Jun 16, 2020

sagemaker-bot commented Jun 16, 2020

sagemaker-bot commented Jun 16, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

metrizable left a comment

metrizable Jun 17, 2020

metrizable Jun 17, 2020

metrizable Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

sagemaker-bot commented Jun 17, 2020

metrizable left a comment

doc: add some clarification to Processing docs #1600

doc: add some clarification to Processing docs #1600

Conversation

laurenyu commented Jun 16, 2020

Merge Checklist

General

Tests

sagemaker-bot commented Jun 16, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 16, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 16, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 16, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

metrizable left a comment

Choose a reason for hiding this comment

metrizable Jun 17, 2020

Choose a reason for hiding this comment

metrizable Jun 17, 2020

Choose a reason for hiding this comment

metrizable Jun 17, 2020

Choose a reason for hiding this comment

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

sagemaker-bot commented Jun 17, 2020

AWS CodeBuild CI Report

metrizable left a comment

Choose a reason for hiding this comment