Skip to content

Add support for Batch Transform and update README with TF Pipe Mode #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jul 17, 2018

Conversation

laurenyu
Copy link
Contributor

Description of changes:

  • Add a Transformer class for creating batch transform jobs
  • Add a section to the TF README about using pipe mode

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have updated the changelog with a description of my changes (if appropriate)
  • I have updated any necessary documentation (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

laurenyu and others added 14 commits July 16, 2018 16:06
This change adds wait_for_transform_job function to Session. It's a part of
batch inference support. Transform job creation and attach are non-blocking.
This method provides a way to block and wait for the job to finish.
Attach enables attaching to an existing transform job and returns a Transformer
instance.

Wait blocks is a blocking operation which reports the progress of a transform
job and will only return when the job finishes.
Use dict.get to retrive optional fields from job_details to avoid KeyError
@laurenyu laurenyu requested a review from owen-t July 17, 2018 11:52
@codecov-io
Copy link

Codecov Report

Merging #298 into master will increase coverage by 0.39%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #298      +/-   ##
==========================================
+ Coverage   92.66%   93.05%   +0.39%     
==========================================
  Files          49       50       +1     
  Lines        3313     3456     +143     
==========================================
+ Hits         3070     3216     +146     
+ Misses        243      240       -3
Impacted Files Coverage Δ
src/sagemaker/job.py 94.73% <100%> (ø) ⬆️
src/sagemaker/estimator.py 88.7% <100%> (+1.31%) ⬆️
src/sagemaker/session.py 88.52% <100%> (+1.61%) ⬆️
src/sagemaker/transformer.py 100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ea3fd0...a75f3e1. Read the comment docs.

@laurenyu laurenyu merged commit 704cd31 into aws:master Jul 17, 2018
@laurenyu laurenyu deleted the batch-transform-release branch July 17, 2018 13:53
knakad pushed a commit to knakad/sagemaker-python-sdk that referenced this pull request Dec 4, 2019
knakad pushed a commit that referenced this pull request Dec 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants