Skip to content

fix: crashing when hyperparameters are queried over estimator attached to completed script mode job #718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Mar 27, 2019
Merged

fix: crashing when hyperparameters are queried over estimator attached to completed script mode job #718

merged 7 commits into from
Mar 27, 2019

Conversation

akorobko
Copy link
Contributor

@akorobko akorobko commented Mar 22, 2019 •
edited
Loading

Issue #, if available:
#717

Description of changes:
Do not format path if one of the components is None. In such situation we do not have any other option.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

  • I have read the CONTRIBUTING doc
  • I used the commit message format described in CONTRIBUTING
  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have updated any necessary documentation (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jesterhazy
Copy link
Contributor

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@akorobko akorobko changed the title Prevent crashing when hyperparameters are queried over estimator attached to completed script mode job fix: crashing when hyperparameters are queried over estimator attached to completed script mode job Mar 23, 2019
@yangaws
Copy link
Contributor

yangaws commented Mar 24, 2019

Hi @akorobko ,

Thanks for your contribution. Can you also add a unit test for this?

The unit test can be added in this file: https://github.com/aws/sagemaker-python-sdk/blob/master/tests/unit/test_tf_estimator.py

You can also refer to the existed attach tests.

@jesterhazy
Copy link
Contributor

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@akorobko
Copy link
Contributor Author

@yangaws Thanks for the feedback. I have added the test for this particular case (similar to test_attach but for the script mode). Test fails without fix and succeeds with the fix.

@jesterhazy
Copy link
Contributor

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@akorobko
Copy link
Contributor Author

The failure is not related to my fix and happening somewhere else.

@yangaws
Copy link
Contributor

yangaws commented Mar 26, 2019

@akorobko Yep seems it's just a flaky test that times out. I will re-run your test.

@jesterhazy
Copy link
Contributor

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@akorobko
Copy link
Contributor Author

It still fails at some distributed GPU unit test which is unrelated to my changes.

@yangaws
Copy link
Contributor

yangaws commented Mar 27, 2019

@akorobko
There're flakiness in our tests, especially when too many PRs running it at same time. Sorry for sending you failure notification emails all the time. We will definitely deal with it.

@jesterhazy
Copy link
Contributor

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@akorobko
Copy link
Contributor Author

@yangaws thanks, all checks have succeeded.

@yangaws yangaws merged commit 781ffd1 into aws:master Mar 27, 2019
mizanfiu pushed a commit to mizanfiu/sagemaker-python-sdk that referenced this pull request Dec 13, 2022
@mufaddal-rohawala @mufiAmazon
feature: Support Amazon SageMaker AutoMLStep (aws#718)
bc84809 
Co-authored-by: Xinlu Tu <[email protected]>
Co-authored-by: Xinlu Tu <[email protected]>

@xinlutu2
feat: Close feature gaps between Python SageMaker SDK and CreateAutoMLJob API includes ENSEMBLING mode (aws#681)

@xinlutu2
feature: add AutoMLStep for SageMaker Pipelines Workflows (aws#693) 

@xinlutu2
feature: add AutoMLStep integration test (aws#713)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yangaws yangaws yangaws approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@akorobko @jesterhazy @yangaws