Skip to content

MXNetModel.deploy() not returning until endpoint deployment complete #731

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nemalipuri opened this issue Mar 29, 2019 · 3 comments · Fixed by #842
Closed

MXNetModel.deploy() not returning until endpoint deployment complete #731

nemalipuri opened this issue Mar 29, 2019 · 3 comments · Fixed by #842

Comments

@nemalipuri
Copy link

System Information

  • Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): MXNet
  • Framework Version: 1.3.0
  • Python Version: 3.6
  • CPU or GPU: CPU
  • Python SDK Version: 1.18.x
  • Are you using a custom image: No

Describe the problem

When creating a new endpoint using MXNetModel.deploy() it is not returning until deployment complete.
I do not want to wait for the endpoint deployment to complete. deploy() is not accepting 'wait' argument.
I need to execute deploy() from Lambda, so expecting deploy() to return response immediately.

Minimal repro / logs

deploy() got an unexpected keyword argument 'wait'

  • Exact command to reproduce:
    model.deploy(initial_instance_count=1, instance_type='ml.p2.xlarge', endpoint_name='p-sagemaker-endpoint-fm-t1', update_endpoint=False, wait=False)
@nemalipuri
Copy link
Author

I see below call is not passing 'wait' to session.create_endpoint.

self.sagemaker_session.endpoint_from_production_variants(self.endpoint_name, [production_variant], tags)

@jesterhazy
Copy link
Contributor

@nemalipuri

You're right, the deploy function doesn't take a wait argument the way Estimator.fit(...) does. I'll mark this as a feature request.

However, if you are intending to run this from lambda, you will run into another problem. The dependencies for this package are too large to fit in a lambda code archive. That means you would need to package them in a Lambda Layer, even after the missing wait feature is added.

If all you are doing in the lambda is starting the endpoint, I'd recommend that you avoid both issues by using the plain boto3 sdk to deploy the endpoint.

@nemalipuri
Copy link
Author

@jesterhazy

I agree, thought of using boto3 but I preferred using mxnet api because deploy() creates Model, endpoint config and hosting endpoint in a single call.

Regarding package size limit, yes Layer did not take direct import of zip file since it's size is more than 50MB. I'd to place it in S3 and imported it there into Lambda layer.

When I'm planning to product-ionize my work I will use boto3.
Thanks for marking this as a Feature Request.

mizanfiu pushed a commit to mizanfiu/sagemaker-python-sdk that referenced this issue Dec 13, 2022
mizanfiu pushed a commit to mizanfiu/sagemaker-python-sdk that referenced this issue Dec 13, 2022
claytonparnell pushed a commit that referenced this issue Dec 15, 2022
* Add list_feature_groups API (#647)

* feat: Feature/get record api (#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (#664)

* feat: Add DatasetBuilder class (#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (#699)

* feat: Add pandas.Dataframe as base case (#708)

* feat: Add with_feature_group method in DatasetBuilder (#726)

* feat: Handle merge and timestamp filters (#727)

* feat: Add to_dataframe method in DatasetBuilder (#729)

* Address TODOs (#731)

* Unit test for DatasetBuilder (#734)

* fix: Fix list_feature_groups max_results (#744)

* Add integration tests for create_dataset (#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
claytonparnell pushed a commit to claytonparnell/sagemaker-python-sdk that referenced this issue Dec 16, 2022
* Add list_feature_groups API (aws#647)

* feat: Feature/get record api (aws#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (aws#664)

* feat: Add DatasetBuilder class (aws#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (aws#699)

* feat: Add pandas.Dataframe as base case (aws#708)

* feat: Add with_feature_group method in DatasetBuilder (aws#726)

* feat: Handle merge and timestamp filters (aws#727)

* feat: Add to_dataframe method in DatasetBuilder (aws#729)

* Address TODOs (aws#731)

* Unit test for DatasetBuilder (aws#734)

* fix: Fix list_feature_groups max_results (aws#744)

* Add integration tests for create_dataset (aws#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (aws#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
mufaddal-rohawala pushed a commit to mufaddal-rohawala/sagemaker-python-sdk that referenced this issue Dec 19, 2022
* Add list_feature_groups API (aws#647)

* feat: Feature/get record api (aws#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (aws#664)

* feat: Add DatasetBuilder class (aws#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (aws#699)

* feat: Add pandas.Dataframe as base case (aws#708)

* feat: Add with_feature_group method in DatasetBuilder (aws#726)

* feat: Handle merge and timestamp filters (aws#727)

* feat: Add to_dataframe method in DatasetBuilder (aws#729)

* Address TODOs (aws#731)

* Unit test for DatasetBuilder (aws#734)

* fix: Fix list_feature_groups max_results (aws#744)

* Add integration tests for create_dataset (aws#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (aws#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
mufaddal-rohawala pushed a commit that referenced this issue Dec 20, 2022
* Add list_feature_groups API (#647)

* feat: Feature/get record api (#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (#664)

* feat: Add DatasetBuilder class (#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (#699)

* feat: Add pandas.Dataframe as base case (#708)

* feat: Add with_feature_group method in DatasetBuilder (#726)

* feat: Handle merge and timestamp filters (#727)

* feat: Add to_dataframe method in DatasetBuilder (#729)

* Address TODOs (#731)

* Unit test for DatasetBuilder (#734)

* fix: Fix list_feature_groups max_results (#744)

* Add integration tests for create_dataset (#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this issue Mar 4, 2023
* Add list_feature_groups API (aws#647)

* feat: Feature/get record api (aws#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (aws#664)

* feat: Add DatasetBuilder class (aws#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (aws#699)

* feat: Add pandas.Dataframe as base case (aws#708)

* feat: Add with_feature_group method in DatasetBuilder (aws#726)

* feat: Handle merge and timestamp filters (aws#727)

* feat: Add to_dataframe method in DatasetBuilder (aws#729)

* Address TODOs (aws#731)

* Unit test for DatasetBuilder (aws#734)

* fix: Fix list_feature_groups max_results (aws#744)

* Add integration tests for create_dataset (aws#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (aws#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this issue Mar 4, 2023
* Add list_feature_groups API (aws#647)

* feat: Feature/get record api (aws#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (aws#664)

* feat: Add DatasetBuilder class (aws#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (aws#699)

* feat: Add pandas.Dataframe as base case (aws#708)

* feat: Add with_feature_group method in DatasetBuilder (aws#726)

* feat: Handle merge and timestamp filters (aws#727)

* feat: Add to_dataframe method in DatasetBuilder (aws#729)

* Address TODOs (aws#731)

* Unit test for DatasetBuilder (aws#734)

* fix: Fix list_feature_groups max_results (aws#744)

* Add integration tests for create_dataset (aws#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (aws#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this issue Apr 18, 2023
* Add list_feature_groups API (aws#647)

* feat: Feature/get record api (aws#650)

Co-authored-by: Eric Zou <[email protected]>

* Add delete_record API (aws#664)

* feat: Add DatasetBuilder class (aws#667)

Co-authored-by: Eric Zou <[email protected]>

* feat: Add to_csv method in DatasetBuilder (aws#699)

* feat: Add pandas.Dataframe as base case (aws#708)

* feat: Add with_feature_group method in DatasetBuilder (aws#726)

* feat: Handle merge and timestamp filters (aws#727)

* feat: Add to_dataframe method in DatasetBuilder (aws#729)

* Address TODOs (aws#731)

* Unit test for DatasetBuilder (aws#734)

* fix: Fix list_feature_groups max_results (aws#744)

* Add integration tests for create_dataset (aws#743)

* feature: Aggregate commits

* fix: as_of, event_range, join, default behavior and duplicates… (aws#764)

* Bug fixed - as_of, event_range, join, default behavior and duplicates and tests

Bugs:
1. as_of was not working properly on deleted events
2. Same event_time_range
3. Join was not working when including feature names
4. Default sql was returning only most recent, whereas it should all excluding duplicates
5. Include duplicates was not return all non-deleted data
6. instanceof(dataframe) case was also applied to non-df cases while join
7. Include column was returning unnecessary columns.

* Fix on pylint error

* Fix on include_duplicated_records for panda data frames

* Fix format issue for black

* Bug fixed related to line break

* Bug fix related to dataframe and inclde_deleted_record and include_duplicated_record

* Addressed comments and code refactored

* changed to_csv to to_csv_file and added error messages for query limit and recent record limit

* Revert a change which was not intended

* Resolved the leak of feature group deletion in integration test

* Added doc update for dataset builder

* Fix the issue in doc

Co-authored-by: Yiming Zou <[email protected]>
Co-authored-by: Brandon Chatham <[email protected]>
Co-authored-by: Eric Zou <[email protected]>
Co-authored-by: jiapinw <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants