3826: Fix LambdaStep Creation #3828

dipanjank · 2023-04-29T10:51:03Z

Issue #, if available:
#3826

Description of changes:

Fix LambdaStep Creation with suitable defaults for vpc_config, environment and layers when they are not specified by the caller. (dict, dict and list, respectively).
In addition. removed the architectures argument. This argument is not supported completely in boto3 Lambda client's create_function method and so it would have never worked. Even though it is documented, specifying it generates an "invalid parameter" error, e.g.

def create_function():
    lambda_client = boto3.client("lambda")
    response = lambda_client.create_function(
        FunctionName="hello_world",
        Runtime="python3.9",
        Handler="lambda_handler",
        Role="arn:aws:iam::<account-id>:role/service-role/test_hello-role-vomn97mn",
        Code={"ZipFile": build_code()},
        VpcConfig={},
        Environment={},
        Layers=[],
        Architectures=["x86-64"],
    )
    print(response)

Output:

Traceback (most recent call last):
  File "/Users/[email protected]/Library/Application Support/JetBrains/PyCharm2022.3/scratches/scratch_93.py", line 42, in <module>
    create_function(lc)
  File "/Users/[email protected]/Library/Application Support/JetBrains/PyCharm2022.3/scratches/scratch_93.py", line 16, in create_function
    Architectures=["x86-64"],
  File "/Users/[email protected]/opt/anaconda3/envs/dev/lib/python3.6/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/[email protected]/opt/anaconda3/envs/dev/lib/python3.6/site-packages/botocore/client.py", line 678, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/Users/[email protected]/opt/anaconda3/envs/dev/lib/python3.6/site-packages/botocore/client.py", line 726, in _convert_to_request_dict
    api_params, operation_model)
  File "/Users/[email protected]/opt/anaconda3/envs/dev/lib/python3.6/site-packages/botocore/validate.py", line 337, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in input: "Architectures", must be one of: FunctionName, Runtime, Role, Handler, Code, Description, Timeout, MemorySize, Publish, VpcConfig, PackageType, DeadLetterConfig, Environment, KMSKeyArn, TracingConfig, Tags, Layers, FileSystemConfigs, ImageConfig, CodeSigningConfigArn

Testing done:

$ pytest tests/unit/test_lambda_helper.py 
================================================================================================= test session starts ==================================================================================================
platform darwin -- Python 3.9.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/[email protected]/PycharmProjects/sagemaker-python-sdk, configfile: tox.ini
plugins: timeout-2.1.0, rerunfailures-10.2, xdist-2.4.0, anyio-3.6.2, forked-1.6.0, cov-3.0.0
collected 28 items                                                                                                                                                                                                     

tests/unit/test_lambda_helper.py ............................                                                                                                                                                    [100%]

============================================================================================ 28 passed in 63.52s (0:01:03) ============================================================================================

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…#3804) Co-authored-by: Michael Trinh <[email protected]>

…eters are not specified.

hxrsmurf

LGTM

knikure

/bot run all

sagemaker-bot · 2023-05-02T18:25:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 1ed2077
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-02T18:31:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1ed2077
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-02T18:36:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 1ed2077
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-02T18:46:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1ed2077
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-02T20:41:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1ed2077
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-05-02T20:41:50Z

Codecov Report

Merging #3828 (973810a) into master (10663b3) will decrease coverage by 0.90%.
The diff coverage is 49.48%.

@@            Coverage Diff             @@
##           master    #3828      +/-   ##
==========================================
- Coverage   89.88%   88.99%   -0.90%     
==========================================
  Files        1060      247     -813     
  Lines      101208    25133   -76075     
==========================================
- Hits        90973    22367   -68606     
+ Misses      10235     2766    -7469

Impacted Files	Coverage Δ
src/sagemaker/lineage/query.py	`76.92% <46.73%> (ø)`
src/sagemaker/lambda_helper.py	`95.90% <100.00%> (ø)`
...runtime_environment/runtime_environment_manager.py	`79.75% <100.00%> (ø)`

... and 1304 files with indirect coverage changes

knikure

/bot run all

sagemaker-bot · 2023-05-05T17:40:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 973810a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-05T17:47:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 973810a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-05T17:53:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 973810a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

dipanjank · 2023-05-05T17:57:01Z

Hi @knikure the branch keeps getting out of date - do you want me to update it now?

sagemaker-bot · 2023-05-05T18:03:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 973810a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-05T19:50:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 973810a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

spoorn and others added 3 commits April 29, 2023 12:43

fix: Update Clarify SHAPConfig baseline to allow JSON structures (aws…

57c3d52

…#3804) Co-authored-by: Michael Trinh <[email protected]>

fix: Make sure LambdaStep can be created successfully when some param…

5d0fbca

…eters are not specified.

fix: remove unnecessary file

1e6df60

dipanjank requested a review from a team as a code owner April 29, 2023 10:51

dipanjank requested review from mufaddal-rohawala and removed request for a team April 29, 2023 10:51

Merge branch 'master' into 3826-bugfix-lambda-step

d2f0ade

hxrsmurf approved these changes Apr 29, 2023

View reviewed changes

knikure linked an issue May 1, 2023 that may be closed by this pull request

lambda_helper Lambda Is Not Able to Create - Requires VpcConfig #3826

Closed

qidewenwhen approved these changes May 2, 2023

View reviewed changes

Merge branch 'master' into 3826-bugfix-lambda-step

1ed2077

knikure reviewed May 2, 2023

View reviewed changes

Merge branch 'master' into 3826-bugfix-lambda-step

75242da

dipanjank added 2 commits May 3, 2023 08:09

Merge branch 'master' into 3826-bugfix-lambda-step

9f093f4

Merge branch 'master' into 3826-bugfix-lambda-step

973810a

knikure reviewed May 5, 2023

View reviewed changes

mufaddal-rohawala approved these changes May 9, 2023

View reviewed changes

mufaddal-rohawala self-requested a review May 9, 2023 00:49

mufaddal-rohawala merged commit 28b57fd into aws:master May 9, 2023

dipanjank deleted the 3826-bugfix-lambda-step branch May 9, 2023 10:35

dipanjank restored the 3826-bugfix-lambda-step branch May 9, 2023 10:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3826: Fix LambdaStep Creation #3828

3826: Fix LambdaStep Creation #3828

dipanjank commented Apr 29, 2023 •

edited

Loading

hxrsmurf left a comment

knikure left a comment

sagemaker-bot commented May 2, 2023

sagemaker-bot commented May 2, 2023

sagemaker-bot commented May 2, 2023

sagemaker-bot commented May 2, 2023

sagemaker-bot commented May 2, 2023

codecov-commenter commented May 2, 2023 •

edited

Loading

knikure left a comment

sagemaker-bot commented May 5, 2023

sagemaker-bot commented May 5, 2023

sagemaker-bot commented May 5, 2023

dipanjank commented May 5, 2023

sagemaker-bot commented May 5, 2023

sagemaker-bot commented May 5, 2023

3826: Fix LambdaStep Creation #3828

3826: Fix LambdaStep Creation #3828

Conversation

dipanjank commented Apr 29, 2023 • edited Loading

Merge Checklist

General

Tests

hxrsmurf left a comment

Choose a reason for hiding this comment

knikure left a comment

Choose a reason for hiding this comment

sagemaker-bot commented May 2, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 2, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 2, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 2, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 2, 2023

AWS CodeBuild CI Report

codecov-commenter commented May 2, 2023 • edited Loading

Codecov Report

knikure left a comment

Choose a reason for hiding this comment

sagemaker-bot commented May 5, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 5, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 5, 2023

AWS CodeBuild CI Report

dipanjank commented May 5, 2023

sagemaker-bot commented May 5, 2023

AWS CodeBuild CI Report

sagemaker-bot commented May 5, 2023

AWS CodeBuild CI Report

dipanjank commented Apr 29, 2023 •

edited

Loading

codecov-commenter commented May 2, 2023 •

edited

Loading