Skip to content

SageMaker Bring Your Own Container - on local mode - connection to S3 #3084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
idanmoradarthas opened this issue Apr 27, 2022 · 7 comments
Closed

Comments

@idanmoradarthas
Copy link

idanmoradarthas commented Apr 27, 2022

Describe the feature you'd like
During implementation of SageMaker BYOC with local mode with Python SDK, we notice that it connects to S3 (to Sagemaker object's - default bucket), places all of the artifacts in there, and then downloading it again into the local container. We thought that local mode means that no external connections are done, but that's not the case in our implementation.

How would this feature be used? Please describe.
We want local mode to be able to be completely local. So if we want to, we can indicate to it to fully work without the need to connect to web at all.

Additional context
We used this code as a reference: https://github.com/aws-samples/amazon-sagemaker-local-mode/blob/main/scikit_learn_bring_your_own_container_local_processing/scikit_learn_bring_your_own_container_local_processing.py

@alar0330
Copy link

Seems to be related: #3031

@mufaddal-rohawala
Copy link
Member

Fix released. Should be fixed in release >= v2.89.0

@clausagerskov
Copy link

@mufaddal-rohawala this is not fixed

@clausagerskov
Copy link

this bug fix listed in the release was to change a required "s3:ListAllMyBuckets" to a required "s3:ListBucket".
that does not at all address the issue here, which is fully local development

@clausagerskov
Copy link

@shreyapandit

@fahran-wallace
Copy link

I'm experiencing the same I think, using sagemaker[local]==2.222.0.

A code snippet extracted from this example: https://github.com/aws-samples/amazon-sagemaker-local-mode/blob/main/xgboost_script_mode_local_training_and_serving/xgboost_script_mode_local_training_and_serving.py

from sagemaker import TrainingInput
from sagemaker.xgboost import XGBoost, XGBoostModel
from sagemaker.local import LocalSession

DUMMY_IAM_ROLE = 'arn:aws:iam::111111111111:role/service-role/AmazonSageMaker-ExecutionRole-20200101T000001'
LOCAL_SESSION = LocalSession()
LOCAL_SESSION.config = {'local': {'local_code': True}}  # Ensure full code locality, see: https://sagemaker.readthedocs.io/en/stable/overview.html#local-mode
FRAMEWORK_VERSION = "1.7-1"

def main():
    xgb_inference_model = XGBoostModel(
        model_data="./tests/resources/models/fake/model.json",
        role=DUMMY_IAM_ROLE,
        entry_point="inference.py",
        source_dir="./src",
        framework_version=FRAMEWORK_VERSION,
        sagemaker_session=LOCAL_SESSION
    )

    print('Deploying endpoint in local mode')
    predictor = xgb_inference_model.deploy(
        initial_instance_count=1,
        instance_type="local",
    )

def test_inference_endpoint():
    main()

Expected behaviour: Model repacked locally
Actual behaviour: Failure when creating an S3 bucket during the deploy job. I...really do not want my code being uploaded to random S3 buckets!

operation_name = 'CreateBucket'
api_params = {'Bucket': 'sagemaker-eu-west-1-xxxxxxxxxxxx', 'CreateBucketConfiguration': {'LocationConstraint': 'eu-west-1'}}```





@clausagerskov
Copy link

so is this fixed then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants