You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I tried to set the parameter parallelism_config for a SageMaker pipeline but pipeline isn't honoring this config and still starting all available to run steps.
Hi @jrevuelta-chwy, thanks for reaching out. And thanks for providing us with such detailed info which helps us a lot on reproducing the issue and investigation.
After some investigation, it turns out the parallelism_config works as expected, but it's the pipeline.upsert to be blamed. It failed to pass the parallelism_config to the pipeline.update. I guess you've reused the common name "MyPipeline" (like I did) over multiple pipelines. Thus the first time you run the code snippet above, it's actually updating an existing pipeline without properly updating the parallelism_config
I've opened a PR to fix the issue.
To unblock yourself before the fix release, please try configuring the parallelism_config in pipeline.start:
Describe the bug
I tried to set the parameter parallelism_config for a SageMaker pipeline but pipeline isn't honoring this config and still starting all available to run steps.
doc: https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-pipeline.html#build-and-manage-pipeline-execution
To reproduce
Run this script on SageMaker Studio:
and then this script in training/train.py
Expected behavior
I expected to only see one train step running concurrently.
Screenshots or logs


Attached screenshots.
System information
A description of your system. Please provide:
sagemaker 2.173.0
boto3 1.28.7
PyTorch
2.0.0 on SageMaker
3.11 and 3.10
CPU
N
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: