Skip to content

list_runs doesn't work twice. Raises a ValueError about the name length of the run. #3673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JanetMatsen opened this issue Feb 21, 2023 · 3 comments · Fixed by #3821
Closed
Assignees
Labels
component: experiments Relates to the SageMaker Experiements Platform type: bug

Comments

@JanetMatsen
Copy link

JanetMatsen commented Feb 21, 2023

Describe the bug
I have a dummy Sagemaker notebook where I use an experiment name BO-POC-synthetic-data.
When I list_runs for that experiment once, it works.
When I do list_runs for that experiment a second time, it makes

ValueError: The run_name (length: 73) must have length less than or equal to 59

To reproduce

from smexperiments.experiment import Experiment
from sagemaker.experiments.run import Run, load_run, list_runs

EXPERIMENT_NAME = "BO-POC-synthetic-data"

with Run(experiment_name=EXPERIMENT_NAME) as run:
    
    run.log_parameter("my_param", "AP") 
    run.log_metric("my_metric", 0.8)

    print(f"experiment name: {run.experiment_name}")
    print(run.experiment_config)

runs = list_runs(EXPERIMENT_NAME)      # <---- WORKS ONCE
runs = list_runs(EXPERIMENT_NAME)      # <---- FAILS SECOND TIME

Note that there are two trial components generated.
Conceptually, I'd only expect one trial component; why is there one with the experiment name repeated twice?
I suspect the longer one is causing the exception.
Screenshot 2023-02-21 at 2 00 35 PM

Expected behavior
I'd expect to be able to list-runs twice.

Screenshots or logs
Screenshot 2023-02-21 at 1 57 48 PM

Also, a zipped version of my Studio notebook:
JM-2023-02-21_bug_report_Sagemaker_Experiments_bug_report.html.zip

System information
A description of your system. Please provide:

  • SageMaker Python SDK version: 2.133.0
  • Framework name (eg. PyTorch) or algorithm (eg. KMeans): None
  • Framework version:
  • Python version: Python 3.9.11
  • CPU or GPU: CPU (sagemaker Studio)
  • Custom Docker image (Y/N): N

Additional context

@JanetMatsen
Copy link
Author

Here are some more examples of experiment names that pass (no exception in list_runs), and fail (ValueError in list_runs):

EXPERIMENT_NAME = "BO-POC-synthetic-data" # Fails
EXPERIMENT_NAME = "some-other-experiment-name" # works
EXPERIMENT_NAME = "BO-POC-synthetic" # Fails
EXPERIMENT_NAME = "POC-synthetic"  # works
EXPERIMENT_NAME = "O-POC-synthetic" # fails (verified twice)
EXPERIMENT_NAME = "O-POC-s" # works
EXPERIMENT_NAME = "O-POC-syn" # works
EXPERIMENT_NAME = "O-POC-synth" # works
EXPERIMENT_NAME = "O-POC-synthet" # works
EXPERIMENT_NAME = "O-POC-syntheti" # works
EXPERIMENT_NAME = "POC-synthetic" # fails. <--- the minimal failing example string I've found so far. 

@qidewenwhen qidewenwhen added the component: experiments Relates to the SageMaker Experiements Platform label Mar 2, 2023
@danabens danabens self-assigned this Apr 27, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue Apr 27, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue Apr 27, 2023
@danabens
Copy link
Member

Hi, thanks for providng detailed reproduction steps.

why does list_runs fail?

There was a bug in the list_runs behavior that resulted in a second Trial Component creation.

Conceptually, I'd only expect one trial component; why is there one with the experiment name repeated twice?

You are correct in expecting a single trial component.

Fixed in #3821

danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue Apr 27, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue Apr 27, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue Apr 28, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue Apr 28, 2023
trajanikant pushed a commit to danabens/sagemaker-python-sdk that referenced this issue May 1, 2023
@JanetMatsen
Copy link
Author

Thank you, @danabens.

trajanikant pushed a commit to danabens/sagemaker-python-sdk that referenced this issue May 4, 2023
trajanikant pushed a commit to danabens/sagemaker-python-sdk that referenced this issue May 4, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue May 9, 2023
danabens added a commit to danabens/sagemaker-python-sdk that referenced this issue May 9, 2023
@knikure knikure linked a pull request May 9, 2023 that will close this issue
9 tasks
trajanikant pushed a commit to danabens/sagemaker-python-sdk that referenced this issue May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: experiments Relates to the SageMaker Experiements Platform type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants