Skip to content

"AttributeError: 'NoneType' object has no attribute 'len'" when deploying inference component #4725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
goriri opened this issue Jun 11, 2024 · 0 comments · Fixed by #4735
Closed

Comments

@goriri
Copy link

goriri commented Jun 11, 2024

# Deploy model to an endpoint
# https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.model.Model.deploy
llm = llm_model.deploy(
  initial_instance_count=1, # number of instances
  instance_type=instance_type, # base instance type
  resources=llama3_8b_resource_config, # resource config for multi-replica
  container_startup_health_check_timeout=health_check_timeout, # 10 minutes to be able to load the model
  endpoint_name=f"llama-3-8b-instruct-{str(uuid.uuid4())}", # name needs to be unique
  endpoint_type=EndpointType.INFERENCE_COMPONENT_BASED, # needed to use resource config
)

https://github.com/aws/sagemaker-python-sdk/blame/382fde11a534b9848176994c16b15c1872acc92a/src/sagemaker/session.py#L4700

Seems like if tags is None, it won't get processed at all in the lines that argument the tags.
A simple tags = tags or [] could probably fix the issue but it seems that the tags manipulation code is copied around multiple times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants