You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Expected behavior
The uncompressed model resources should be fetched from S3
Screenshots or logs
INFO:botocore.credentials:Found credentials from IAM Role: BaseNotebookInstanceEc2InstanceRole
sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml
sagemaker.config INFO - Not applying SDK defaults from location: /home/ec2-user/.config/sagemaker/config.yaml
INFO:botocore.credentials:Found credentials from IAM Role: BaseNotebookInstanceEc2InstanceRole
WARNING:sagemaker.jumpstart:Using model 'huggingface-llm-falcon-7b-bf16' with wildcard version identifier '*'. You can pin to version '2.0.0' for more stable results. Note that models may have different input/output signatures after a major version upgrade.
ModelBuilder: INFO: JumpStart Model ID detected.
INFO:sagemaker.jumpstart:No instance type selected for inference hosting endpoint. Defaulting to ml.g5.2xlarge.
ModelBuilder: INFO: JumpStart ID huggingface-llm-falcon-7b-bf16 is packaged with Image URI: 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.0.1-tgi1.1.0-gpu-py39-cu118-ubuntu20.04
ModelBuilder: INFO: Building for TGI JumpStart Model ID...
ModelBuilder: WARNING: 75.98389809091147 percent of docker disk space at /var/lib/docker is used. Please consider freeing up disk space or increasing the EBS volume if you are on a SageMaker Notebook.
ModelBuilder: INFO: Downloading JumpStart artifacts from S3...
{'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-east-1/huggingface-llm/huggingface-llm-falcon-7b-bf16/artifacts/inference-prepack/v1.0.1/', 'S3DataType': 'S3Prefix', 'CompressionType': 'None'}}
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
File <timed exec>:1
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/serve/builder/model_builder.py:579, in ModelBuilder.build(self, mode, role_arn, sagemaker_session)
577 if isinstance(self.model, str):
578 if self._is_jumpstart_model_id():
--> 579 return self._build_for_jumpstart()
580 if self._is_djl():
581 return self._build_for_djl()
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/serve/builder/jumpstart_builder.py:261, in JumpStart._build_for_jumpstart(self)
258 self.pysdk_model = pysdk_model
259 self.image_uri = self.pysdk_model.image_uri
--> 261 self._build_for_tgi_jumpstart()
262 else:
263 raise ValueError(
264 "JumpStart Model ID was not packaged with djl-inference or tgi-inference container."
265 )
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/serve/builder/jumpstart_builder.py:222, in JumpStart._build_for_tgi_jumpstart(self)
220 if self.mode == Mode.LOCAL_CONTAINER:
221 if not hasattr(self, "prepared_for_tgi"):
--> 222 self.prepared_for_tgi = prepare_tgi_js_resources(
223 model_path=self.model_path,
224 js_id=self.model,
225 dependencies=self.dependencies,
226 model_data=self.pysdk_model.model_data,
227 )
228 self._prepare_for_mode()
229 elif self.mode == Mode.SAGEMAKER_ENDPOINT and hasattr(self, "prepared_for_tgi"):
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/serve/model_server/tgi/prepare.py:74, in prepare_tgi_js_resources(model_path, js_id, shared_libs, dependencies, model_data)
58 """Prepare serving when a JumpStart model id is given
59
60 Args:
(...)
70
71 """
72 model_path, code_dir = _create_dir_structure(model_path)
---> 74 return _copy_jumpstart_artifacts(model_data, js_id, code_dir)
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/serve/model_server/tgi/prepare.py:22, in _copy_jumpstart_artifacts(model_data, js_id, code_dir)
19 with _tmpdir(directory=str(code_dir)) as js_model_dir:
20 # TODO: remove if block after 10/30 once everything is shifted to uncompressed
21 print(model_data)
---> 22 if model_data.endswith("tar.gz"):
23 subprocess.run(["aws", "s3", "cp", model_data, js_model_dir])
25 logger.info("Uncompressing JumpStart artifacts for faster loading...")
AttributeError: 'dict' object has no attribute 'endswith'
System information
A description of your system. Please provide:
SageMaker Python SDK version: 2.199
Framework name (eg. PyTorch) or algorithm (eg. KMeans): TGI
Framework version: Latest
Python version: 3.10
CPU or GPU: GPU
Custom Docker image (Y/N): N
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
Describe the bug
When building a JumpStart ID model with ModelBuilder, an AttributeError occurs when fetching the JumpStart resources from S3
To reproduce
Expected behavior
The uncompressed model resources should be fetched from S3
Screenshots or logs
System information
A description of your system. Please provide:
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: