You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Opening this issue on behalf of a SageMaker customer. The customer has pre-compressed and uploaded their source_dir to S3, and wants to set requirements_file to a relative path contained in the source.
The generic Frameworks Estimator allows source_dir to be an S3 location. In this case it skips validation/upload.
Skipping source_dir validation:
If directory is an S3 URI, an UploadedCode object will be returned, but nothing will be
uploaded to S3 (this allow reuse of code already in S3).
However the Tensorflow Estimator runs a validation for requirements_file which fails if the location source_dir/requirements_file is not a valid path on the local os:
raiseValueError('Requirements file {} does not exist.'.format(requirements_file))
Seems like it would be easy to skip the local path validation if source_dir is an S3 location. Something like this could be added to _validate_requirements_file:
if source_dir.lower().startswith('s3://'):
return
I know that support for requirements.txt files is limited between the "legacy" TensorFlow container and the newer "script mode" version. But this is a small bug in the SDK which could easily be fixed.
The text was updated successfully, but these errors were encountered:
Opening this issue on behalf of a SageMaker customer. The customer has pre-compressed and uploaded their
source_dir
to S3, and wants to setrequirements_file
to a relative path contained in the source.The generic Frameworks Estimator allows
source_dir
to be an S3 location. In this case it skips validation/upload.Skipping
source_dir
validation:sagemaker-python-sdk/src/sagemaker/estimator.py
Lines 830 to 833 in 8b33a30
Skipping
source_dir
upload:sagemaker-python-sdk/src/sagemaker/fw_utils.py
Lines 143 to 144 in 8b33a30
However the Tensorflow Estimator runs a validation for
requirements_file
which fails if the locationsource_dir/requirements_file
is not a valid path on the local os:sagemaker-python-sdk/src/sagemaker/tensorflow/estimator.py
Lines 273 to 285 in 8b33a30
Seems like it would be easy to skip the local path validation if
source_dir
is an S3 location. Something like this could be added to_validate_requirements_file
:I know that support for
requirements.txt
files is limited between the "legacy" TensorFlow container and the newer "script mode" version. But this is a small bug in the SDK which could easily be fixed.The text was updated successfully, but these errors were encountered: