change: update MME Pre/Post-Processing model and script paths #153
Conversation
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
| log.info("pip install dependencies from requirements.txt") | ||
| pip_install_cmd = "pip3 install -r {}".format(requirements_file_path) | ||
| try: | ||
| subprocess.check_call(pip_install_cmd.split()) |
There was a problem hiding this comment.
it doesn't matter here, but for future reference: https://docs.python.org/3.7/library/shlex.html#shlex.split
| def _import_handlers(self, model_name=None): | ||
| inference_script = INFERENCE_SCRIPT_PATH | ||
| if model_name: | ||
| inference_script = "/opt/ml/models/{}/model/code/inference.py".format(model_name) |
There was a problem hiding this comment.
it might make more sense to pass this string to the function so that we don't have it duplicated across the code?
| inference_script = INFERENCE_SCRIPT_PATH | ||
| if model_name: | ||
| inference_script = "/opt/ml/models/{}/model/code/inference.py".format(model_name) | ||
| spec = importlib.util.spec_from_file_location('inference', inference_script) |
There was a problem hiding this comment.
nit: use " instead of '
There was a problem hiding this comment.
Yeah, I have a separate PR to address the quotes issue: #149
Co-authored-by: Eric Johnson <[email protected]>
Co-authored-by: Eric Johnson <[email protected]>
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Issue #, if available:
In Single-Model mode, model data is downloaded to
/opt/ml/model/when container starts (BEFORE the first invocation request) arrives. Therefore, pre/post-processing scripts and modules can be imported/installed when container starts.However, in Multi-Model mode, model data is downloaded to
/opt/ml/models/{model_name}/model/when the first invocation request arrives (AFTER python service is running). Therefore, pre/post-processing scripts and modules can only be imported/installed when handling thePOST /modelsrequest.Description of changes:
/opt/ml/models/{model_name}/model/when Multi-Model mode is enabledTests:
Update integrations tests to reflect the real scenario of model data downloading through hosting platform
Also tested and verified from logs that dependencies are installed, custom handlers are used
`# Create Endpoint
endpoint_name = 'DEMO-MultiModelEndpoint-' + strftime("%Y-%m-%d-%H-%M-%S", gmtime())
print('Endpoint name: ' + endpoint_name)
create_endpoint_response = sm_client.create_endpoint(
EndpointName=endpoint_name,
EndpointConfigName=endpoint_config_name)
print('Endpoint Arn: ' + create_endpoint_response['EndpointArn'])
resp = sm_client.describe_endpoint(EndpointName=endpoint_name)
status = resp['EndpointStatus']
print("Endpoint Status: " + status)
print('Waiting for {} endpoint to be in service...'.format(endpoint_name))
waiter = sm_client.get_waiter('endpoint_in_service')
waiter.wait(EndpointName=endpoint_name)
response = runtime_sm_client.invoke_endpoint(
EndpointName=endpoint_name,
ContentType='applicaton/json',
TargetModel='clean-save.tar.gz'.format(str(i)), # clean-save.tar.gz contains inference.py and requirements.txt
Body=payload)
`
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.