Skip to content
This repository was archived by the owner on May 23, 2024. It is now read-only.

Commit 1602159

Browse files
author
Chuyang Deng
committed
update readme
1 parent f3433a6 commit 1602159

File tree

1 file changed

+8
-9
lines changed

1 file changed

+8
-9
lines changed

README.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -645,23 +645,22 @@ Only 90% of the ports will be utilized and each loaded model will be allocated w
645645
For example, if the ``SAGEMAKER_SAFE_PORT_RANGE`` is between 9000 to 9999, the maximum number of models that can be loaded to the endpoint at the same time would be 499 ((9999 - 9000) * 0.9 / 2).
646646

647647
### Using Multi-Model Endpoint with Pre/Post-Processing
648-
Multi-Model Endpoint can be used together with Pre/Post-Processing. However, please note that in Multi-Model mode, the path of ``inference.py`` is ``/opt/ml/models/code`` instead of ``/opt/ml/model/code``.
649-
Also, all loaded models will share the same ``inference.py`` to handle invocation requests. An example of the directory structure of Multi-Model Endpoint and Pre/Post-Processing would look like this:
648+
Multi-Model Endpoint can be used together with Pre/Post-Processing. Each model will need its own ``inference.py`` otherwise default handlers will be used. An example of the directory structure of Multi-Model Endpoint and Pre/Post-Processing would look like this:
650649

651-
model1
650+
/opt/ml/models/model1/model
652651
|--[model_version_number]
653652
|--variables
654653
|--saved_model.pb
655-
model2
654+
/opt/ml/models/model2/model
656655
|--[model_version_number]
657656
|--assets
658657
|--variables
659658
|--saved_model.pb
660-
code
661-
|--lib
662-
|--external_module
663-
|--inference.py
664-
|--requirements.txt
659+
code
660+
|--lib
661+
|--external_module
662+
|--inference.py
663+
|--requirements.txt
665664

666665
## Contributing
667666

0 commit comments

Comments
 (0)