Feature/new fg utils

.pylintrc

@@ -94,6 +94,7 @@ disable=
     useless-object-inheritance, # TODO: Enable this check and fix code once Python 2 is no longer supported.
     super-with-arguments,
     raise-missing-from,
+    E1136 # typing unsubscriptable-oject
 
 [REPORTS]
 # Set the output format. Available formats are text, parseable, colorized, msvs

CHANGELOG.md

@@ -25,6 +25,7 @@
 
  * Correct SageMaker Clarify API docstrings by changing JSONPath to JMESPath
 
+
 ## v2.124.0 (2022-12-16)
 
 ### Features

doc/algorithms/text/machine_translation_hugging_face.rst

@@ -8,3 +8,39 @@ This is a supervised machine translation algorithm which supports many pre-train
 demonstrates how to use the Sagemaker Python SDK for Machine Translation for using these algorithms.
 
 For detailed documentation please refer :ref:`Use Built-in Algorithms with Pre-trained Models in SageMaker Python SDK <built-in-algos>`.
+
+.. list-table:: Available Models
+   :widths: 50 20 20 20 20
+   :header-rows: 1
+   :class: datatable
+
+   * - Model ID
+     - Fine Tunable?
+     - Latest Version
+     - Min SDK Version
+     - Source
+   * - huggingface-translation-opus-mt-en-es
+     - False
+     - 1.1.0
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/Helsinki-NLP/opus-mt-en-es>`__
+   * - huggingface-translation-opus-mt-en-vi
+     - False
+     - 1.1.0
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/Helsinki-NLP/opus-mt-en-vi>`__
+   * - huggingface-translation-t5-base
+     - False
+     - 1.1.0
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/t5-base>`__
+   * - huggingface-translation-t5-large
+     - False
+     - 1.1.0
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/t5-large>`__
+   * - huggingface-translation-t5-small
+     - False
+     - 1.1.0
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/t5-small>`__

doc/algorithms/text/question_answering_pytorch.rst

@@ -7,3 +7,104 @@ This is a supervised question answering algorithm which supports fine-tuning of
 demonstrates how to use the Sagemaker Python SDK for Question Answering for using these algorithms.
 
 For detailed documentation please refer :ref:`Use Built-in Algorithms with Pre-trained Models in SageMaker Python SDK <built-in-algos>`
+
+.. list-table:: Available Models
+   :widths: 50 20 20 20 20
+   :header-rows: 1
+   :class: datatable
+
+   * - Model ID
+     - Fine Tunable?
+     - Latest Version
+     - Min SDK Version
+     - Source
+   * - pytorch-eqa-bert-base-cased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-base-multilingual-cased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-base-multilingual-uncased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-base-uncased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-large-cased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-large-cased-whole-word-masking
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-large-cased-whole-word-masking-finetuned-squad
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-large-uncased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-large-uncased-whole-word-masking
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-bert-large-uncased-whole-word-masking-finetuned-squad
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-distilbert-base-cased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-distilbert-base-multilingual-cased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-distilbert-base-uncased
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-distilroberta-base
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-roberta-base
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-roberta-base-openai-detector
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-roberta-large
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__
+   * - pytorch-eqa-roberta-large-openai-detector
+     - True
+     - 1.2.1
+     - 2.75.0
+     - `Pytorch Hub <https://pytorch.org/hub/huggingface_pytorch-transformers/>`__

doc/algorithms/text/sentence_pair_classification_hugging_face.rst

@@ -7,3 +7,119 @@ This is a supervised sentence pair classification algorithm which supports fine-
 demonstrates how to use the Sagemaker Python SDK for Sentence Pair Classification for using these algorithms.
 
 For detailed documentation please refer `Use Built-in Algorithms with Pre-trained Models in SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html#use-built-in-algorithms-with-pre-trained-models-in-sagemaker-python-sdk>`__
+
+.. list-table:: Available Models
+   :widths: 50 20 20 20 20
+   :header-rows: 1
+   :class: datatable
+
+   * - Model ID
+     - Fine Tunable?
+     - Latest Version
+     - Min SDK Version
+     - Source
+   * - huggingface-spc-bert-base-cased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-base-cased>`__
+   * - huggingface-spc-bert-base-multilingual-cased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-base-multilingual-cased>`__
+   * - huggingface-spc-bert-base-multilingual-uncased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-base-multilingual-uncased>`__
+   * - huggingface-spc-bert-base-uncased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-base-uncased>`__
+   * - huggingface-spc-bert-large-cased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-large-cased>`__
+   * - huggingface-spc-bert-large-cased-whole-word-masking
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-large-cased-whole-word-masking>`__
+   * - huggingface-spc-bert-large-uncased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-large-uncased>`__
+   * - huggingface-spc-bert-large-uncased-whole-word-masking
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/bert-large-uncased-whole-word-masking>`__
+   * - huggingface-spc-distilbert-base-cased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/distilbert-base-cased>`__
+   * - huggingface-spc-distilbert-base-multilingual-cased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/distilbert-base-multilingual-cased>`__
+   * - huggingface-spc-distilbert-base-uncased
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/distilbert-base-uncased>`__
+   * - huggingface-spc-distilroberta-base
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/distilroberta-base>`__
+   * - huggingface-spc-roberta-base
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/roberta-base>`__
+   * - huggingface-spc-roberta-base-openai-detector
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/roberta-base-openai-detector>`__
+   * - huggingface-spc-roberta-large
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/roberta-large>`__
+   * - huggingface-spc-roberta-large-openai-detector
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/roberta-large-openai-detector>`__
+   * - huggingface-spc-xlm-clm-ende-1024
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/xlm-clm-ende-1024>`__
+   * - huggingface-spc-xlm-mlm-ende-1024
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/xlm-mlm-ende-1024>`__
+   * - huggingface-spc-xlm-mlm-enro-1024
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/xlm-mlm-enro-1024>`__
+   * - huggingface-spc-xlm-mlm-tlm-xnli15-1024
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/xlm-mlm-tlm-xnli15-1024>`__
+   * - huggingface-spc-xlm-mlm-xnli15-1024
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `HuggingFace <https://huggingface.co/xlm-mlm-xnli15-1024>`__

doc/algorithms/text/sentence_pair_classification_tensorflow.rst

@@ -7,3 +7,64 @@ This is a supervised sentence pair classification algorithm which supports fine-
 demonstrates how to use the Sagemaker Python SDK for Sentence Pair Classification for using these algorithms.
 
 For detailed documentation please refer `Use Built-in Algorithms with Pre-trained Models in SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html#use-built-in-algorithms-with-pre-trained-models-in-sagemaker-python-sdk>`__
+
+.. list-table:: Available Models
+   :widths: 50 20 20 20 20
+   :header-rows: 1
+   :class: datatable
+
+   * - Model ID
+     - Fine Tunable?
+     - Latest Version
+     - Min SDK Version
+     - Source
+   * - tensorflow-spc-bert-en-cased-L-12-H-768-A-12-2
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/tensorflow/bert_en_cased_L-12_H-768_A-12/2>`__
+   * - tensorflow-spc-bert-en-uncased-L-12-H-768-A-12-2
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2>`__
+   * - tensorflow-spc-bert-en-uncased-L-24-H-1024-A-16-2
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/tensorflow/bert_en_uncased_L-24_H-1024_A-16/2>`__
+   * - tensorflow-spc-bert-en-wwm-cased-L-24-H-1024-A-16-2
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/tensorflow/bert_en_wwm_cased_L-24_H-1024_A-16/2>`__
+   * - tensorflow-spc-bert-en-wwm-uncased-L-24-H-1024-A-16-2
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/tensorflow/bert_en_wwm_uncased_L-24_H-1024_A-16/2>`__
+   * - tensorflow-spc-bert-multi-cased-L-12-H-768-A-12-2
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/tensorflow/bert_multi_cased_L-12_H-768_A-12/2>`__
+   * - tensorflow-spc-electra-base-1
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/google/electra_base/1>`__
+   * - tensorflow-spc-electra-small-1
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/google/electra_small/1>`__
+   * - tensorflow-spc-experts-bert-pubmed-1
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/google/experts/bert/pubmed/1>`__
+   * - tensorflow-spc-experts-bert-wiki-books-1
+     - True
+     - 1.2.3
+     - 2.75.0
+     - `Tensorflow Hub <https://tfhub.dev/google/experts/bert/wiki_books/1>`__