Skip to content

Commit 0496f42

Browse files
author
Deng
committed
change: upgrade framework versions and add gamma test env
1 parent 934fd90 commit 0496f42

15 files changed

+122
-64
lines changed

‎src/sagemaker/fw_utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@
5959
OPT_IN_ACCOUNTS_BY_REGION = {"ap-east-1": "057415533634", "me-south-1": "724002660598"}
6060
ASIMOV_OPT_IN_ACCOUNTS_BY_REGION = {"ap-east-1": "871362719292", "me-south-1": "217643126080"}
6161
DEFAULT_ACCOUNT = "520713654638"
62-
ASIMOV_PROD_ACCOUNT = "763104351884"
62+
ASIMOV_PROD_ACCOUNT = "028651357192" if os.getenv("DLC_GAMMA_TESTING") else "763104351884"
6363
ASIMOV_DEFAULT_ACCOUNT = ASIMOV_PROD_ACCOUNT
6464

6565
MERGED_FRAMEWORKS_REPO_MAP = {

‎src/sagemaker/mxnet/README.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@ The Docker images have the following dependencies installed:
2828
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
2929
| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | 9.0 | 9.0 | 9.2 | 10.0 | 10.1 |
3030
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
31-
| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | 1.16.3 | 1.14.5 | 1.17.2 |
31+
| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | 1.16.3 | 1.14.5 | 1.17.4 |
3232
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
33-
| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | 1.4.1 | 1.4.1 | 1.6.0 |
33+
| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | 1.4.1 | 1.4.1 | 1.4.1 |
3434
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
35-
| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | 2.2.4.1 | 2.2.4.1 | 2.2.4.2 |
35+
| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | 2.2.4.1 | 2.2.4.1 | 2.2.4.1 |
3636
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
3737

3838
The Docker images extend Ubuntu 16.04.

‎src/sagemaker/pytorch/README.rst

Lines changed: 40 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ SageMaker PyTorch Estimators and Models
44

55
With PyTorch Estimators and Models, you can train and host PyTorch models on Amazon SageMaker.
66

7-
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``.
7+
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``.
88

99
We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
1010

@@ -37,43 +37,45 @@ This Python version applies to both the Training Job, created by fit, and the En
3737

3838
The PyTorch Docker images have the following dependencies installed:
3939

40-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
41-
| Dependencies | pytorch 0.4.0 | pytorch 1.0.0 | pytorch 1.1.0 | pytorch 1.2.0 | pytorch 1.3.1 |
42-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
43-
| boto3 | >=1.7.35 | >=1.9.11 | 1.9.82 | 1.9.249 | 1.10.34 |
44-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
45-
| botocore | >=1.10.35 | >=1.12.11 | >= 1.12.11 | 1.12.249 | 1.13.34 |
46-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
47-
| CUDA (GPU image only) | 9.0 | 9.0 | 10.1 | 10.0 | 10.1 |
48-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
49-
| numpy | >=1.14.3 | >=1.15.2 | 1.16.4 | 1.16.4 | 1.16.4 |
50-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
51-
| Pillow | >=5.1.0 | >=5.2.0 | 6.0.0 | 5.4.1 | 6.2.1 |
52-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
53-
| pip | >=10.0.1 | >=18.0 | >=18.0 | 19.3 | 19.3.1 |
54-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
55-
| python-dateutil | >=2.7.3 | >=2.7.3 | >=2.7.3 | 2.8.0 | 2.8.0 |
56-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
57-
| retrying | >=1.3.3 | >=1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 |
58-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
59-
| s3transfer | >=0.1.13 | >=0.1.13 | >=0.1.13 | 0.2.1 | 0.2.1 |
60-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
61-
| sagemaker-containers | >=2.1.0 | >=2.1.0 | 2.4.10.post0 | 2.5.4 | 2.5.4 |
62-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
63-
| sagemaker-inference (inference only) | N/A | N/A | N/A | N/A | 1.1.2 |
64-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
65-
| sagemaker-pytorch-container | 1.0 | 1.1 | 1.2 | 1.2 | 1.3 |
66-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
67-
| setuptools | >=39.2.0 | >=40.4.3 | >=40.4.3 | 41.4.0 | 42.0.2.post20191203 |
68-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
69-
| six | >=1.11.0 | >=1.11.0 | 1.12.0 | 1.12.0 | 1.12.0 |
70-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
71-
| torch | 0.4.0 | 1.0.0 | 1.1.0 | 1.2.0 | 1.3.1 |
72-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
73-
| torchvision | 0.2.1 | 0.2.1 | 0.3.0 | 0.4.0a0+9232c4a | 0.4.2 |
74-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
75-
| Python | 2.7 or 3.5 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 |
76-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
40+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
41+
| Dependencies | pytorch 0.4.0 | pytorch 1.0.0 | pytorch 1.1.0 | pytorch 1.2.0 | pytorch 1.3.1 | pytorch 1.4.0 |
42+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
43+
| boto3 | >=1.7.35 | >=1.9.11 | 1.9.82 | 1.9.249 | 1.10.34 | 1.12.4 |
44+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
45+
| botocore | >=1.10.35 | >=1.12.11 | >= 1.12.11 | 1.12.249 | 1.13.34 | 1.15.4 |
46+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
47+
| CUDA (GPU image only) | 9.0 | 9.0 | 10.1 | 10.0 | 10.1 | 10.1 |
48+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
49+
| numpy | >=1.14.3 | >=1.15.2 | 1.16.4 | 1.16.4 | 1.16.4 | 1.16.4 |
50+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
51+
| Pillow | >=5.1.0 | >=5.2.0 | 6.0.0 | 5.4.1 | 6.2.1 | 6.2.0 |
52+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
53+
| pip | >=10.0.1 | >=18.0 | >=18.0 | 19.3 | 19.3.1 | 20.0.2 |
54+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
55+
| python-dateutil | >=2.7.3 | >=2.7.3 | >=2.7.3 | 2.8.0 | 2.8.0 | 2.8.1 |
56+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
57+
| retrying | >=1.3.3 | >=1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 |
58+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
59+
| s3transfer | >=0.1.13 | >=0.1.13 | >=0.1.13 | 0.2.1 | 0.2.1 | 0.3.3 |
60+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
61+
| sagemaker-containers | >=2.1.0 | >=2.1.0 | 2.4.10.post0 | 2.5.4 | 2.5.4 | 2.5.4 |
62+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
63+
| sagemaker-inference (inference only) | N/A | N/A | N/A | N/A | 1.1.2 | 1.1.2 |
64+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
65+
| sagemaker-pytorch-container | 1.0 | 1.1 | 1.2 | 1.2 | 1.3 | N/A |
66+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
67+
| sagemaker-pytorch-inference | N/A | N/A | N/A | N/A | N/A | 1.1.2 |
68+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
69+
| setuptools | >=39.2.0 | >=40.4.3 | >=40.4.3 | 41.4.0 | 42.0.2.post20191203 | N/A |
70+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
71+
| six | >=1.11.0 | >=1.11.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 |
72+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
73+
| torch | 0.4.0 | 1.0.0 | 1.1.0 | 1.2.0 | 1.3.1 | 1.4.0 |
74+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
75+
| torchvision | 0.2.1 | 0.2.1 | 0.3.0 | 0.4.0a0+9232c4a | 0.4.2 | 0.5.0 |
76+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
77+
| Python | 2.7 or 3.5 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 |
78+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
7779

7880
The Docker images extend Ubuntu 16.04.
7981

‎src/sagemaker/pytorch/defaults.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@
1818
The default version is no longer updated so as to not break existing workflows.
1919
"""
2020

21-
LATEST_VERSION = "1.3.1"
21+
LATEST_VERSION = "1.4.0"
2222
"""The latest version of PyTorch included in the SageMaker pre-built Docker images."""
2323

2424
PYTHON_VERSION = "py3"
2525

26-
LATEST_PY2_VERSION = "1.3.1"
26+
LATEST_PY2_VERSION = "1.4.0"

‎src/sagemaker/tensorflow/README.rst

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -30,41 +30,41 @@ SageMaker TensorFlow Docker containers
3030
The latest containers include the following Python packages:
3131

3232
+--------------------------------+---------------+---------------+
33-
| Dependencies | TF 1.15 | TF 2.0 |
33+
| Dependencies | TF 1.15.2 | TF 2.1 |
3434
+--------------------------------+---------------+---------------+
35-
| awscli | 1.16.296 | 1.16.303 |
35+
| awscli | 1.18.1 | 1.18.3 |
3636
+--------------------------------+---------------+---------------+
37-
| boto3 | Latest | Latest |
37+
| boto3 | 1.12.1 | 1.12.3 |
3838
+--------------------------------+---------------+---------------+
39-
| botocore | Latest | Latest |
39+
| botocore | 1.15.1 | 1.15.3 |
4040
+--------------------------------+---------------+---------------+
41-
| h5py | 2.9.0 | 2.10.0 |
41+
| h5py | 2.10.0 | 2.10.0 |
4242
+--------------------------------+---------------+---------------+
4343
| horovod | 0.18.2 | 0.18.2 |
4444
+--------------------------------+---------------+---------------+
4545
| keras | 2.3.1 | 2.3.1 |
4646
+--------------------------------+---------------+---------------+
4747
| mpi4py | 3.0.2 | 3.0.3 |
4848
+--------------------------------+---------------+---------------+
49-
| numpy | 1.17.4 | 1.17.4 |
49+
| numpy | 1.18.1 | 1.18.1 |
5050
+--------------------------------+---------------+---------------+
51-
| pandas | 0.24.2 | 0.25.3 |
51+
| pandas | 0.24.2 | 1.0.1 |
5252
+--------------------------------+---------------+---------------+
53-
| pip | 19.3.1 | 19.3.1 |
53+
| pip | 20.0.2 | 20.0.2 |
5454
+--------------------------------+---------------+---------------+
55-
| Pillow | 6.1.0 | 6.2.1 |
55+
| Pillow | 6.2.1 | 7.0.0 |
5656
+--------------------------------+---------------+---------------+
5757
| Python | 2.7 or 3.6 | 2.7 or 3.6 |
5858
+--------------------------------+---------------+---------------+
5959
| requests | 2.22.0 | 2.22.0 |
6060
+--------------------------------+---------------+---------------+
61-
| sagemaker-containers | >=2.6.1 | >=2.6.2 |
61+
| sagemaker-containers | 2.7.0 | 2.8.0 |
6262
+--------------------------------+---------------+---------------+
63-
| sagemaker-tensorflow-container | 1.15.0.1.0.0 | 2.0.0.1.0.0 |
63+
| sagemaker-tensorflow-container | 1.15.0.1.1.0 | 2.0.0.1.1.0 |
6464
+--------------------------------+---------------+---------------+
65-
| scipy | 1.2.2 | 1.2.2 |
65+
| scipy | 1.2.2 | 1.4.1 |
6666
+--------------------------------+---------------+---------------+
67-
| tensorflow | 1.15.0 | 2.0.0 |
67+
| tensorflow | 1.15.2 | 2.1.0 |
6868
+--------------------------------+---------------+---------------+
6969

7070
Script Mode TensorFlow Docker images support both Python 2.7 and Python 3.6. The Docker images extend Ubuntu 16.04.

‎src/sagemaker/tensorflow/defaults.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
This is no longer updated so as to not break existing workflows.
1919
"""
2020

21-
LATEST_VERSION = "2.0.0"
21+
LATEST_VERSION = "2.1.0"
2222
"""The latest version of TensorFlow included in the SageMaker pre-built Docker images."""
2323

24-
LATEST_PY2_VERSION = "2.0.0"
24+
LATEST_PY2_VERSION = "2.1.0"

‎src/sagemaker/tensorflow/estimator.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,12 +199,12 @@ class TensorFlow(Framework):
199199

200200
LATEST_VERSION = defaults.LATEST_VERSION
201201

202-
_LATEST_1X_VERSION = "1.15.0"
202+
_LATEST_1X_VERSION = "1.15.2"
203203

204204
_LOWEST_SCRIPT_MODE_ONLY_VERSION = [1, 13]
205205
# 2.0.0 still supports py2
206206
# we will need to update this version number if future versions still support py2
207-
_HIGHEST_PYTHON_2_VERSION = [2, 0, 0]
207+
_HIGHEST_PYTHON_2_VERSION = [2, 1, 0]
208208

209209
def __init__(
210210
self,

‎tests/conftest.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
from sagemaker.sklearn.defaults import SKLEARN_VERSION
3030
from sagemaker.tensorflow.estimator import TensorFlow
3131

32-
DEFAULT_REGION = "us-west-2"
32+
DEFAULT_REGION = "us-east-1" if os.getenv("DLC_GAMMA_TESTING") else "us-west-2"
3333

3434
NO_M4_REGIONS = ["eu-west-3", "eu-north-1", "ap-east-1", "sa-east-1", "me-south-1"]
3535

0 commit comments

Comments
 (0)