Skip to content

Upgrade SKLearn to 1.2.1 #163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 24, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ If you want to run local integration tests, then use:

# Example
pytest test/integration --docker-base-name preprod-sklearn ``\``
--tag 1.0-1 ``\``
--tag 1.2-1 ``\``
--py-version 3 ``\``
--framework-version |FRAMEWORK_VERSION|

Expand Down Expand Up @@ -242,4 +242,4 @@ SageMaker Scikit-learn Container is licensed under the Apache 2.0 License. It is
.com, Inc. or its affiliates. All Rights Reserved. The license is available at:
http://aws.amazon.com/apache2.0/

.. |FRAMEWORK_VERSION| replace:: 1.0-1
.. |FRAMEWORK_VERSION| replace:: 1.2-1
32 changes: 0 additions & 32 deletions ci/buildspec-extension.yml

This file was deleted.

61 changes: 0 additions & 61 deletions ci/buildspec.yml

This file was deleted.

45 changes: 0 additions & 45 deletions ci/buildspec_arm.yml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -116,4 +116,4 @@ ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PYTHONIOENCODING=UTF-8 LANG=C.U
# Install Scikit-Learn
# Scikit-learn 0.20 was the last version to support Python 2.7 and Python 3.4.
# Scikit-learn now requires Python 3.6 or newer.
RUN python -m pip install --no-cache -I scikit-learn==1.0.2
RUN python -m pip install --no-cache -I scikit-learn==1.2.1
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
FROM preprod-sklearn:1.0-1
FROM preprod-sklearn:1.2-1

RUN pip freeze | grep -q 'scikit-learn==1.0.2'; \
RUN pip freeze | grep -q 'scikit-learn==1.2.1'; \
if [ $? -eq 0 ]; \
then echo 'scikit-learn version 1.0.2 requirement met'; \
else echo 'ERROR: Expected scikit-learn version is 1.0.2, check base images for scikit-learn version' && \
then echo 'scikit-learn version 1.2.1 requirement met'; \
else echo 'ERROR: Expected scikit-learn version is 1.2.1, check base images for scikit-learn version' && \
exit 1; fi

RUN pip install --upgrade --no-cache --no-deps sagemaker-scikit-learn-extension==2.5.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ The SageMaker Scikit-learn Extension Container is built in 3 steps. The first 2

The "base" Dockerfile encompass the installation of the framework and all of the dependencies needed.

Tagging scheme is based on <Scikit-learn_version>-<SageMaker_version>-cpu-py<python_version>. (e.g. 1.0-1-cpu-py3)
Tagging scheme is based on <Scikit-learn_version>-<SageMaker_version>-cpu-py<python_version>. (e.g. 1.2-1-cpu-py3)

All "final" Dockerfiles build images using base images that use the tagging scheme above.

```
docker build -t sklearn-base:1.0-1-cpu-py3 -f docker/1.0-1/base/Dockerfile.cpu .
docker build -t sklearn-base:1.2-1-cpu-py3 -f docker/1.2-1/base/Dockerfile.cpu .
```

Notice that this Dockerfile has the updated version of sklearn (1.0.2) installed.
Notice that this Dockerfile has the updated version of sklearn (1.2.1) installed.

### Step 2: Final Image

Expand All @@ -38,7 +38,7 @@ python setup.py bdist_wheel
Then build the final image, like in the sagemaker-sklearn-container

```
docker build -t preprod-sklearn:1.0-1-cpu-py3 -f docker/1.0-1/final/Dockerfile.cpu .
docker build -t preprod-sklearn:1.2-1-cpu-py3 -f docker/1.2-1/final/Dockerfile.cpu .
```

### Step 3: Build the extension image for SageMaker Scikit-learn Extension Container
Expand All @@ -47,10 +47,10 @@ The "extension" Dockerfiles encompass the installation of the SageMaker Autopilo

The "extension" Dockerfiles use final images for building.

Build the third additional Dockerfile needed for SageMaker Scikit-learn Extension Container. This Dockerfile specifies a hard dependency on a certain version of scikit-learn (i.e. v1.0.2).
Build the third additional Dockerfile needed for SageMaker Scikit-learn Extension Container. This Dockerfile specifies a hard dependency on a certain version of scikit-learn (i.e. v1.2.1).

Tagging scheme is based on extension-<Scikit-learn-Extension_version>-<SageMaker_version>-cpu-py<python_version>. (e.g. extension-2.5-1-cpu-py3). Make sure the "extension" image is tagged in accordance with the `extension` (i.e. `extension-2.5-1-cpu-py3`).

```
docker build -t preprod-sklearn-extension:2.5-1-cpu-py3 -f docker/1.0-1/extension/Dockerfile.cpu .
docker build -t preprod-sklearn-extension:2.5-1-cpu-py3 -f docker/1.2-1/extension/Dockerfile.cpu .
```
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM sklearn-base:1.0-1
ENV SAGEMAKER_SKLEARN_VERSION 1.0-1
FROM sklearn-base:1.2-1
ENV SAGEMAKER_SKLEARN_VERSION 1.2-1

LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port=true

Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ retrying==1.3.3
sagemaker-containers==2.8.6.post2
sagemaker-inference==1.2.0
sagemaker-training==4.0.1
scikit-learn==1.0.2
scikit-learn==1.2.1
scipy==1.5.3
six==1.15.0
jinja2==3.0.3
Expand Down
2 changes: 1 addition & 1 deletion test/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def pytest_addoption(parser):
parser.addoption('--install-container-support', '-C', action='store_true')
parser.addoption('--docker-base-name', default='sk-learn')
parser.addoption('--region', default='us-west-2')
parser.addoption('--framework-version', default='1.0.2')
parser.addoption('--framework-version', default='1.2.1')
parser.addoption('--py-version', choices=['2', '3'], default=str(sys.version_info.major))
parser.addoption('--processor', choices=['cpu'], default='cpu')
# If not specified, will default to {framework-version}-{processor}-py{py-version}
Expand Down
Binary file modified test/resources/models/pickled-model-1/sklearn-model
Binary file not shown.
Binary file modified test/resources/models/pickled-model-2/sklearn-model
Binary file not shown.
4 changes: 2 additions & 2 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
[tox]
envlist = {py3}-sklearn{1.0},flake8
envlist = {py3}-sklearn{1.2},flake8

[flake8]
max-line-length = 120

[testenv]
deps =
sklearn1.0: scikit-learn==1.0.2
sklearn1.2: scikit-learn==1.2.1
-r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
conda_deps=
Expand Down