Skip to content

infra: properly fail PR build if has-matching-changes fails #1266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jan 28, 2020
15 changes: 2 additions & 13 deletions buildspec-localmodetests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,5 @@ phases:

# local mode tests
- start_time=`date +%s`
- HAS_MATCHING_CHANGES_OUTPUT=$(has-matching-changes "tests/" "src/*.py" "setup.py" "setup.cfg" "buildspec.yml")
- |
if [ "$HAS_MATCHING_CHANGES" = "Changes Found" ] ; then
tox -e py36 -- tests/integ -m local_mode --durations 50
fi
- ./ci-scripts/displaytime.sh 'py36 local mode' $start_time

- start_time=`date +%s`
- |
if [ "$HAS_MATCHING_CHANGES" = "Changes Found" ] ; then
tox -e py27 -- tests/integ -m local_mode --durations 50
fi
- ./ci-scripts/displaytime.sh 'py27 local mode' $start_time
- execute-command-if-has-matching-changes "tox -e py27,py36 -- tests/integ -m local_mode --durations 50" "tests/" "src/*.py" "setup.py" "setup.cfg" "buildspec.yml"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(optional) I'd be tempted to save the command in a variable to make it look a little cleaner

- ./ci-scripts/displaytime.sh 'py27,py36 local mode' $start_time
6 changes: 1 addition & 5 deletions buildspec-notebooktests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,5 @@ phases:
# run notebook test
- echo "running notebook test"
- start_time=`date +%s`
- HAS_MATCHING_CHANGES_OUTPUT=$(has-matching-changes "src/*.py" "setup.py" "setup.cfg" "tests/scripts/run-notebook-test.sh")
- |
if [ "$HAS_MATCHING_CHANGES" = "Changes Found" ] ; then
./tests/scripts/run-notebook-test.sh
fi
- execute-command-if-has-matching-changes "./tests/scripts/run-notebook-test.sh" "src/*.py" "setup.py" "setup.cfg" "tests/scripts/run-notebook-test.sh"
- ./ci-scripts/displaytime.sh 'notebook test' $start_time
10 changes: 2 additions & 8 deletions buildspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,12 @@ phases:

# run integration tests
- start_time=`date +%s`
- HAS_MATCHING_CHANGES_OUTPUT=$(has-matching-changes "tests/" "src/*.py" "setup.py" "setup.cfg" "buildspec.yml")
- |
if [ "$HAS_MATCHING_CHANGES" = "Changes Found" ] ; then
python3 -u ci-scripts/queue_build.py
fi
- execute-command-if-has-matching-changes "python3 -u ci-scripts/queue_build.py" "tests/" "src/*.py" "setup.py" "setup.cfg" "buildspec.yml"
- ./ci-scripts/displaytime.sh 'build queue' $start_time

- start_time=`date +%s`
- |
if [ "$HAS_MATCHING_CHANGES" = "Changes Found" ] ; then
tox -e py36 -- tests/integ -m "not local_mode" -n 512 --reruns 3 --reruns-delay 5 --durations 50 --boto-config '{"region_name": "us-east-2"}'
fi
execute-command-if-has-matching-changes "tox -e py36 -- tests/integ -m \"not local_mode\" -n 512 --reruns 3 --reruns-delay 5 --durations 50 --boto-config '{\"region_name\": \"us-east-2\"}'" "tests/" "src/*.py" "setup.py" "setup.cfg" "buildspec.yml"
- ./ci-scripts/displaytime.sh 'py36 tests/integ' $start_time

post_build:
Expand Down
2 changes: 1 addition & 1 deletion doc/using_chainer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -509,7 +509,7 @@ The ChainerModel constructor takes the following arguments:
which should be executed as the entry point to model hosting.
- ``source_dir (str):`` Optional. Path (absolute or relative) to a
directory with any other training source code dependencies including
tne entry point file. Structure within this directory will be
the entry point file. Structure within this directory will be
preserved when training on SageMaker.
- ``enable_cloudwatch_metrics (boolean):`` Optional. If true, training
and hosting containers will generate Cloudwatch metrics under the
Expand Down
2 changes: 1 addition & 1 deletion doc/using_mxnet.rst
Original file line number Diff line number Diff line change
Expand Up @@ -720,7 +720,7 @@ The MXNetModel constructor takes the following arguments:
which should be executed as the entry point to model hosting.
- ``source_dir (str):`` Optional. Path (absolute or relative) to a
directory with any other training source code dependencies including
tne entry point file. Structure within this directory will be
the entry point file. Structure within this directory will be
preserved when training on SageMaker.
- ``container_log_level (int):`` Log level to use within the container.
Valid values are defined in the Python logging module.
Expand Down
2 changes: 1 addition & 1 deletion doc/using_sklearn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,7 @@ The sklearn_model constructor takes the following arguments:
which should be executed as the entry point to model hosting.
- ``source_dir (str):`` Optional. Path (absolute or relative) to a
directory with any other training source code dependencies including
tne entry point file. Structure within this directory will be
the entry point file. Structure within this directory will be
preserved when training on SageMaker.
- ``enable_cloudwatch_metrics (boolean):`` Optional. If true, training
and hosting containers will generate Cloudwatch metrics under the
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker/chainer/estimator.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def __init__(
NCCL_DEBUG=WARN' will pass that option string to the mpirun
command.
source_dir (str): Path (absolute or relative) to a directory with
any other training source code dependencies aside from tne entry
any other training source code dependencies aside from the entry
point file (default: None). Structure within this directory are
preserved when training on Amazon SageMaker.
hyperparameters (dict): Hyperparameters that will be used for
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker/mxnet/estimator.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ def __init__(
file which should be executed as the entry point to training.
This should be compatible with either Python 2.7 or Python 3.5.
source_dir (str): Path (absolute or relative) to a directory with
any other training source code dependencies aside from tne entry
any other training source code dependencies aside from the entry
point file (default: None). Structure within this directory are
preserved when training on Amazon SageMaker.
hyperparameters (dict): Hyperparameters that will be used for
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker/pytorch/estimator.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ def __init__(
file which should be executed as the entry point to training.
This should be compatible with either Python 2.7 or Python 3.5.
source_dir (str): Path (absolute or relative) to a directory with
any other training source code dependencies aside from tne entry
any other training source code dependencies aside from the entry
point file (default: None). Structure within this directory are
preserved when training on Amazon SageMaker.
hyperparameters (dict): Hyperparameters that will be used for
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker/rl/estimator.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ def create_model(
hosting. This should be compatible with Python 3.5 (default:
self.entry_point)
source_dir (str): Path (absolute or relative) to a directory with
any other training source code dependencies aside from tne entry
any other training source code dependencies aside from the entry
point file (default: self.source_dir). Structure within this
directory are preserved when hosting on Amazon SageMaker.
dependencies (list[str]): A list of paths to directories (absolute
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker/sklearn/estimator.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def __init__(
executing your model training code. List of supported versions
https://github.com/aws/sagemaker-python-sdk#sklearn-sagemaker-estimators
source_dir (str): Path (absolute or relative) to a directory with
any other training source code dependencies aside from tne entry
any other training source code dependencies aside from the entry
point file (default: None). Structure within this directory are
preserved when training on Amazon SageMaker.
hyperparameters (dict): Hyperparameters that will be used for
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker/xgboost/estimator.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ def __init__(
training code. List of supported versions
https://github.com/aws/sagemaker-python-sdk#xgboost-sagemaker-estimators
source_dir (str): Path (absolute or relative) to a directory with any other training
source code dependencies aside from tne entry point file (default: None).
source code dependencies aside from the entry point file (default: None).
Structure within this directory are preserved when training on Amazon SageMaker.
hyperparameters (dict): Hyperparameters that will be used for training (default: None).
The hyperparameters are made accessible as a dict[str, str] to the training code
Expand Down