Skip to content

CI: Fixing possible bugs in the CI #23727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Nov 24, 2018
Merged

CI: Fixing possible bugs in the CI #23727

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
57aca9a
Fixing possible bugs in the CI
datapythonista Nov 15, 2018
08ccb80
Moving Python 3.7 test to 3.6, as moto is failing on 3.7 at the moment
datapythonista Nov 20, 2018
cfc89a5
Testing with Italian locale (was supposed to be tested, but Chinese w…
datapythonista Nov 20, 2018
c8d20b1
Restoring -m 'not single and slow' to see if removing it caused the v…
datapythonista Nov 20, 2018
9718fd5
callng multi/slow tests in a consistent way with the rest
datapythonista Nov 21, 2018
b3f5744
Reducing verbosity of slow tests, to avoid reaching the maximum size …
datapythonista Nov 21, 2018
22b8fc2
restoring -m slow so the build passes, and added comment explaining why
datapythonista Nov 21, 2018
5afef71
Moving azure 37 build back to 3.7, and installing moto from pip, as c…
datapythonista Nov 21, 2018
6f17fe5
Filtering xlrd warnings in Python 3.7
datapythonista Nov 21, 2018
23d0c17
Merging from master
datapythonista Nov 21, 2018
d08c0a8
Merge remote-tracking branch 'upstream/master' into ci_bugs
datapythonista Nov 23, 2018
e1a6f89
Using a context manager to ignore xlrd warnings
datapythonista Nov 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion ci/azure/linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
CONDA_ENV: pandas
TEST_ARGS: "--skip-slow --skip-network"

py36_locale:
py37_locale:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In CONDA_PY we have 37, as well as in the yaml file, so I assume the name is wrong.

ENV_FILE: ci/deps/azure-37-locale.yaml
CONDA_PY: "37"
CONDA_ENV: pandas
Expand All @@ -27,6 +27,7 @@ jobs:
CONDA_PY: "36"
CONDA_ENV: pandas
TEST_ARGS: "--only-slow --skip-network"
LOCALE_OVERRIDE: "zh_CN.UTF-8"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have _locale in the name of the build, but we weren't setting LOCALE_OVERRIDE

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should be testing ZH and IT is this the missing one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure which is the build that had to use Italian, but we didn't have one for what I've seen.

Set the locale to it_IT.UTF8 here, I guess that's what we want.


steps:
- script: |
Expand Down
2 changes: 1 addition & 1 deletion ci/deps/azure-37-locale.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ dependencies:
- pymysql
- pytables
- python-dateutil
- python=3.6*
- python=3.7*
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the name of the file we say it's 37, I guess the version here is wrong.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this is breaking the installation of this build, as moto can't be installed with python 3.7

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like conda-forge/moto-feedstock#15 is failing on a dependency.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@datapythonista @TomAugspurger
Moto is supporting python 3.7. starting with 1.3.7 (the latest version), but that isn't reflected in the requirements yet: getmoto/moto#1886

It's possible to install it through pip - I did that in #23731, but then the problem is that 1.3.7 requires a boto version that will run into #23754, probably because of a moto issue: getmoto/moto#1941

- pytz
- s3fs
- scipy
Expand Down
5 changes: 3 additions & 2 deletions ci/script_multi.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ source activate pandas

if [ -n "$LOCALE_OVERRIDE" ]; then
export LC_ALL="$LOCALE_OVERRIDE";
export LANG="$LOCALE_OVERRIDE";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have this in ci/script_single.sh but not here. I don't see why in one case should be needed and not in the other. I guess it's missing here (or could be removed there).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could add this would prob be ok
but all of the locale tests are done in the single i think

echo "Setting LC_ALL to $LOCALE_OVERRIDE"

pycmd='import pandas; print("pandas detected console encoding: %s" % pandas.get_option("display.encoding"))'
Expand All @@ -32,8 +33,8 @@ elif [ "$COVERAGE" ]; then

elif [ "$SLOW" ]; then
TEST_ARGS="--only-slow --skip-network"
echo pytest -m "not single and slow" -v --durations=10 --junitxml=test-data-multiple.xml --strict $TEST_ARGS pandas
pytest -m "not single and slow" -v --durations=10 --junitxml=test-data-multiple.xml --strict $TEST_ARGS pandas
echo pytest -n 2 -m "not single" -v --durations=10 --junitxml=test-data-multiple.xml --strict $TEST_ARGS pandas
pytest -n 2 -m "not single" -v --durations=10 --junitxml=test-data-multiple.xml --strict $TEST_ARGS pandas
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in script_multi.py, so I assume for the slow tests with also want to test with two processes, like in the others.

I don't think the slow in the -m is needed, as we already have the --only-slow in TEST_ARGS that is the way we call the slow tests everywhere else (and it's handled in conftest.py

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can try but this has issues before iirc


else
echo pytest -n 2 -m "not single" --durations=10 --junitxml=test-data-multiple.xml --strict $TEST_ARGS pandas
Expand Down