Skip to content

Commit cc1972a

Browse files
shoyershoyer
authored
Set up CI with Azure Pipelines (and remove Appveyor) (pydata#3039)
* Set up CI with Azure Pipelines For now, only duplicates our Linux pytest setup. [skip ci] * Fix image_name variable * Add Azure config for multiple platforms * Fix strategy * fix path * Try conda init * Try a variable for conda activate * Remove coverage report * Fix azure windows * Simplify conda activation * Finish porting jobs to azure * Fix template * Azure tweaks * Fix variables section * Tweak yml * Fix unit-tests.yml * Azure fixes * Tweak * Tweaking... * fix conda env * Debug * fix parameters usage * Fix conda install * Fix displayName order * Env fix-up * more minimal test matrix * Minor fixup * Add code coverage * Fix windows environments * Use --cov-append * remove HTML coverage report * Use Azure for coveralls * Add stage for finalizing test coverage * Coverage stage * Fix steps * fix coverage stage * Fix yaml * Don't use Azure for coverage * Remove Appveyor from CI * Fix flakey backend test on OS X * Fix variables used in naming jobs in test output * fix the correct test * whitespace * Try consolidating allowed failure jobs * Try an expression
1 parent a52110c commit cc1972a

9 files changed

+173
-136
lines changed

‎appveyor.yml

-38
This file was deleted.

‎azure-pipelines.yml

+97
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
variables:
2+
pytest_extra_flags: ''
3+
allow_failure: false
4+
5+
jobs:
6+
7+
- job: Linux
8+
strategy:
9+
matrix:
10+
py35-min:
11+
conda_env: py35-min
12+
py36:
13+
conda_env: py36
14+
py37:
15+
conda_env: py37
16+
py36-dask-dev:
17+
conda_env: py36-dask-dev
18+
py36-zarr-dev:
19+
conda_env: py36-zarr-dev
20+
py36-pandas-dev:
21+
conda_env: py36-pandas-dev
22+
py36-flakey:
23+
conda_env: py36
24+
pytest_extra_flags: --run-flaky --run-network-tests
25+
allow_failure: true
26+
# TODO: add numpy-dev to the test-matrix
27+
continueOnError: ${{ variables['allow_failure'] }}
28+
pool:
29+
vmImage: 'ubuntu-16.04'
30+
steps:
31+
- template: ci/azure/unit-tests.yml
32+
33+
- job: MacOSX
34+
strategy:
35+
matrix:
36+
py36:
37+
conda_env: py36
38+
pool:
39+
vmImage: 'macOS-10.13'
40+
steps:
41+
- template: ci/azure/unit-tests.yml
42+
43+
- job: Windows
44+
strategy:
45+
matrix:
46+
py36:
47+
conda_env: py36-windows
48+
py37:
49+
conda_env: py37-windows
50+
pool:
51+
vmImage: 'vs2017-win2016'
52+
steps:
53+
- template: ci/azure/unit-tests.yml
54+
55+
- job: CodeChecks
56+
variables:
57+
conda_env: py37
58+
pool:
59+
vmImage: 'ubuntu-16.04'
60+
steps:
61+
- template: ci/azure/install.yml
62+
- bash: |
63+
source activate test_env
64+
flake8
65+
displayName: flake8 lint checks
66+
- bash: |
67+
source activate test_env
68+
mypy .
69+
displayName: mypy type checks
70+
# allow mypy failures until we have a nice way of displaying results in
71+
# pull requests
72+
continueOnError: true
73+
74+
- job: Docs
75+
pool:
76+
vmImage: 'ubuntu-16.04'
77+
steps:
78+
- template: ci/azure/install.yml
79+
parameters:
80+
env_file: doc/environment.yml
81+
- bash: |
82+
source activate test_env
83+
cd doc
84+
sphinx-build -n -j auto -b html -d _build/doctrees . _build/html
85+
displayName: Build HTML docs
86+
87+
- job: LinuxHypothesis
88+
variables:
89+
conda_env: py36-hypothesis
90+
pool:
91+
vmImage: 'ubuntu-16.04'
92+
steps:
93+
- template: ci/azure/install.yml
94+
- bash: |
95+
source activate test_env
96+
pytest properties
97+
displayName: Property based tests

‎ci/azure/add-conda-to-path.yml

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# https://docs.microsoft.com/en-us/azure/devops/pipelines/languages/anaconda
2+
steps:
3+
4+
- bash: |
5+
echo "##vso[task.prependpath]$CONDA/bin"
6+
displayName: Add conda to PATH (Linux)
7+
condition: eq(variables['Agent.OS'], 'Linux')
8+
9+
- bash: |
10+
echo "##vso[task.prependpath]$CONDA/bin"
11+
sudo chown -R $USER $CONDA
12+
displayName: Add conda to PATH (OS X)
13+
condition: eq(variables['Agent.OS'], 'Darwin')
14+
15+
- powershell: |
16+
Write-Host "##vso[task.prependpath]$env:CONDA\Scripts"
17+
displayName: Add conda to PATH (Windows)
18+
condition: eq(variables['Agent.OS'], 'Windows_NT')

‎ci/azure/install.yml

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
parameters:
2+
env_file: ci/requirements-$CONDA_ENV.yml
3+
4+
steps:
5+
6+
- template: add-conda-to-path.yml
7+
8+
- bash: |
9+
conda env create -n test_env --file ${{ parameters.env_file }}
10+
displayName: Install dependencies
11+
12+
- bash: |
13+
source activate test_env
14+
pip install --no-deps -e .
15+
displayName: Install xarray
16+
17+
- bash: |
18+
source activate test_env
19+
conda list
20+
python xarray/util/print_versions.py
21+
displayName: Version info

‎ci/azure/unit-tests.yml

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
steps:
2+
3+
- template: install.yml
4+
5+
- bash: |
6+
source activate test_env
7+
python -OO -c "import xarray"
8+
displayName: Import xarray
9+
10+
- bash: |
11+
source activate test_env
12+
pytest xarray --junitxml=junit/test-results.xml \
13+
--cov=xarray --cov-config=ci/.coveragerc --cov-report=xml $EXTRA_FLAGS
14+
displayName: Run tests
15+
16+
# TODO: publish coverage results to Azure, once we can merge them across
17+
# multiple jobs: https://stackoverflow.com/questions/56776185
18+
19+
- bash: |
20+
source activate test_env
21+
COVERALLS_REPO_TOKEN=$(coverallsToken) COVERALLS_PARALLEL=true coveralls
22+
condition: succeededOrFailed()
23+
displayName: Upload coverage to coveralls
24+
25+
- task: PublishTestResults@2
26+
condition: succeededOrFailed()
27+
inputs:
28+
testResultsFiles: '**/test-*.xml'
29+
failTaskOnFailedTests: true
30+
testRunTitle: 'Test results for Python $(conda_env) $(pytest_extra_flags)'

‎ci/install_python.ps1

-97
This file was deleted.

‎ci/requirements-py36-windows.yml

+2
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,9 @@ dependencies:
1111
- matplotlib
1212
- netcdf4
1313
- pytest
14+
- pytest-cov
1415
- pytest-env
16+
- coveralls
1517
- numpy>=1.12
1618
- pandas>=0.19
1719
- scipy

‎ci/requirements-py37-windows.yml

+2
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@ dependencies:
1212
- matplotlib
1313
- netcdf4
1414
- pytest
15+
- pytest-cov
1516
- pytest-env
17+
- coveralls
1618
- numpy>=1.12
1719
- pandas>=0.19
1820
- scipy

‎xarray/tests/test_backends.py

+3-1
Original file line numberDiff line numberDiff line change
@@ -867,7 +867,9 @@ def test_encoding_same_dtype(self):
867867
ds = Dataset({'x': ('y', np.arange(10.0, dtype='f4'))})
868868
kwargs = dict(encoding={'x': {'dtype': 'f4'}})
869869
with self.roundtrip(ds, save_kwargs=kwargs) as actual:
870-
assert actual.x.encoding['dtype'] == 'f4'
870+
encoded_dtype = actual.x.encoding['dtype']
871+
# On OS X, dtype sometimes switches endianness for unclear reasons
872+
assert encoded_dtype.kind == 'f' and encoded_dtype.itemsize == 4
871873
assert ds.x.encoding == {}
872874

873875
def test_append_write(self):

0 commit comments

Comments
 (0)