Skip to content

CI: clean azure pipeline #38636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 34 commits into from
Closed

CI: clean azure pipeline #38636

Changes from 18 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4912eb1
CI: use conda from ci
Dec 22, 2020
80c751e
CI: use conda from ci
Dec 22, 2020
b0ff882
CI: use conda from ci
Dec 22, 2020
7e685e8
CI: use conda from ci
Dec 22, 2020
0371e8f
CI: use conda from ci
Dec 22, 2020
2fe866a
CI: use conda from ci
Dec 22, 2020
e54d80b
CI: use conda from ci
Dec 22, 2020
079bb27
CI: use conda from ci
Dec 22, 2020
bd1af56
CI: use conda from ci
Dec 22, 2020
f323354
CI: use conda from ci
Dec 22, 2020
7c29e8b
CI: use conda from ci
Dec 23, 2020
ea745b1
CI: use conda from ci
Dec 23, 2020
bc95cde
CI: use conda from ci
Dec 23, 2020
c78dc47
CI: use conda from ci
Dec 23, 2020
c2ee4a3
CI: use conda from ci
Dec 23, 2020
fdb081b
CI: use conda from ci
Dec 23, 2020
2be2ae7
CI: use conda from ci
Dec 23, 2020
1b50515
CI: use conda from ci
Dec 23, 2020
2a4285d
Merge remote-tracking branch 'upstream/master' into refactor-azure-ci
fangchenli Jun 11, 2021
37a287a
rebase
fangchenli Jun 11, 2021
fda6a84
fix build commend
fangchenli Jun 11, 2021
a7a01d8
fix env commend
fangchenli Jun 11, 2021
f0f9b47
fix env commend
fangchenli Jun 11, 2021
144c50d
fix env commend
fangchenli Jun 11, 2021
2c8813f
update vmimage version
fangchenli Jun 11, 2021
bde28ca
fix windows
fangchenli Jun 11, 2021
14b5cb7
fix windows
fangchenli Jun 11, 2021
61932dd
fix windows
fangchenli Jun 11, 2021
a92dbf6
str
fangchenli Jun 11, 2021
025d125
Merge remote-tracking branch 'upstream/master' into refactor-azure-ci
fangchenli Jun 12, 2021
8a61398
resolve conflict
fangchenli Jul 17, 2021
6f73e01
resolve conflict
fangchenli Jul 17, 2021
49cefe9
Merge remote-tracking branch 'upstream/master' into refactor-azure-ci
fangchenli Jul 17, 2021
2d32a50
resolve conflict
fangchenli Jul 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 32 additions & 13 deletions ci/azure/posix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,24 +73,42 @@ jobs:

steps:
- script: |
if [ "$(uname)" == "Linux" ]; then
sudo apt-get update
sudo apt-get install -y libc6-dev-i386 $EXTRA_APT
fi
sudo apt-get update
sudo apt-get install -y libc6-dev-i386 $EXTRA_APT
displayName: 'Install extra packages'
condition: eq('${{ parameters.name }}', 'Linux')

- script: echo '##vso[task.prependpath]$(HOME)/miniconda3/bin'
displayName: 'Set conda path'
- bash: echo '##vso[task.prependpath]$(HOME)/miniconda3/bin'
displayName: 'Add conda to PATH'

- script: ci/setup_env.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this is basically inlining setup_env?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we doing this? that is called by every build

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We save about 2-3 min build time by using the CI's conda. And the Windows builds are not using the script. We probably could combine all Azure CI setups into one file. Now we have one for posix and one for windows. They are almost identical.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok the problem is there are other things that setup_env is doing which this is not. not averse to this, but we need to do it inside setup_env.sh

displayName: 'Setup environment and build pandas'
- bash: sudo chown -R $USER $CONDA
displayName: Take ownership of conda installation
condition: eq('${{ parameters.name }}', 'macOS')

- script: |
source activate pandas-dev
- bash: conda update --quiet --yes conda
displayName: Update conda

- bash: conda env create --quiet --file $(ENV_FILE)
displayName: Create Anaconda environment

- bash: |
eval "$(conda shell.bash hook)"
conda activate pandas-dev
conda list
python setup.py build_ext -q -j 4
python -m pip install --no-build-isolation -e .
displayName: 'Build'

- bash: |
eval "$(conda shell.bash hook)"
conda activate pandas-dev
ci/run_tests.sh
displayName: 'Test'

- script: source activate pandas-dev && pushd /tmp && python -c "import pandas; pandas.show_versions();" && popd
- bash: |
eval "$(conda shell.bash hook)"
conda activate pandas-dev
pushd /tmp && python -c "import pandas; pandas.show_versions();" && popd
displayName: 'Build versions'

- task: PublishTestResults@2
Expand All @@ -101,7 +119,8 @@ jobs:
testRunTitle: ${{ format('{0}-$(CONDA_PY)', parameters.name) }}
displayName: 'Publish test results'

- script: |
source activate pandas-dev
- bash: |
eval "$(conda shell.bash hook)"
conda activate pandas-dev
python ci/print_skipped.py
displayName: 'Print skipped tests'