CI: Fixes to conda environment names #23866

charlesdong1991 · 2018-11-23T09:13:26Z

closes CLN: Fixes to conda environment names #23851
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

…to conda_env_name

charlesdong1991 · 2018-11-23T09:24:12Z

sorry, @datapythonista , i made a terrible mistake in Git, and collected all other people's commits ahead of me, so i created this one, and closed the previous PR so that other people won't get affected. sorry for this.

datapythonista · 2018-11-23T09:29:06Z

That happens very often. Not sure what people does to get into that state, but what usually fixes it is described in the first point of this post: https://datapythonista.github.io/blog/useful-git-commands.html

If you do that, you can keep using the same PR.

datapythonista

couple of things that I think need to be changed. Also, if you take a look at the CI, may give more info on what is wrong

datapythonista · 2018-11-23T09:30:35Z

ci/incremental/setup_conda_environment.cmd

@@ -11,11 +11,10 @@ call deactivate
 @rem Display root environment (for debugging)
 conda list
 @rem Clean up any left-over from a previous build
-conda remove --all -q -y -n %CONDA_ENV%
+conda remove -all -q -y -n pandas-dev


I think the removed - is needed

datapythonista · 2018-11-23T09:32:13Z

ci/incremental/setup_conda_environment.cmd


-call activate %CONDA_ENV%


I think this, and another one deleted, need to be moved to .travis.yml before those scripts are called

same comment as above

charlesdong1991 · 2018-11-23T09:36:00Z

oh! awesome post!!! @datapythonista thank you very much!! just read it!! i am looking into the CI now to debug what is going wrong there, and thanks for reviews!!

datapythonista

Added some more comments, I think with them this should work.

This issue requires a decent knowledge of how the CI works, if too complex for you, let me know and I'll finish the PR myself.

datapythonista · 2018-11-23T23:38:17Z

ci/azure/linux.yml

@@ -35,21 +35,24 @@ jobs:
        ci/incremental/install_miniconda.sh
        export PATH=$HOME/miniconda3/bin:$PATH
        echo "Setting up Conda environment"
+        source activate pandas-dev


if I'm not wrong, the environment is created in the next file, and it doesn't exist here, so it can't be activated (and it's not needed)

datapythonista · 2018-11-23T23:39:04Z

ci/azure/macos.yml

@@ -22,21 +22,24 @@ jobs:
        ci/incremental/install_miniconda.sh
        export PATH=$HOME/miniconda3/bin:$PATH
        echo "Setting up Conda environment"
+        source activate pandas-dev


same as before

ci/incremental/build.cmd

datapythonista · 2018-11-23T23:41:03Z

ci/incremental/setup_conda_environment.cmd

 @rem Scipy, CFFI, jinja2 and IPython are optional dependencies, but exercised in the test suite
-conda env create -n %CONDA_ENV% --file=ci\deps\azure-windows-%CONDA_PY%.yaml
+conda env create -n pandas-dev --file=ci\deps\azure-windows-%CONDA_PY%.yaml


you don't need the -n pandas-dev, the name of the environment is already defined in the yaml file

datapythonista · 2018-11-23T23:41:17Z

ci/incremental/setup_conda_environment.cmd


-call activate %CONDA_ENV%


same comment as above

datapythonista · 2018-11-23T23:41:40Z

ci/incremental/setup_conda_environment.sh


 echo
 echo "[create env]"
-time conda env create -q -n "${CONDA_ENV}" --file="${ENV_FILE}" || exit 1
+time conda env create -q -n pandas-dev --file="${ENV_FILE}" || exit 1


same about -n pandas-dev not needed

ci/incremental/setup_conda_environment.sh

datapythonista · 2018-11-23T23:43:53Z

ci/script_single.sh

@@ -2,7 +2,6 @@

 echo "[script_single]"

-source activate pandas



Can you remove one of the blank lines besides de source activate and leave just one

.travis.yml

datapythonista · 2018-11-24T20:28:13Z

@charlesdong1991 looks good in general, still some details that make the CI fail. Let me take a look and push couple of small fixes, and see if I can leave everything green myself. Thanks!

charlesdong1991 · 2018-11-24T20:32:07Z

@datapythonista i am so sorry that due to my limited knowledge of how CI works, i tried my best to understand what it is going on as a whole in pandas, and changed code accordingly... but it still fails... please finish the issue... sorry, i would like to help on this issue but it's bit of out of my scope... and appreciate a lot for helping me and explaining me what is going on... shall I close the PR so that you or other talented contributor can work on it?

…1991/pandas into conda_env_name_backup_backup

datapythonista · 2018-11-24T20:53:01Z

Thanks for helping with this @charlesdong1991, your changes were very useful. But as you said, and as I mentioned before, a good knowledge of the CI is needed to understand in detail when the activations need to happen in some cases.

Don't close the PR, all your changes are useful, and I pushed to the same PR the missing bits. With some luck now the CI is green, and we can merge the PR.

Thanks again for helping with this. I'm working on simplifying the CI, and once that is done I'll add information about the CI to the docs, so it's easier for contributors to understand what's going on.

charlesdong1991 · 2018-11-24T21:04:30Z

Thanks again @datapythonista , and appreciate your patience! I tried to step out my comfort zone and solved the topic that i am not very familiar with so as to learn more... but probably this is not a good place since it will also waste your precious time to review... i will be more cautious next time when picking up issues.. thanks again, and look forward to the DOC for CI... wish this time it can go green... enjoy your weekend ^^

codecov · 2018-11-24T22:09:41Z

Codecov Report

Merging #23866 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #23866      +/-   ##
==========================================
- Coverage   92.29%   92.29%   -0.01%     
==========================================
  Files         161      161              
  Lines       51497    51498       +1     
==========================================
  Hits        47530    47530              
- Misses       3967     3968       +1

Flag	Coverage Δ
#multiple	`90.69% <ø> (-0.01%)`	⬇️
#single	`42.43% <ø> (ø)`	⬆️

Impacted Files	Coverage Δ
pandas/io/pytables.py	`92.3% <0%> (-0.05%)`	⬇️
pandas/core/reshape/pivot.py	`96.55% <0%> (ø)`	⬆️
pandas/io/json/normalize.py	`96.87% <0%> (ø)`	⬆️
pandas/io/packers.py	`88.08% <0%> (ø)`	⬆️
pandas/core/util/hashing.py	`98.4% <0%> (ø)`	⬆️
pandas/core/algorithms.py	`95.11% <0%> (ø)`	⬆️
pandas/util/_decorators.py	`91.34% <0%> (ø)`	⬆️
pandas/tseries/offsets.py	`96.98% <0%> (ø)`	⬆️
pandas/core/indexes/datetimes.py	`96.2% <0%> (ø)`	⬆️
pandas/core/indexes/multi.py	`95.49% <0%> (ø)`	⬆️
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5c90e5...fbde055. Read the comment docs.

datapythonista

All green now, thanks @charlesdong1991

@jreback do you mind taking a look?

jreback · 2018-11-25T02:52:16Z

thanks @charlesdong1991

charlesdong1991 added 16 commits November 22, 2018 16:40

change pandas to pandas-dev

88bc2b7

add source activate

fc88e2b

remove source activate

fb34dcc

clean the script

04e721c

debug

cd81131

debug

463be05

add pandas-dev back at two lines

6efdedd

change pandas to pandas-dev

24bc4f6

add source activate

28a3ebf

remove source activate

9f56258

clean the script

bf6bca2

debug

2c43b2f

debug

3a2a193

add pandas-dev back at two lines

bacb230

test again after catching the latest master

393f754

Merge branch 'conda_env_name' of github.com:charlesdong1991/pandas in…

840c566

…to conda_env_name

datapythonista requested changes Nov 23, 2018

View reviewed changes

charlesdong1991 added 10 commits November 23, 2018 11:02

debugging

d969685

debugging

a0a978a

debugging

e024ba0

debugging

26df0d4

debugging

53f8eaa

debugging

de8c1bc

debugging

3b6d646

debugging

1d9e93a

debugging

e3c9fc1

debugging

603bd0b

charlesdong1991 added 4 commits November 23, 2018 13:39

debugging

8a37dd0

debugging

e59c787

debugging

4b0b03e

debugging

45e5f06

jreback added the CI Continuous Integration label Nov 23, 2018

datapythonista requested changes Nov 23, 2018

View reviewed changes

charlesdong1991 added 3 commits November 24, 2018 20:27

debugging

1c4ddfb

debugging

61b7f6f

debugging

997d680

charlesdong1991 and others added 4 commits November 24, 2018 21:37

debugging

49231a7

Some more fixes

a0331bf

Merge branch 'conda_env_name_backup_backup' of github.com:charlesdong…

adea341

…1991/pandas into conda_env_name_backup_backup

Removing unneeded activation

afe3b1f

datapythonista mentioned this pull request Nov 24, 2018

CI: Move CircleCI job to Azure pipelines #23821

Closed

fixing wrong environment name in circle

fbde055

datapythonista approved these changes Nov 25, 2018

View reviewed changes

jreback added this to the 0.24.0 milestone Nov 25, 2018

jreback merged commit 002ab43 into pandas-dev:master Nov 25, 2018

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

CI: Fixes to conda environment names (pandas-dev#23866)

4cf768d

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

CI: Fixes to conda environment names (pandas-dev#23866)

e3ecf04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Fixes to conda environment names #23866

CI: Fixes to conda environment names #23866

charlesdong1991 commented Nov 23, 2018 •

edited

Loading

charlesdong1991 commented Nov 23, 2018

datapythonista commented Nov 23, 2018

datapythonista left a comment

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

charlesdong1991 commented Nov 23, 2018

datapythonista left a comment

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

datapythonista Nov 23, 2018

datapythonista commented Nov 24, 2018

charlesdong1991 commented Nov 24, 2018

datapythonista commented Nov 24, 2018

charlesdong1991 commented Nov 24, 2018 •

edited

Loading

codecov bot commented Nov 24, 2018 •

edited

Loading

datapythonista left a comment

jreback commented Nov 25, 2018

CI: Fixes to conda environment names #23866

CI: Fixes to conda environment names #23866

Conversation

charlesdong1991 commented Nov 23, 2018 • edited Loading

charlesdong1991 commented Nov 23, 2018

datapythonista commented Nov 23, 2018

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charlesdong1991 commented Nov 23, 2018

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

datapythonista commented Nov 24, 2018

charlesdong1991 commented Nov 24, 2018

datapythonista commented Nov 24, 2018

charlesdong1991 commented Nov 24, 2018 • edited Loading

codecov bot commented Nov 24, 2018 • edited Loading

Codecov Report

datapythonista left a comment

Choose a reason for hiding this comment

jreback commented Nov 25, 2018

charlesdong1991 commented Nov 23, 2018 •

edited

Loading

charlesdong1991 commented Nov 24, 2018 •

edited

Loading

codecov bot commented Nov 24, 2018 •

edited

Loading