Skip to content

CI: Simplify and unify environment creation #24498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
datapythonista opened this issue Dec 30, 2018 · 4 comments · Fixed by #24632
Closed

CI: Simplify and unify environment creation #24498

datapythonista opened this issue Dec 30, 2018 · 4 comments · Fixed by #24632
Labels
CI Continuous Integration Clean
Milestone

Comments

@datapythonista
Copy link
Member

datapythonista commented Dec 30, 2018

When creating a pandas environment for the CI, we currently have two separate systems, one for Travis and one for Azure. Summarizing, those are the involved steps.

travis

ci/prep_cython_cache.sh
ci/install_travis.sh         -> locale, download/install conda, ccache, create env, setup.py develop
ci/submit_cython_cache.sh
ci/install_db_travis.sh      -> create dbs
ci/before_script_travis.sh   -> xvfb (clipboard)

azure

ci/incremental/install_miniconda.sh          -> download/install conda
ci/incremental/setup_conda_environment.sh    -> create env, locale
ci/incremental/build.sh                      -> setup.py build_ext

Except for the cython cache, I think all the rest could be unified in a single file ci/setup_env.sh that performs all the steps.

This would simplify and avoid duplicate code and inconsistencies. As well as making future changes simpler.

@datapythonista datapythonista added CI Continuous Integration Clean labels Dec 30, 2018
@saurav2608
Copy link

@datapythonista - Can I take a shot at this?

@datapythonista
Copy link
Member Author

sure, go ahead

@saurav2608
Copy link

@datapythonista - I see two different build commands. In ci/install_travis.sh the command is python setup.py develop || exit 1. And in ci/incremental/build.sh the command is python setup.py build_ext -q --inplace. What is the difference between these two?

@datapythonista
Copy link
Member Author

Afaik, setup.py develop installs the package locally (compiling and with dependencies), setup.py build_ext is just the compiling part.

Not sure about the reasoning here, may be things changed in the setuptools/pip since this was first set up. I think setup.py develop is not used much nowadays. When not using conda, pip install -e . is usually the preferred way, as pip is smarter than setuptools with dependencies. If we're using conda, I think setup.py develop should be unnecessary, but may be there is a reason I don't know in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Clean
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants