Skip to content

CI: Use conda compiler in environment.yml #27062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 20 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ci/deps/azure-35-compat.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ channels:
dependencies:
- beautifulsoup4=4.6.0
- bottleneck=1.2.1
- ccache
- cxx-compiler
- jinja2=2.8
- numexpr=2.6.2
- numpy=1.13.3
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/azure-36-locale.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ channels:
dependencies:
- beautifulsoup4==4.6.0
- bottleneck=1.2.*
- ccache
- cxx-compiler
- cython=0.28.2
- lxml
- matplotlib=2.2.2
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/azure-36-locale_slow.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ channels:
- conda-forge
dependencies:
- beautifulsoup4
- ccache
- cxx-compiler
- cython>=0.28.2
- gcsfs
- html5lib
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/azure-37-locale.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ channels:
- conda-forge
dependencies:
- beautifulsoup4
- ccache
- cxx-compiler
- cython>=0.28.2
- html5lib
- ipython
Expand Down
3 changes: 3 additions & 0 deletions ci/deps/azure-37-numpydev.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
name: pandas-dev
channels:
- defaults
- conda-forge
dependencies:
- python=3.7.*
- pytz
- ccache
- cxx-compiler
- Cython>=0.28.2
# universal
# pytest < 5 until defaults has pytest-xdist>=1.29.0
Expand Down
3 changes: 3 additions & 0 deletions ci/deps/azure-macos-35.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
name: pandas-dev
channels:
- defaults
- conda-forge
dependencies:
- beautifulsoup4
- bottleneck
- ccache
- cxx-compiler
- cython>=0.28.2
- html5lib
- jinja2
Expand Down
1 change: 1 addition & 0 deletions ci/deps/azure-windows-36.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ dependencies:
- blosc
- bottleneck
- boost-cpp<1.67
- cxx-compiler
- fastparquet>=0.2.1
- matplotlib=3.0.2
- numexpr
Expand Down
1 change: 1 addition & 0 deletions ci/deps/azure-windows-37.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ channels:
- defaults
- conda-forge
dependencies:
- cxx-compiler
- beautifulsoup4
- bottleneck
- gcsfs
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/travis-36-cov.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ channels:
dependencies:
- beautifulsoup4
- botocore>=1.11
- ccache
- cxx-compiler
- cython>=0.28.2
- dask
- fastparquet>=0.2.1
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/travis-36-locale.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ dependencies:
- beautifulsoup4
- blosc=1.14.3
- python-blosc
- ccache
- cxx-compiler
- cython>=0.28.2
- fastparquet=0.2.1
- gcsfs=0.2.2
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/travis-36-slow.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ channels:
- conda-forge
dependencies:
- beautifulsoup4
- ccache
- cxx-compiler
- cython>=0.28.2
- html5lib
- lxml
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/travis-37.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ channels:
dependencies:
- python=3.7.*
- botocore>=1.11
- ccache
- cxx-compiler
- cython>=0.28.2
- numpy
- python-dateutil
Expand Down
64 changes: 41 additions & 23 deletions ci/setup_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -56,28 +56,6 @@ conda update -n base conda
echo "conda info -a"
conda info -a

echo
echo "set the compiler cache to work"
if [ -z "$NOCACHE" ] && [ "${TRAVIS_OS_NAME}" == "linux" ]; then
echo "Using ccache"
export PATH=/usr/lib/ccache:/usr/lib64/ccache:$PATH
GCC=$(which gcc)
echo "gcc: $GCC"
CCACHE=$(which ccache)
echo "ccache: $CCACHE"
export CC='ccache gcc'
elif [ -z "$NOCACHE" ] && [ "${TRAVIS_OS_NAME}" == "osx" ]; then
echo "Install ccache"
brew install ccache > /dev/null 2>&1
echo "Using ccache"
export PATH=/usr/local/opt/ccache/libexec:$PATH
gcc=$(which gcc)
echo "gcc: $gcc"
CCACHE=$(which ccache)
echo "ccache: $CCACHE"
else
echo "Not using ccache"
fi

echo "source deactivate"
source deactivate
Expand All @@ -103,6 +81,46 @@ fi
echo "activate pandas-dev"
source activate pandas-dev

echo
echo "set the compiler cache to work"
if [ -z "$NOCACHE" ] && [ "${TRAVIS_OS_NAME}" == "linux" ]; then
echo "Install ccache"
conda install ccache -y
echo "Using ccache"
ccache --show-stats
ccache --version

if [ -z "$GCC" ]; then
GCC=$(which gcc)
fi
echo "gcc: $GCC"
CCACHE=ccache
echo "ccache: $CCACHE"
export CC="ccache $CC"
elif [ -z "$NOCACHE" ] && [ "${TRAVIS_OS_NAME}" == "osx" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this conditional branch be combined with the one above? Seems like mostly copy / paste save one off things so would be easier to read if consolidated

echo "Install ccache"
conda install ccache -y
echo "Using ccache"
ccache --show-stats
ccache --version
gcc=$CC
if [ -z "$CC" ]; then
gcc=$(which gcc)
fi
echo "gcc: $gcc"
CCACHE=ccache
echo "ccache: $CCACHE"
else
echo "Not using ccache"
if [ -z "$CC" ]; then
GCC=$(which gcc)
else
gcc="${CC}"
GCC="${CC}"
fi
echo "gcc: $gcc"
fi

echo
echo "remove any installed pandas package"
echo "w/o removing anything else"
Expand All @@ -116,7 +134,7 @@ conda list pandas
# Make sure any error below is reported as such

echo "Build extensions and install pandas"
python setup.py build_ext -q --inplace
python setup.py build_ext -q --inplace --compiler=unix -I ${CONDA_PREFIX}/include/ -L ${CONDA_PREFIX}/lib/ -R ${CONDA_PREFIX}/lib/
python -m pip install -e .

echo
Expand Down
4 changes: 4 additions & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ dependencies:

# building
- cython>=0.28.2
# This conda-forge package is necessary to install the compilers for each platform.
# That is useful to isolate the compilation from the system and to avoid any kinda of
# problem related to systems configuration
- cxx-compiler

# code checks
- black
Expand Down