Increasing minimum cython version silently breaks build_ext --inplace

[toobaz]

Code Sample, a copy-pastable example if possible

pietro@ecomp:~/nobackup/repo/pandas_test$ python3 setup.py build_ext --inplace
running build_ext

In [1]: import pandas as pd
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/home/pietro/nobackup/repo/pandas_test/pandas/__init__.py in <module>()
     25 try:
---> 26     from pandas._libs import (hashtable as _hashtable,
     27                              lib as _lib,

/home/pietro/nobackup/repo/pandas_test/pandas/_libs/__init__.py in <module>()
      3
----> 4 from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime
      5

/home/pietro/nobackup/repo/pandas_test/pandas/_libs/tslibs/conversion.pxd in init pandas._libs.tslib (pandas/_libs/tslib.c:19007)()
     10
---> 11 cdef class _TSObject:
     12     cdef:

/home/pietro/nobackup/repo/pandas_test/pandas/_libs/tslibs/__init__.py in <module>()
      3
----> 4 from .conversion import normalize_date, localize_pydatetime, tz_convert_single
      5 from .nattype import NaT, iNaT

ImportError: cannot import name 'normalize_date'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-1-af55e7023913> in <module>()
----> 1 import pandas as pd

/home/pietro/nobackup/repo/pandas_test/pandas/__init__.py in <module>()
     33                       "pandas from the source directory, you may need to run "
     34                       "'python setup.py build_ext --inplace --force' to build "
---> 35                       "the C extensions first.".format(module))
     36
     37 from datetime import datetime

ImportError: C extension: 'normalize_date' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.

Problem description

When the minimum required Cython version is increased (above the installed one), but there is a version of the compiled code available, python setup.py build_ext --inplace silently skips compilation.

It should be at least emit a warning that the available Cython version is too old.

Expected Output

A warning when running python setup.py build_ext --inplace.

The above is "fixed" if I remove the check for the cython >= '0.28.2'.

Output of pd.show_versions()

(after disabling the check for the minimum Cython version, and re-running python setup.py build_ext --inplace )

INSTALLED VERSIONS

commit: 5cb5880
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-6-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.24.0.dev0+269.g5cb5880d8.dirty
pytest: 3.0.6
pip: 9.0.1
setuptools: 33.1.1
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.18.1
pyarrow: None
xarray: None
IPython: 5.2.2
sphinx: None
patsy: 0.4.1+dev
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.2.0
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: None
lxml: 3.7.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[WillAyd]

Does adding --force not help?

[toobaz]

Does adding --force not help?

Unfortunately not. Doing a clean might instead, not tested.

[jbrockmendel]

@toobaz has this resolved itself? not clear how this is actionable

[jbrockmendel]

The first line in a the .c file generaetd by cython tells us /* Generated by Cython 0.29.14 */, so in principle it wouldn't be hard to check if it was created by an old cython. This does sound like something that should be checked within cythonize though. cc @scoder.

[scoder]

--force should rebuild everything. If it keeps old files, that is a bug.

Without --force, I think rebuilding something existing just because it was created by a different Cython version seems questionable, or at least not like an obvious decision, to me. CPython's distutils won't do this either when you update your Python patchlevel version, or switch between differently old virtualenvs.

[mroeschke]

Let us know if you are still experiencing this issue, and happy to reopen if there are still issues.

We have bumped up our min Cython version to 0.29.16 since then #33334