Build error on OS X 10.7: tseries #77

fonnesbeck · 2011-07-21T19:32:25Z

I'm trying to build pandas on OSX 10.7, but run into an error building tseries:

building 'pandas._tseries' extension
C compiler: llvm-gcc-4.2 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -arch x86_64 -pipe

creating build/temp.macosx-10.7-x86_64-2.7
creating build/temp.macosx-10.7-x86_64-2.7/pandas
creating build/temp.macosx-10.7-x86_64-2.7/pandas/src
compile options: '-I/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
llvm-gcc-4.2: pandas/src/tseries.c
i686-apple-darwin11-llvm-gcc-4.2: pandas/src/tseries.c: No such file or directory
i686-apple-darwin11-llvm-gcc-4.2: no input files
i686-apple-darwin11-llvm-gcc-4.2: pandas/src/tseries.c: No such file or directory
i686-apple-darwin11-llvm-gcc-4.2: no input files
error: Command "llvm-gcc-4.2 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -arch x86_64 -pipe -I/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pandas/src/tseries.c -o build/temp.macosx-10.7-x86_64-2.7/pandas/src/tseries.o" failed with exit status 1

I did not get this on Snow Leopard. Running Python 2.7.1 with a current, working numpy 2.0.

wesm · 2011-07-21T19:35:37Z

You're building from head right? Do you have Cython installed? I looks like when it fails to cythonize the pyx files it should be printing an error message but it's not, should try to fix that

fonnesbeck · 2011-07-21T19:37:09Z

I have cython installed from pypi, so it may be incompatible with Lion. I am trying to build cython from github now to see if that will fix things.

I am building from your master branch, yes.

wesm · 2011-07-21T19:54:30Z

See if you can get

cython pandas/src/tseries.pyx
cython pandas/src/sparse.pyx

to work. If so then I need to figure out what's wrong with the setup.py (sigh, the disaster of distutils and python packaging)

fonnesbeck · 2011-07-21T19:57:29Z

OK, that worked, but subsequent building caused the following:

compile options: '-I/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
llvm-gcc-4.2: pandas/src/sparse.c
In file included from /Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/ndarraytypes.h:1523,
from /Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/ndarrayobject.h:17,
from /Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/arrayobject.h:14,
from pandas/src/sparse.c:237:
/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API"
/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/__multiarray_api.h:1543: warning: ‘_import_array’ defined but not used
/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/__ufunc_api.h:242: warning: ‘_import_umath’ defined but not used
In file included from /Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/ndarraytypes.h:1523,
from /Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/ndarrayobject.h:17,
from /Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/arrayobject.h:14,
from pandas/src/sparse.c:237:
/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API"
/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/__multiarray_api.h:1543: warning: ‘_import_array’ defined but not used
/Library/Python/2.7/site-packages/numpy-2.0.0.dev_e2af7b7_20110721-py2.7-macosx-10.7-intel.egg/numpy/core/include/numpy/__ufunc_api.h:242: warning: ‘_import_umath’ defined but not used
llvm-gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -Wl,-F. -arch i386 -arch x86_64 build/temp.macosx-10.7-intel-2.7/pandas/src/sparse.o -o build/lib.macosx-10.7-intel-2.7/pandas/_sparse.so

Are there any unwanted build flags in there?

fonnesbeck · 2011-07-21T20:00:16Z

Aha! Restricting the build to 64-bit seems to do the trick. I can build successfully by restricting the LDFLAGS and ARCHFLAGS to -arch x86_64.

fonnesbeck · 2011-07-21T20:04:03Z

Spoke too soon:

In [1]: import pandas
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/Users/fonnescj/<ipython-input-1-d6ac987968b6> in <module>()
----> 1 import pandas

/Library/Python/2.7/site-packages/pandas/__init__.py in <module>()
     10 from pandas.info import __doc__
     11 
---> 12 from pandas.core.api import *
     13 from pandas.core.common import set_printoptions
     14 from pandas.io.parsers import read_csv, read_table, ExcelFile

/Library/Python/2.7/site-packages/pandas/core/api.py in <module>()
      6 import pandas.core.datetools as datetools
      7 
----> 8 from pandas.core.common import isnull, notnull
      9 from pandas.core.index import Index
     10 from pandas.core.daterange import DateRange

/Library/Python/2.7/site-packages/pandas/core/common.py in <module>()
      9 import numpy as np
     10 
---> 11 import pandas._tseries as _tseries
     12 
     13 # XXX: HACK for NumPy 1.5.1 to suppress warnings


ImportError: dynamic module does not define init function (init_tseries)

wesm · 2011-07-21T20:08:45Z

Ouch. May have to wait for the Cython folks to test on 10.7.

fonnesbeck · 2011-07-21T20:10:31Z

I guess so. I have not been able to build SciPy either, so the problems are not confined to cython.

keflavich · 2011-07-22T16:16:44Z

I have the exact same issue reported at the top, but on 10.6 Snow Leopard. I'm using the 64 bit version of the OS.
As fonnesbeck reported, I succeeded in getting the module to install using the cython commands, but then ran into the same import error:
ImportError: dynamic module does not define init function (init_tseries)

I've tried the installs with both python2.6 and 2.7. I'm running numpy 1.4 on python2.6 and 2.0 on python2.7, and the errors are identical.

wesm · 2011-07-22T16:18:52Z

@keflavich what version of Cython

keflavich · 2011-07-22T16:26:48Z

python2.7: 0.14.1
python2.6: 0.12.1

I installed with easy_install on both. Those are also based on the presence of eggs in my site-packages directories; cython doesn't seem to have a version attribute

wesm · 2011-07-22T16:29:24Z

can you post the exact output of running

python setup.py clean
python setup.py build_ext --inplace

keflavich · 2011-07-22T16:33:13Z

$ python setup.py build_ext --inplace
running build_ext
building 'pandas._tseries' extension
C compiler: gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

creating build
creating build/temp.macosx-10.6-x86_64-2.6
creating build/temp.macosx-10.6-x86_64-2.6/pandas
creating build/temp.macosx-10.6-x86_64-2.6/pandas/src
compile options: '-I/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'
gcc: pandas/src/tseries.c
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include/numpy/__ufunc_api.h:197: warning: '_import_umath' defined but not used
pandas/src/tseries.c:1507: warning: '__pyx_f_7tseries_get_int16_ptr' defined but not used
pandas/src/tseries.c:1548: warning: '__pyx_f_7tseries_get_int32_ptr' defined but not used
pandas/src/tseries.c:1589: warning: '__pyx_f_7tseries_get_int64_ptr' defined but not used
pandas/src/tseries.c:1630: warning: '__pyx_f_7tseries_get_double_ptr' defined but not used
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.6-x86_64-2.6/pandas/src/tseries.o -o /Users/adam/repos/pandas/pandas/_tseries.so
building 'pandas._sparse' extension
C compiler: gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

compile options: '-I/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'
gcc: pandas/src/sparse.c
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include/numpy/__multiarray_api.h:1188: warning: '_import_array' defined but not used
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include/numpy/__ufunc_api.h:197: warning: '_import_umath' defined but not used
pandas/src/sparse.c: In function '__pyx_pf_6sparse_12reindex_block':
pandas/src/sparse.c:15436: warning: '__pyx_v_length' is used uninitialized in this function
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.6-x86_64-2.6/pandas/src/sparse.o -o /Users/adam/repos/pandas/pandas/_sparse.so

wesm · 2011-07-22T16:39:13Z

here's what mine looks like from a fresh git clone (I'm using EPD 7.0):

12:35 ~/code/fresh-pandas  (master)$ python setup.py build_ext --inplace
running build_ext
cythoning pandas/src/tseries.pyx to pandas/src/tseries.c
building 'pandas._tseries' extension
C compiler: gcc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.5.sdk -DNDEBUG -g -O3 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.5.sdk -I/Library/Frameworks/EPD64.framework/Versions/7.0/include

creating build
creating build/temp.macosx-10.5-x86_64-2.7
creating build/temp.macosx-10.5-x86_64-2.7/pandas
creating build/temp.macosx-10.5-x86_64-2.7/pandas/src
compile options: '-I/Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-packages/numpy/core/include -I/Library/Frameworks/EPD64.framework/Versions/7.0/include/python2.7 -c'
gcc: pandas/src/tseries.c
gcc -g -arch x86_64 -L/usr/local/lib -L/Library/Frameworks/EPD64.framework/Versions/7.0/lib -bundle -undefined dynamic_lookup -g -arch x86_64 -L/usr/local/lib -L/Library/Frameworks/EPD64.framework/Versions/7.0/lib -arch x86_64 build/temp.macosx-10.5-x86_64-2.7/pandas/src/tseries.o -o /Users/wesm/code/fresh-pandas/pandas/_tseries.so
cythoning pandas/src/sparse.pyx to pandas/src/sparse.c
building 'pandas._sparse' extension
C compiler: gcc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.5.sdk -DNDEBUG -g -O3 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.5.sdk -I/Library/Frameworks/EPD64.framework/Versions/7.0/include

compile options: '-I/Library/Frameworks/EPD64.framework/Versions/7.0/lib/python2.7/site-packages/numpy/core/include -I/Library/Frameworks/EPD64.framework/Versions/7.0/include/python2.7 -c'
gcc: pandas/src/sparse.c
gcc -g -arch x86_64 -L/usr/local/lib -L/Library/Frameworks/EPD64.framework/Versions/7.0/lib -bundle -undefined dynamic_lookup -g -arch x86_64 -L/usr/local/lib -L/Library/Frameworks/EPD64.framework/Versions/7.0/lib -arch x86_64 build/temp.macosx-10.5-x86_64-2.7/pandas/src/sparse.o -o /Users/wesm/code/fresh-pandas/pandas/_sparse.so

how did you install NumPy? I really don't know what to tell you guys (@fonnesbeck can you make sense of any of that?) -- this looks like some kind of distutils / Cython problem and is likely worth asking the Cython guys. I don't know enough about such matters to debug this

keflavich · 2011-07-22T16:45:26Z

Maybe EPD is more internally self consistent...
What version of cython are you using? I suppose I can also try explicitly passing some other compiler flags... you've specified -arch x86_64, while I haven't.

fonnesbeck · 2011-07-22T16:46:43Z

Which Xcode are you using @keflavich? Try installing my packages at http://stronginference.com/scipy-superpack/ which I was able to use pandas under.

wesm · 2011-07-22T16:47:02Z

Enthought have likely modified the distutils config to play nice with the scientific Python stack. Yet another reason why I've given up on building everything myself! Using Cython 0.14.1.

wesm · 2011-07-22T16:47:48Z

I similarly highly recommend you use Chris's superpack, what I used to use on OS X before EPD (!)

keflavich · 2011-07-22T16:52:59Z

Thanks for those tips. I've attempted the scipy-superpack install before but never successfully. I'm still in build-everything-myself-hell because I don't want to risk undoing all the work I put into the installs.

I think I'm using xcode 3.2.6. I installed numpy (and scipy, and matplotlib) from source, all 64-bit.

fonnesbeck · 2011-07-22T16:55:12Z

OK, I'm not sure the superpack will work, then, as it is built around Xcode 4.

wesm · 2011-07-22T17:24:58Z

Shouldn't the superpack be essentially equivalent to your builds (barring being on the newer XCode)?

wesm · 2011-07-24T00:57:58Z

@fonnesbeck did you ever get this resolved (I saw that you got SciPy to build)?

fonnesbeck · 2011-07-24T01:26:18Z

Not yet. I had to monkey with some includes on SciPy to get it working, but its not clear if its a similar issue for pandas.

snth · 2011-07-29T20:02:51Z

I have the same problem as keflavich but on Ubuntu 11.04 32-bit. Python 2.7.1+ with all packages from the Ubuntu repositories. Fresh git clone of pandas as of about an hour ago.

$: ~/Documents/src/pandas-fresh$ python setup.py build_ext --inplace
running build_ext
cythoning pandas/src/tseries.pyx to pandas/src/tseries.c
building 'pandas._tseries' extension
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC

creating build
creating build/temp.linux-i686-2.7
creating build/temp.linux-i686-2.7/pandas
creating build/temp.linux-i686-2.7/pandas/src
compile options: '-I/usr/lib/pymodules/python2.7/numpy/core/include -I/usr/include/python2.7 -c'
gcc: pandas/src/tseries.c
/usr/lib/pymodules/python2.7/numpy/core/include/numpy/__ufunc_api.h:196:1: warning: ‘_import_umath’ defined but not used
pandas/src/tseries.c:1482:33: warning: ‘__pyx_f_6pandas_8_tseries_get_int16_ptr’ defined but not used
pandas/src/tseries.c:1519:33: warning: ‘__pyx_f_6pandas_8_tseries_get_int32_ptr’ defined but not used
pandas/src/tseries.c:1556:33: warning: ‘__pyx_f_6pandas_8_tseries_get_int64_ptr’ defined but not used
pandas/src/tseries.c:1593:34: warning: ‘__pyx_f_6pandas_8_tseries_get_double_ptr’ defined but not used
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions build/temp.linux-i686-2.7/pandas/src/tseries.o -o /home/toby/Documents/src/pandas-fresh/pandas/_tseries.so
cythoning pandas/src/sparse.pyx to pandas/src/sparse.c
building 'pandas._sparse' extension
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC

compile options: '-I/usr/lib/pymodules/python2.7/numpy/core/include -I/usr/include/python2.7 -c'
gcc: pandas/src/sparse.c
/usr/lib/pymodules/python2.7/numpy/core/include/numpy/__multiarray_api.h:1187:1: warning: ‘_import_array’ defined but not used
/usr/lib/pymodules/python2.7/numpy/core/include/numpy/__ufunc_api.h:196:1: warning: ‘_import_umath’ defined but not used
pandas/src/sparse.c: In function ‘__pyx_pf_6pandas_7_sparse_reindex_block’:
pandas/src/sparse.c:14793:13: warning: ‘__pyx_v_length’ may be used uninitialized in this function
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions build/temp.linux-i686-2.7/pandas/src/sparse.o -o /home/toby/Documents/src/pandas-fresh/pandas/_sparse.so

wesm · 2011-07-29T21:31:21Z

@snth so when you try important from the source directory, then, you're getting the same import error?

wesm · 2011-07-29T21:35:57Z

@snth @keflavich can you please try building off of the current git HEAD (pushed moments ago). I switched to build using regular distutils instead of numpy's distutils

same goes for you @fonnesbeck =P

snth · 2011-07-29T21:42:58Z

No, import error is slightly different. I'll clone your latest HEAD now. Here's my previous import error so long.

Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/__init__.py", line 12, in <module>
    from pandas.core.api import *
  File "pandas/core/api.py", line 8, in <module>
    from pandas.core.common import isnull, notnull
  File "pandas/core/common.py", line 11, in <module>
    import pandas._tseries as _tseries
ImportError: No module named _tseries
>>>

wesm · 2011-07-29T21:46:15Z

that's strange because it looks like everything worked fine from your "python setup.py built_ext --inplace" output. Is there a _tseries.so file under your pandas-fresh/pandas/ directory?

snth · 2011-07-29T21:59:43Z

Yes, there is. There's also a _tseries.so under /usr/local/lib/python2.7/dist-packages/pandas if I do a normal install.

Just tried your latest git HEAD and I'm still getting the same behaviour.

Incidentally, as I was checking for the _tseries.so file, I ran python from within the /usr/local/lib/python2.7/dist-packages/pandas directory and I could "import pandas" just fine. Do I have a PYTHONPATH problem?

wesm · 2011-07-29T22:03:07Z

can you copy/paste the full output when you try "import pandas" completely outside of dist-packages and outside of the pandas source directory?

Secondly, can you show me the list of files in dist-packages? Looks like some kind of path problem but very hard for me to diagnose

snth · 2011-07-29T22:15:57Z

Sorry, I was running python from the pandas source directory. From everywhere else it runs fine. I also cleaned out the latest install and rebuilt the previous version which I had started with and that runs fine as well.

Really sorry to have wasted your time.

wesm · 2011-07-29T22:17:31Z

awesome, no worries at all. it's basically a rite of passage in writing python code to struggle with builds at times =) i would like to resolve the issues that kevlavich and fonnesbeck have, though

wesm · 2011-09-12T17:54:45Z

Cleared the milestone as this seems idiosyncratic, won't let it block the 0.4 release. But leaving the issue open until we figure out what's going on (!)

wesm · 2011-10-02T02:55:40Z

Unless I'm mistaken this was an idiosyncratic issue and has been resolved :)

fonnesbeck · 2011-10-03T13:53:25Z

Correct.

Move test to correct place

wesm closed this as completed Oct 2, 2011

gerigk mentioned this issue May 30, 2012

Segfault in test_conv_annual (test_period.py , when running nosetests) #1351

Closed

dan-nadler pushed a commit to dan-nadler/pandas that referenced this issue Sep 23, 2019

Merge pull request pandas-dev#77 from manahl/move-test

7c694c1

Move test to correct place

yuji38kwmt mentioned this issue May 1, 2023

BUG: pandas.to_datetime("2023-01-01T15:58:12+09:00", format="%Y-%m-%dT%H:%M:%S.%f%z") has ValueError #53016

Closed

3 tasks

determ1ne mentioned this issue May 24, 2023

BUG: inconsistent behavior and crash in DataFrame.__setitem__ when >=3d ndarray is used #53366

Open

3 tasks

Build error on OS X 10.7: tseries #77

Build error on OS X 10.7: tseries #77

Comments

fonnesbeck commented Jul 21, 2011

wesm commented Jul 21, 2011

fonnesbeck commented Jul 21, 2011

wesm commented Jul 21, 2011

fonnesbeck commented Jul 21, 2011

fonnesbeck commented Jul 21, 2011

fonnesbeck commented Jul 21, 2011

wesm commented Jul 21, 2011

fonnesbeck commented Jul 21, 2011

keflavich commented Jul 22, 2011

wesm commented Jul 22, 2011

keflavich commented Jul 22, 2011

wesm commented Jul 22, 2011

keflavich commented Jul 22, 2011

wesm commented Jul 22, 2011

keflavich commented Jul 22, 2011

fonnesbeck commented Jul 22, 2011

wesm commented Jul 22, 2011

wesm commented Jul 22, 2011

keflavich commented Jul 22, 2011

fonnesbeck commented Jul 22, 2011

wesm commented Jul 22, 2011

wesm commented Jul 24, 2011

fonnesbeck commented Jul 24, 2011

snth commented Jul 29, 2011

wesm commented Jul 29, 2011

wesm commented Jul 29, 2011

snth commented Jul 29, 2011

wesm commented Jul 29, 2011

snth commented Jul 29, 2011

wesm commented Jul 29, 2011

snth commented Jul 29, 2011

wesm commented Jul 29, 2011

wesm commented Sep 12, 2011

wesm commented Oct 2, 2011

fonnesbeck commented Oct 3, 2011