problem with building pandas - not using gcc as compiler

[aixtools]

"""the example is merely
pip install pandas - which fails, so download and run

python ./setup.py build

I fear it is a specific behavior of gcc compared to not gcc/gnu 'features' (aka NGNU for 'Not "GNU Not Unix"')

I will attach the .cpp, as well as a .i (-E) output, as well as verbose include file processing (.info file)

The build stops with the following messages:

root@x064:[/data/prj/python/pipbuilds/pandas-0.19.0]das/msgpack/_unpacker.cpp -o build/temp.aix-5.3-2.7/pandas/msgpack/_unpacker.o                                           <
"pandas/msgpack/../src/msgpack/unpack_template.h", line 173.13: 1540-0063 (S) The text "..." is unexpected.
"pandas/msgpack/../src/msgpack/unpack_template.h", line 75.19: 1540-0700 (I) The previous message was produced while processing "unpack_execute<1>(unpack_context *, const char *, size_t, size_t *)".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 468.45: 1540-0700 (I) The previous message was produced while processing "unpack_construct".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 173.13: 1540-0063 (S) The text "..." is unexpected.
"pandas/msgpack/../src/msgpack/unpack_template.h", line 75.19: 1540-0700 (I) The previous message was produced while processing "unpack_execute<0>(unpack_context *, const char *, size_t, size_t *)".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 469.40: 1540-0700 (I) The previous message was produced while processing "unpack_skip".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 433.29: 1540-0063 (S) The text "..." is unexpected.
"pandas/msgpack/../src/msgpack/unpack_template.h", line 412.19: 1540-0700 (I) The previous message was produced while processing "unpack_container_header<144,220>(unpack_context *, const char *, size_t, size_t *)".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 470.46: 1540-0700 (I) The previous message was produced while processing "read_array_header".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 433.29: 1540-0063 (S) The text "..." is unexpected.
"pandas/msgpack/../src/msgpack/unpack_template.h", line 412.19: 1540-0700 (I) The previous message was produced while processing "unpack_container_header<128,222>(unpack_context *, const char *, size_t, size_t *)".
"pandas/msgpack/../src/msgpack/unpack_template.h", line 471.44: 1540-0700 (I) The previous message was produced while processing "read_map_header".

reading carefully - it comes down to errors while processing these four lines:
  +468  static const execute_fn unpack_construct = &unpack_execute<true>;
  +469  static const execute_fn unpack_skip = &unpack_execute<false>;
  +470  static const execute_fn read_array_header = &unpack_container_header<0x90, 0xdc>;
  +471  static const execute_fn read_map_header = &unpack_container_header<0x80, 0xde>;

These become (from the .i file)
#line 468
static const execute_fn unpack_construct = &unpack_execute<1>;
static const execute_fn unpack_skip = &unpack_execute<0>;
static const execute_fn read_array_header = &unpack_container_header<0x90, 0xdc>;
static const execute_fn read_map_header = &unpack_container_header<0x80, 0xde>;
#line 314 "pandas/msgpack/_unpacker.cpp"

And - whether it is pandas or perhaps cython - my knowledge of C does not understand how:
&unpack_execute<1> and &unpack_execute<0>
are useable against:

static inline int unpack_execute(unpack_context* ctx, const char* data, size_t len, size_t* off)

FYI: _packer.cpp is fine.

Will try again to attach file - getting message "We don't support that file type", even after putting them into a zip file.

[aixtools]

please let me know how to attach a text or zip file - if interested!

[jreback]

pls show architecture, version of cython & python. as well as how you got the source.

and version of gcc.

[aixtools]

AIX 5.3, (not gcc, but xlc), 64-bit
root@x064:[/]pip list
Cython (0.24.1)
numpy (1.11.2)
pip (8.1.1)
python-dateutil (2.5.3)
pytz (2016.7)
requests (2.11.1)
setuptools (20.10.1)
six (1.10.0)

pip install pandas

pip download pandas

root@x065:[/data/prj/python/pipbuilds]ls *.gz
numpy-1.11.2.tar.gz pandas-0.19.0.tar.gz

[aixtools]

I repeated the process, on AIX 6.1 - 32-bit mode, python built using gcc rather xlc.

AIX 6.1, python-2.7.12, gcc-4.7.4
root@x065:[/data/prj/python/pipbuilds]pip list
Cython (0.24.1)
numpy (1.11.2)
pip (8.1.1)
python-dateutil (2.5.3)
pytz (2016.7)
setuptools (20.10.1)
six (1.10.0)

This ends one(?) file earlier - I think the first .cpp (packer.cpp), with different warnings:
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -D_LARGE_FILES -D__BIG_ENDIAN_=1 -Ipandas/src/msgpack -Ipandas/src/klib -Ipandas/src -I/opt/lib/python2.7/site-packages/numpy/core/include -I/opt/include/python2.7 -c pandas/msgpack/_packer.cpp -o build/temp.aix-6.1-2.7/pandas/msgpack/_packer.o -Wno-unused-function
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled by default]
pandas/msgpack/_packer.cpp: In function 'Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*)':
pandas/msgpack/_packer.cpp:7517:9: error: call of overloaded 'abs(const Py_ssize_t&)' is ambiguous
pandas/msgpack/_packer.cpp:7517:9: note: candidates are:
In file included from /opt/include/python2.7/Python.h:42:0,
from pandas/msgpack/_packer.cpp:4:
/opt/lib/gcc/powerpc-ibm-aix5.3.7.0/4.7.4/include-fixed/stdlib.h:309:14: note: int abs(int)
In file included from /opt/include/python2.7/pyport.h:325:0,
from /opt/include/python2.7/Python.h:58,
from pandas/msgpack/_packer.cpp:4:
/opt/lib/gcc/powerpc-ibm-aix5.3.7.0/4.7.4/include-fixed/math.h:947:14: note: float abs(float)
/opt/lib/gcc/powerpc-ibm-aix5.3.7.0/4.7.4/include-fixed/math.h:977:20: note: long double abs(long double)
error: command 'gcc' failed with exit status 1

The "error" being this line:
pandas/msgpack/_packer.cpp:7517:9: error: call of overloaded 'abs(const Py_ssize_t&)' is ambiguous

FYI: when using xlc - _packer.o completed.

45482770 116 -rw-r----- 1 root system 117901 Oct 7 10:46 ./pandas-0.19.0/build/temp.aix-5.3-2.7/pandas/msgpack/_packer.o
root@x065:[/data/prj/python/pipbuilds]oslevel
6.1.0.0

Note the directory above is build/temp.aix5.3-2.7 not aix6.1-2.7

[jreback]

we don't support aix directly, meaning that it's not tested at all
so would accept compatible patches

we also make no guarantees about big endian (powerpc)
again would accept patches

[aixtools]

Well, I would consider working on a patch. I am just a humble packager with some programming skills.
Perhaps you could give a hint about where to look? As the code seems to be generated I do not think debugging generated code will help much.

In other words - I accept hints and tips :)

[jreback]

@aixtools no real idea what is going on here, maybe @llllllllll has some

[aixtools]

And how would I get, rather this question - get any attention from @llllllllll - as I have never been properly introduced.

[jreback]

@aixtools there is nothing to do, he will answer when/if he can. otherwise others may be able to offer assistance.

[llllllllll]

It looks like the compiler does not support case ranges like:

switch(x) {
case a ... b:
}

The current CPP branching defines USE_CASE_RANGE when !defined(_MSC_VER). I think we should invert this to be: If you are compiling with something that supports gnu C, or ifdef __GNUC__.

diff --git a/pandas/src/msgpack/unpack_template.h b/pandas/src/msgpack/unpack_template.h
index 95af673..fba372d 100644
--- a/pandas/src/msgpack/unpack_template.h
+++ b/pandas/src/msgpack/unpack_template.h
@@ -17,7 +17,7 @@
  */

 #ifndef USE_CASE_RANGE
-#if !defined(_MSC_VER)
+#ifdef __GNUC__
 #define USE_CASE_RANGE
 #endif
 #endif

In the short term, you can define _MSC_VER which will disable the case range here, hopefully that doesn't cause anything to use a microsoft specific feature.

[aixtools]

Many thanks to @llllllllll . As this was closed had fallen low on my radar. However, I hear from a colleague this took care of the problem. Kudos!

[jreback]

@aixtools if you'd like to do a PR with your change would be great.

[aixtools]

Have to learn how to PR I guess. :)

I have git working again (with the strict SSL requirements) - so, eventually, it shall come.

[aixtools]

I tried doing everything locally (on my server) but could not find the exact syntax for a pull-request. So, tried a couple of paths to get a PR done. Hope this is satisfactory!