Skip to content

can't install pandas use pip on mac #3074

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yishenggudou opened this issue Mar 17, 2013 · 34 comments
Closed

can't install pandas use pip on mac #3074

yishenggudou opened this issue Mar 17, 2013 · 34 comments
Labels
Build Library building on various platforms
Milestone

Comments

@yishenggudou
Copy link

the error log is

Searching for pandas
Reading http://pypi.python.org/simple/pandas/
Reading http://pandas.pydata.org
Reading http://pandas.sourceforge.net
Download error: [Errno 60] Operation timed out -- Some packages may not be found!
Best match: pandas 0.10.1
Downloading http://pypi.python.org/packages/source/p/pandas/pandas-0.10.1.zip#md5=b9127d9695aa501e3696a608427622ca
Processing pandas-0.10.1.zip
Running pandas-0.10.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Oiy1Kb/pandas-0.10.1/egg-dist-tmp-Vl3z2X
error: Setup script exited with pandas requires NumPy >= 1.6.1 due to datetime64 dependency
haibos-MacBook-Air:tyrannosaurus timger$ sudo pip install pandas
Password:
Downloading/unpacking pandas
Downloading pandas-0.10.1.zip (2.8Mb): 2.8Mb downloaded
Running setup.py egg_info for package pandas
pandas requires NumPy >= 1.6.1 due to datetime64 dependency
Complete output from command python setup.py egg_info:
pandas requires NumPy >= 1.6.1 due to datetime64 dependency


Command python setup.py egg_info failed with error code 1 in /Users/timger/bitbucket/tyrannosaurus/build/pandas
Storing complete log in /Users/timger/Library/Logs/pip.log

@ghost
Copy link

ghost commented Mar 17, 2013

what does

In [2]: import numpy
In [3]: numpy.__version__

produce on your box?

@ghost
Copy link

ghost commented Mar 17, 2013

This might be #2740.

does pip install git://github.com/pydata/pandas.git work for you?
if yes, this' is already fixed and wil be on pypi in 0.11.

If you don't want to run git master, manually install numpy and
then try it again.

correction: pip install git+git://github.com/pydata/pandas.git

@yishenggudou
Copy link
Author

@y-p
my numpy vision is 1.7.0

In [1]: import numpy
In [3]: numpy.__version__
Out[3]: '1.7.0'

@yishenggudou
Copy link
Author

@y-p
I run pip install git://github.com/pydata/pandas.git

Downloading/unpacking git://github.com/pydata/pandas.git
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/basecommand.py", line 104, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/commands/install.py", line 245, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/req.py", line 985, in prepare_files
    self.unpack_url(url, location, self.is_download)
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/req.py", line 1103, in unpack_url
    return unpack_vcs_link(link, loc, only_download)
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/download.py", line 294, in unpack_vcs_link
    vcs_backend.unpack(location)
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/vcs/__init__.py", line 233, in unpack
    self.obtain(location)
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/vcs/git.py", line 98, in obtain
    url, rev = self.get_url_rev()
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/vcs/git.py", line 184, in get_url_rev
    url, rev = super(Git, self).get_url_rev()
  File "/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg/pip/vcs/__init__.py", line 118, in get_url_rev
    url = self.url.split('+', 1)[1]
IndexError: list index out of range

Storing complete log in /Users/timger/Library/Logs/pip.log

@yishenggudou
Copy link
Author

@y-p
i clone the pandas use git clone git://github.com/pydata/pandas.git
and install use python setup.py install
show the error message

haibos-MacBook-Air:pandas timger$ sudo python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to pandas.egg-info/requires.txt
writing pandas.egg-info/PKG-INFO
writing top-level names to pandas.egg-info/top_level.txt
writing dependency_links to pandas.egg-info/dependency_links.txt
reading manifest file 'pandas.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'setupegg.py'
no previously-included directories found matching 'doc/build'
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
writing manifest file 'pandas.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_py
copying pandas/version.py -> build/lib.macosx-10.7-x86_64-2.7/pandas
running build_ext
Traceback (most recent call last):
  File "setup.py", line 720, in <module>
    **setuptools_kwargs)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/install.py", line 73, in run
    self.do_egg_install()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/install.py", line 93, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/bdist_egg.py", line 179, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/bdist_egg.py", line 166, in call_command
    self.run_command(cmdname)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/install_lib.py", line 20, in run
    self.build()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/command/install_lib.py", line 111, in build
    self.run_command('build_ext')
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "setup.py", line 343, in build_extensions
    self.check_cython_extensions(self.extensions)
  File "setup.py", line 340, in check_cython_extensions
    """ % src)
Exception: Cython-generated file 'pandas/tslib.c' not found.
                Cython is required to compile pandas from a development branch.
                Please install Cython or download a release package of pandas.

@yishenggudou
Copy link
Author

@y-p
but i have installed cython

haibos-MacBook-Air:pandas timger$ cython 
Cython (http://cython.org) is a compiler for code written in the
Cython language.  Cython is based on Pyrex by Greg Ewing.

Usage: cython [options] sourcefile.{pyx,py} ...

Options:
  -V, --version                  Display version number of cython compiler
  -l, --create-listing           Write error messages to a listing file
  -I, --include-dir <directory>  Search for include files in named directory
                                 (multiple include directories are allowed).
  -o, --output-file <filename>   Specify name of generated C file
  -t, --timestamps               Only compile newer source files
  -f, --force                    Compile all source files (overrides implied -t)
  -q, --quiet                    Don't print module names in recursive mode
  -v, --verbose                  Be verbose, print file names on multiple compilation
  -p, --embed-positions          If specified, the positions in Cython files of each
                                 function definition is embedded in its docstring.
  --cleanup <level>              Release interned objects on python exit, for memory debugging.
                                 Level indicates aggressiveness, default 0 releases nothing.
  -w, --working <directory>      Sets the working directory for Cython (the directory modules
                                 are searched from)
  --gdb                          Output debug information for cygdb

  -D, --no-docstrings            Strip docstrings from the compiled module.
  -a, --annotate                 Produce a colorized HTML version of the source.
  --line-directives              Produce #line directives pointing to the .pyx source
  --cplus                        Output a C++ rather than C file.
  --embed[=<method_name>]        Generate a main() function that embeds the Python interpreter.
  -2                             Compile based on Python-2 syntax and code semantics.
  -3                             Compile based on Python-3 syntax and code semantics.
  --fast-fail                    Abort the compilation on the first error
  --warning-error, -Werror       Make all warnings into errors
  --warning-extra, -Wextra       Enable extra warnings
  -X, --directive <name>=<value>[,<name=value,...] Overrides a compiler directive
haibos-MacBook-Air:pandas timger$ which cython
/usr/local/bin/cython


haibos-MacBook-Air:pandas timger$ ls -alh /Library/Python/2.7/site-packages/|grep cython
-rw-r--r--    1 root    wheel   407B Jul 29  2012 cython.py
-rw-r--r--    1 root    wheel   500B Jul 29  2012 cython.pyc

@ghost
Copy link

ghost commented Mar 18, 2013

the pip/git fail is due to me quoting the wrong url, it should be:

pip install git+git://github.com/pydata/pandas.git

but cloning and running should be equiv.

what does

λ python
Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Cython
>>> Cython.__version__
'0.18'

give you?
cython has a runnable command in /*/bin/, but it needs to be availble from site-packages as well.
so if you're using a virtualenv, it needs to be installed inside the env. having it installed globally isn't enough.

@yishenggudou
Copy link
Author

my cython version is 0.16

In [7]: import Cython

In [8]: Cython.__version__
Out[8]: '0.16'

@ghost
Copy link

ghost commented Mar 18, 2013

What OSX version are you on?
Please try these in order and post the output of each:

from Cython.Distutils import build_ext as _build_ext
from setuptools import setup, Command

check version of setuptools if installed:
SO

command line:

python setup.py clean
python setup.py build_ext

if you get no errors, ensure the c files are available:

ls -l pandas/*.c

-rw-r--r-- 1 root root 7321909 Mar 18 05:50 pandas/algos.c
-rw-r--r-- 1 root root  957656 Mar 18 05:50 pandas/hashtable.c
-rw-r--r-- 1 root root  706479 Mar 18 05:49 pandas/index.c
-rw-r--r-- 1 root root 2486151 Mar 18 05:50 pandas/lib.c
-rw-r--r-- 1 root root 2556906 Mar 18 05:49 pandas/tslib.c

@yishenggudou
Copy link
Author

In [80]: from Cython.Distutils import build_ext as _build_ext

In [81]: from setuptools import setup, Command

In [82]: print set
set      setattr  setup  

some error show after my clone pandas an run setup.py install manual

haibos-MacBook-Air:pandas timger$ sudo python setup.py clean
running clean
haibos-MacBook-Air:pandas timger$ sudo python setup.py build_ext
running build_ext
Traceback (most recent call last):
  File "setup.py", line 720, in <module>
    **setuptools_kwargs)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python/2.7.3/lib/python2.7/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "setup.py", line 343, in build_extensions
    self.check_cython_extensions(self.extensions)
  File "setup.py", line 340, in check_cython_extensions
    """ % src)
Exception: Cython-generated file 'pandas/index.c' not found.
                Cython is required to compile pandas from a development branch.
                Please install Cython or download a release package of pandas.


@yishenggudou
Copy link
Author

my cython can be import from ipython

In [82]: import Cython

In [83]: C
CNAME      CNAME.old  Command    Cython     

In [83]: Cython?
Type:       module
String Form:<module 'Cython' from '/Library/Python/2.7/site-packages/Cython/__init__.pyc'>
File:       /Library/Python/2.7/site-packages/Cython/__init__.py
Docstring:  <no docstring>

and i have set the dir to PYTHONPATH

haibos-MacBook-Air:pandas timger$ echo ${PYTHONPATH}
:/Library/Python/2.7/site-packages/:/Applications/Kivy.app/Contents/Resources/kivy:/usr/local/lib/python2.7/site-packages/:/usr/local/lib/python2.7/site-packages/python_tesseract-tesseract-py2.7-macosx-10.7-x86_64.egg:/Library/Python/2.7/site-packages:/Library/Python/2.7/site-packages:/Library/Python/2.7/site-packages

@ghost
Copy link

ghost commented Mar 20, 2013

are you inside a virtualenv? did you try running using sudo?
I don' t own a mac, so it's difficult to debug this.

That warning message should only happen if the cython check in setup.py
yields False for some reason.
edit setup.py

try:
    from Cython.Distutils import build_ext as _build_ext
    # from Cython.Distutils import Extension # to get pyrex debugging symbols
    cython = True
except ImportError:
    cython = False

and add the following directly below.

print cython
1/0

Then run "sudo python setup.py develop", does it report False or True?

@yishenggudou
Copy link
Author

yes i run use sudo
i edit the setup.py

try:
 31     import Cython
 32     sys.path.insert(0, os.path.join(os.path.dirname(__file__), "fake_pyrex"))
 33     cython = True
 34 except ImportError:
 35     cython = False
 36     pass
 37 print '-'*100
 38 print cython
 39 print '-'*100 
haibos-MacBook-Air:pandas timger$ sudo python setup.py install
Password:
----------------------------------------------------------------------------------------------------
False
----------------------------------------------------------------------------------------------------

@ghost
Copy link

ghost commented Mar 20, 2013

good. reset to master.
replace

try:
    from Cython.Distutils import build_ext as _build_ext
    # from Cython.Distutils import Extension # to get pyrex debugging symbols
    cython = True
except ImportError:
    cython = False

by

    from Cython.Distutils import build_ext as _build_ext

run "setup.py develop" again. you should get an ImportError exception.
you said that line doesn't raise an exception when you run it from the python
prompt. figure out why that's failing there and not in the prompt and that
should solve your problem.

?

@yishenggudou
Copy link
Author

ok i add py lib path to sys.path

import sys
sys.path.insert(0,"/Library/Python/2.7/site-packages/")                                                                                                              
import shutil

it work ok

Thank you very much

@ghost
Copy link

ghost commented Mar 20, 2013

was this something in your particular setup or do all macs installations have this problem?
Having a python installed without site-packages being on the path is very strange behaviour.

edit: for future reference, I encountered similar behaviour when inside a venv, with
a venv cython install, when using sudo to invoke "setup.py develop". sudo escapes
the venv sire-packages path in some way and Cython isn't available, though installed.

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

On Mac, within a venv, I've had to run

pip install cython
before upgrading pandas.

pip install --upgrade pandas

Not sure how that works with the above solution, but that did the trick for me.

Versions: cython-0.24.1, pandas-0.19.0. (I was upgrading from pandas-0.16.2)

@jorisvandenbossche
Copy link
Member

@jdejoode Normally, you shouldn't need cython, as there are binary wheels available for mac (or do you have 32bit?)

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

It is not 32 bit. The binary wheel did not install because of the cython error. Do note that I have a boatload of other packages installed in this venv.

@jorisvandenbossche
Copy link
Member

@jdejoode Can you show the output of pip install --upgrade pandas ?

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

I deleted some information, but the basic calls are still here:

Running setup.py install for pandas
    Complete output from command /Users/User/.virtualenvs/nlp/bin/python2.7 -c "import setuptools, tokenize;__file__='/private/var/folders/hz/.../T/pip-build-.../pandas/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/hz/.../T/pip-...-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/User/.virtualenvs/nlp/bin/../include/site/python2.7/pandas:
    running install
    running build
    running build_py
    package init file 'pandas/io/tests/sas/__init__.py' not found (or not a regular file)
    package init file 'pandas/io/tests/sas/__init__.py' not found (or not a regular file)
    UPDATING build/lib.macosx-10.10-x86_64-2.7/pandas/_version.py
    set build/lib.macosx-10.10-x86_64-2.7/pandas/_version.py to '0.19.0'
    running build_ext
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/hz/.../T/pip-build-.../pandas/setup.py", line 680, in <module>
        **setuptools_kwargs)
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/Users/User/.virtualenvs/nlp/lib/python2.7/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 563, in run
        self.run_command('build')
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build.py", line 127, in run
        self.run_command(cmd_name)
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_ext.py", line 337, in run
        self.build_extensions()
      File "/private/var/folders/hz/.../T/pip-build-Zjh0Ug/pandas/setup.py", line 371, in build_extensions
        build_ext.build_extensions(self)
      File "/private/var/folders/hz/.../T/pip-build-Zjh0Ug/pandas/setup.py", line 129, in build_extensions
        raise ImportError('Building pandas requires cython')
    ImportError: Building pandas requires cython

    ----------------------------------------
  Rolling back uninstall of pandas`

@jorisvandenbossche
Copy link
Member

Then it seems it is, for some reason, not downloading a wheel, but the source archive (can you see that in the first output of pip install? Normally you see there which file is downloaded from PyPI).
For installing from source, it is expected to need cython

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

I am not sure I understand, @jorisvandenbossche. I still have the entire traceback (window is still open). Do you need specific information?

@jreback
Copy link
Contributor

jreback commented Oct 4, 2016

u might have an old version of pip

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

pip 7.1.2

@jorisvandenbossche
Copy link
Member

@jdejoode If I use pip to install, I see something like:

(pandas-test-0190) joris@joris-XPS-13-9350:~$ pip install pandas
Collecting pandas
  Downloading pandas-0.19.0-cp27-cp27mu-manylinux1_x86_64.whl (16.3MB)
...

So where you can see it is downloading a wheel (notice the .whl extension), and not a source archive (.tar.gz extension without the manylinux (or in your case macosx) part in the filename)

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

I simply get

Collecting pandas Downloading pandas-0.19.0.tar.gz (8.3MB)

I have a running version of anaconda on my machine. Would that influence it?

@jorisvandenbossche
Copy link
Member

So it is downloading the sources, not the wheel.
Can you first upgrade pip to see if that solves the issue? As pip is not recognizing there is an appropriate wheel for your system.

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

[edit:style]

I can confirm that after upgrading pip, I no longer get the error. Pip uses the following wheel:

pandas-0.19.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl

@jorisvandenbossche
Copy link
Member

OK, good to hear!

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

maybe a higher pip version could be included in the dependencies of pandas? or is that overkill?

@jorisvandenbossche
Copy link
Member

pip is not a dependency of pandas (you can install pandas in other ways without pip). But newer versions of pip normally warn you if you don't have the latest version

@jdejoode
Copy link

jdejoode commented Oct 4, 2016

ok. might have silenced that. unsubscribing to the issue now. thanks for all your work.

@jdejoode
Copy link

I can just confirm that I also had the issue in a Docker container... upgrading with an old version of pip will run into the cython dependency error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Library building on various platforms
Projects
None yet
Development

No branches or pull requests

4 participants