Skip to content

splitting pandas dataframe - np.array_split error #8846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yemu opened this issue Nov 18, 2014 · 2 comments · Fixed by #8847
Closed

splitting pandas dataframe - np.array_split error #8846

yemu opened this issue Nov 18, 2014 · 2 comments · Fixed by #8847
Labels
Bug Compat pandas objects compatability with Numpy or Python functions
Milestone

Comments

@yemu
Copy link

yemu commented Nov 18, 2014

I just noticed that after upgrading to numpy 1.9.0, when I'm trying to split dataframe with pandas 0.15.1 with the code:

split_dfs = np.array_split(big_df,8)    

I get the error:

Traceback (most recent call last):
  File "./test.py", line 127, in <module>
    split_dfs = np.array_split(big_df,8)
  File "/usr/lib/python2.7/site-packages/numpy/lib/shape_base.py", line 426, in array_split
    if sub_arys[-1].size == 0 and sub_arys[-1].ndim != 1:
  File "/usr/lib/python2.7/site-packages/pandas-0.15.1-py2.7-linux-x86_64.egg/pandas   /core/generic.py", line 1936, in __getattr__
(type(self).__name__, name))
AttributeError: 'DataFrame' object has no attribute 'size'

with pandas 0.15.1 and numpy 1.8.1 it works fine.
I'm using pandas 0.15.1 on arch linux and python2.7

@jorisvandenbossche
Copy link
Member

This is due to a change in numpy: numpy/numpy#4102 (.size is used to check if a FutureWarning should be raised or not, introduced in numpy 1.9.0).

But, this is a numpy function, and not really guaranteed to work with pandas dataframes although it did before (or should numpy do an asarray? @jreback , or should DataFrame have a size attribute?)

For a pandas-native split functionality, see the enhancement request: #7387

@jreback jreback added Bug Compat pandas objects compatability with Numpy or Python functions labels Nov 18, 2014
@jreback jreback added this to the 0.15.2 milestone Nov 18, 2014
@jreback
Copy link
Contributor

jreback commented Nov 18, 2014

I think they became more 'strict' in 1.9.1. Fixed in #8847

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Compat pandas objects compatability with Numpy or Python functions
Projects
None yet
3 participants