apply with values return wrong output without convert to list

[jesrael]

from SO

df = pd.DataFrame({'a':[1,2], 'b':[2,3]})
print (df)
   a  b
0  1  2
1  2  3

And output is wrong:

print (df.apply(lambda x: [x.values]))
a    [[8791439669712, 8791439669712]]
b    [[8791439669712, 8791439669712]]
dtype: object

But if convert to list all working nice:

print (df.apply(lambda x: [x.values.tolist()]))
a    [[1, 2]]
b    [[2, 3]]
dtype: object

---

print (df.apply(lambda x: [list(x.values)]))
a    [[1, 2]]
b    [[2, 3]]
dtype: object

What is problem?

[gfyoung]

@jesrael : Thanks for reporting this! I'm not getting your output, but I do get something buggy too:

df.apply(lambda x: [x.values])
a    [[2, 2]]
b    [[2, 2]]

Feel free to investigate and put up a PR to patch this!

[jreback]

This is actually a numpy bug on how they handle object arrays; and of course a complete mis-use of the guarantees of apply.

In [13]: pd.lib.reduce(df.values, lambda x: [x])
Out[13]: array([list([array([0, 1])]), list([array([0, 1])])], dtype=object)

In [14]: pd.lib.reduce(df.values, lambda x: list([x]))
Out[14]: 
array([list([array([4569766464, 4569081312])]),
       list([array([4569766464, 4569081312])])], dtype=object)

[rhshadrach]

I'm now getting

        a       b
0  [1, 2]  [2, 3]

which I think is the expected output.

[srkds]

take