Performance decrease of groupby.first for datetime64 in 0.14

[Marigold]

I have seen a dramatic (around 100x) slowdown of groupby.first method for datetime64 type after updating to 0.14.

Here is an example:

from time import time
d = pd.DataFrame({'a': pd.date_range('1/1/2011', periods=100000, freq='s'), 'b': range(100000)})
t = time()
d.groupby('b').first()
print(time() - t)

Result:

3.87164497375

Same for int type

from time import time
d = pd.DataFrame({'a': range(100000), 'b': range(100000)})
t = time()
d.groupby('b').first()
print(time() - t)

Result:

0.0148799419403

[jreback]

what version were you using?

[Marigold]

This is from 0.14.0, but I have experienced the same on master yesterday.
UPDATE: tested again on current master. Still the same.

[jreback]

I see that

did u have a version where it was fast?

[Marigold]

Latest 0.13 was definitely fine. I will check it again just to make sure.

UPDATE: Just tested 0.13.1 and it takes even longer. Maybe the bug is on the numpy side? I will try to go deeper into that and post the findings here.

[Marigold]

Downgrading numpy or pandas didn't help. Do you get the same results as me? Could you put me on the right direction where to look? Thanks.

[jreback]

This was falling back to python space for the aggregation, so was much slower. fix shortly.

[jreback]

see #7560

[jreback]

give a try on master

[Marigold]

datetime works fine now, but I experience the same problem with object

from time import time
d = pd.DataFrame({'a': ['foo'] * 100000, 'b': range(100000)})
t = time()
d.groupby('b').first()
print(time() - t)

[jreback]

@Marigold ok fixed up...thanks for the report. was not checking these cases, and had a regression.

[Marigold]

Works perfectly. Thanks for a quick feedback.