Broken nunique on Series group by

[cpcloud]

The following code works in 0.16.2 and not in latest master:

data = pd.DataFrame(
    [[100, 1, 'Alice'],
     [200, 2, 'Bob'],
     [300, 3, 'Charlie'],
     [-400, 4, 'Dan'],
     [500, 5, 'Edith']],
    columns=['amount', 'id', 'name']
)

expected = data.groupby(['id', 'amount'])['name'].nunique()

Going to bisect this today unless someone beats me to it.

[jreback]

#10894

[cpcloud]

Can we just revert that change and keep the test?

[cpcloud]

The only other solution I see is to coerce val to str if it's an object dtype

[jreback]

if its not int64 factorize it

ipdb> p np.lexsort((pd.factorize(val)[0],ids))
array([0, 1, 2, 3, 4])

[cpcloud]

ok

[jreback]

or rather it has a TypeError, though this shouldn't have gotten this far if that was the case....

[cpcloud]

Using factorize won't give the same results and is therefore incorrect:

In [8]: np.lexsort(([1, 2, 3], list('cba')))
Out[8]: array([2, 1, 0])

In [9]: np.lexsort(([1, 2, 3], pd.factorize(list('cba'))[0]))
Out[9]: array([0, 1, 2])

[jreback]

might be faster to just .astype(str) an object array

use is_object_dtype

[cpcloud]

yep

[behzadnouri]

Using factorize won't give the same results and is therefore incorrect:

factorize does work there.

Can we just revert that change and keep the test?

if you do not understand the algo, it would be better to ping whoever wrote the code rather than suggesting to revert it

[cpcloud]

@behzadnouri This is unrelated to my understanding of the algorithm. My suggestion to revert it was based on the fact that it broke existing code.

[behzadnouri]

My suggestion to revert it was based on the fact that it broke existing code.

existing code is not forced to update to master

[jreback]

@behzadnouri no this DID break things. We just weren't fully testing it.

[cpcloud]

existing code is not forced to update to master

That's certainly true. However this broke existing pandas code.