Groupby filter changes ordering

[hayd]

At the moment filter reorders wrt the groups.

Example:

In [1]: data = pd.DataFrame(
    {'pid' : [1,1,1,2,2,3,3,3],
     'tag' : [23,45,62,24,45,34,25,62],
     })

In [2]: g = data.groupby('tag')

In [3]: g.filter(lambda x: len(x) > 1)
Out[3]: 
   pid  tag
1    1   45
4    2   45
2    1   62
7    3   62

If there is a way to efficiently keep the order that would be ideal I think, failing that sort back afterwards (but being wary of sorting with dupe/unordered indexes).

cc #3680

[jtratner]

@hayd What should it output instead?

[hayd]

Basically:

In [12]: g.filter(lambda x: len(x) > 1, dropna=False).dropna()
Out[12]:
   pid  tag
1    1   45
2    1   62
4    2   45
7    3   62

@jreback suggested in #3680 could be more efficient way...

[jreback]

cc @danielballan can you have a look?

[danielballan]

On it.

[jreback]

@danielballan ?

[jreback]

hows this coming?

[danielballan]

I will open this up once #5096 is merged....

[jreback]

@danielballan go for it!

[danielballan]

@hayd, please take a look at #5222 and see if my tests look good to you.