Suggestion: inplace=True option in drop() and dropna() would really help

[bluefir]

You can have a large frame in memory and want to drop a small percentage of labels with NA data just to clean things up. If you use drop() or dropna(), you will get a new object and effectively double the memory footprint. Favoring immutability is great, but under memory constraints this can be a killer.

[wesm]

Agreed-- avoiding the "2x problem" is actually pretty tricky with NumPy arrays under the hood. I have some ideas but it won't be too simple

[bluefir]

How about lazy deletion? Have an object attached to an Index that holds a sequence of index positions that were dropped. Most pandas objects will have it as None and all the current interfaces will work fine. Checking that it is None is also quite cheap I believe. When some labels are dropped in place, no data is actually deleted, but no data is created either. It only takes effect on all data retrieval (take?), including new object being created from the data. In addition, it will follow the spirit of data immutability because you can always "undrop".

[wesm]

What you're describing would be really great (and definitely something I've thought about), but it's a very large and difficult problem and not something that could be easily bolted on (I don't think).

[bluefir]

You certainly know much better than I do! :-) pandas is a great product and it's very fast. That being said, people who want to save on memory should be willing to take a hit, even a significant one, on performance along the classic time-versus-space tradeoff. Memory bottlenecks, just as performance bottlenecks, are usually concentrated in just a few places in the code. If you consider not being that hard core on performance with inplace drops as you are in other areas, it might make it less difficult to implement. Just my two cents. Not another word from me on this topic, I promise :-)

[bburan]

Possible duplicate of #1960?

[jreback]

yep...closing this one (as its the later one)....