DEPR: deprecating series asof GH10343

[bwillers]

closes #10343 and discussion on #10266 - deprecating Series.asof(where) in favour of Series.reindex(where, method='ffill').

[jreback]

this is ok, but I think this could use a small doc-section in missing.rst in the fillna subsection.

also will have to add a reference in the deprecate issue #6581 (will do this upon merging)

[bwillers]

@jreback I'd be happy to add it, but I'm not entirely sure what you're looking for here. I had a look through the missing_data.rst and there's no mention of asof to correct/update. Since it is a reindexing method, it's not clear how it should documented in the missing data section.

[jreback]

@bwillers can you rebase?

[bwillers]

@jreback rebased and green

[jreback]

ffill

[jreback]

Can you put a mini-doc section right after here: http://pandas.pydata.org/pandas-docs/stable/timeseries.html#filling-forward-backward

showing an asof usecase (using the reindexing and fill method). Also a pointer in the whatsnew to this new doc section would be great. thxs.

[max-sixty]

If everyone thinks this is a good idea to deprecate, then so be it. I should say that we use this a lot, and find it a really convenient method. There are lots of use cases where you want to ask "what was the most recent valid value". And asof is much simpler than Series.reindex(where, method='ffill'), particularly when reading code.

Am I missing something? Or are we just particular in our use case?

[bwillers]

@MaximilianR Well, the motivation here is mainly based on asof being a re-implementation of functionality that already exists in the better tested and more widely applicable reindex. I absolutely agree that the basic operation of wanting to know the most recent value is a common one, and that s.dropna().reindex(where, method='ffill') is unfortunately a lot more verbose than s.asof(where).

That said, reindex has the advantage of working for a NDFrame or DataFrame, and working with a MultiIndex, none of which are supported by asof.

The question becomes if the convenience of the shorter form for the specialized use case is worth the code bloat from multiple implementations of the same thing. I'm not personally convinced that it does, but that's definitely something worth discussing.

[max-sixty]

@bwillers completely agree with that synthesis - i.e. the big question being how useful a method needs to be in order to compensate for the additional clutter.

One point to consider for the specific case is how obvious that translation is for the average user. I'd have thought 'not very obvious', but I don't have a great handle on what the average current / future user looks like.

If we do keep it, having it as a simple wrapper of the translation seems wise.

Over to the maintainers on the broader question...

[jorisvandenbossche]

As a reference, asof is used in Python for Data Analysis

[jorisvandenbossche]

Note: I never used asof myself, so difficult to assess.
But, if asof is used and is useful, I think we should reconsider the deprecation. I agree that the equivalent of s.dropna().reindex(where, method='ffill') is not that straightforward, and easy to come up with.

We either deprecate it, or we keep it and see this as an opportunitiy to:

• reimplement it using the existing and better tested reindexing machinery
• in this way, make the function more general (not restricted to timeseries)
• in the docstring, make it clear that this is a shortcut for a reindexing operation, and if you want more options (other filling etc) or use it for a dataframe you should use that. In this way, the function can also be somewhat educational to explain what reindexing does
• also explain this in the tutorial docs

And maybe it would be useful to have some kind of skipna kwarg in reindex?

[bwillers]

@jorisvandenbossche That's not a bad idea - effectively 'replumb' the asof internals into reindex, but leave the compact and fairly intuitive api the same.

[jreback]

@bwillers ok!

[jreback]

ok, I revised the original issue. Let's try to reimplement this using .reindex internally, add tests, and leave the API.

[jreback]

@bwillers you can repurpose this PR or open a new one up to u

[bwillers]

@jreback sounds good, running around like a headless chicken this week but hope to get this done next.