Setting xticks with time series broken in 0.21

[amueller]

Hey. So this worked in 0.20 but not in 0.21:

import matplotlib.pyplot as plt
xticks = pd.date_range(start="10/10/2017", end="11/11/2017",
                       freq='D')
plt.xticks(xticks, xticks.strftime("%a %m-%d"))

TypeError: Cannot compare type 'Timestamp' with type 'float'

I'm not sure that was on purpose. So far every minor major release of pandas since my book came out broke the code in my book :-/ It would be nice to get deprecation warnings.

[jorisvandenbossche]

You ran into http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#no-automatic-matplotlib-converters

[amueller]

Thanks. It would be really nice to have deprecations.

[jorisvandenbossche]

It's certainly an annoying one, but I don't think it was possible to provide a deprecation warning. Given the apparent widespread dependence of people doing only matplotlib but relying on pandas converters, we might need to think about that though

[amueller]

I might not be familiar enough with the technical details, but if you registered the conversion, you could have by default registered a deprecated conversion, right?
This is like the 3rd time something around datetime broke without deprecation for about 20 lines of code.

[jorisvandenbossche]

Or we could consider to re-enable to register on pandas import, until there is basic support in matplotlib itself for datetime64 data (if there would be a clear timeline for that)

[amueller]

I don't care so much for support as for deprecations. This is printed on paper and shipped to people and now they have broken code. I guess it will break eventually, but it just breaks constantly without warning, which is a bit frustrating. I have no idea how to ensure that the code that I write in pandas now will still work in 3 month.

[jorisvandenbossche]

cc @TomAugspurger

you could have by default registered a deprecated conversion, right?

That might work yes.

But more general the problem is that this is very brittle: if someone first did a pandas .plot call, he/she wouldn't see the deprecation warning, but if that call is later removed, following matplotlib code can break ..

This is like the 3rd time something around datetime broke without deprecation for about 20 lines of code.

I know it is not always easy to get done, but that's why we also would really appreciate more feedback like this for a RC
(are the other 2 issues already reported or known?)

[jorisvandenbossche]

I don't care so much for support as for deprecations.

Well, if matplotlib actually supported datetime64, there would be less of a problem if pandas removed the automatic registration of its own converters.

This is printed on paper and shipped to people and now they have broken code. I guess it will break eventually, but it just breaks constantly without warning, which is a bit frustrating. I have no idea how to ensure that the code that I write in pandas now will still work in 3 month.

And I fully understand you here. I think we also underestimated the impact in this case. And I think we should consider reverting this partially

[TomAugspurger]

I might not be familiar enough with the technical details, but if you registered the conversion, you could have by default registered a deprecated conversion, right?

No, I don't think so. The converter wasn't deprecated, it's just no longer registered as a side effect of import pandas. To revert that change (and register it on import) we'll need to import matplotlib when someone does import pandas. But we want to avoid that, since that increases pandas' own import time. I'll see if there's any way to do that lazily though.

I have no idea how to ensure that the code that I write in pandas now will still work in 3 month.

If you're frustrated and venting, I understand and apologize. I underestimated the number of people relying on a pandas import modifying matplotlib's state. If you're looking for a solution, this is compatible going back a long ways:

from pandas.tseries import converter
converter.register()

# your plotting stuff

In general though, the old way of doing things wasn't great. I don't think an import pandas should modify matplotlib's global state, potentially squashing custom formatters someone else had already registered for datetimes.

until there is basic support in matplotlib itself for datetime64 data (if there would be a clear timeline for that)

Matplotlib 2.2 in January hopefully.

[jorisvandenbossche]

Trying to deprecate it properly probably means importing matplotlib again on import untill the deprecation can be removed. But given the feedback from several people I think we should consider that (it wad only about import time correct?)

[TomAugspurger]

it wad only about import time correct?

Correct.

Let me see if there's a way to issue a deprecation warning if it's used without explicitly registering. It may be worth the slower import time for now.

[amueller]

Yeah, I agree, modifying matplotlib state on import is not a great way to do it ;)
And sorry for venting. The other two breaking changes were in pandas 0.20 and pandas 0.19 in the same 20 lines, and also planned behavior changes from what I understood at that time.

[amueller]

(One was

pd.date_range(start="10/10/2017", end="11/11/2017", freq='D').astype("int").reshape(-1, 1)

breaking without deprecation between 0.19 and 0.20 I think, and the other one was

pd.date_range(start="10/10/2017", end="11/11/2017", freq='D').strftime("%s").astype("int")

breaking between 0.18 and 0.19 without deprecation. )

[jorisvandenbossche]

I opened #18301 to make sure we look at this for 0.21.1

[amueller]

thanks :)