(WIP) BUG/CLN: Better timeseries plotting / refactoring tsplot

[sinhrks]

Must be revisited after #7717.
#6608 seems to be solved by following 4 fixes.

• PeriodIndex should support the same freqs as DatetimeIndex (Related to BUG: Series.plot() doesn't work with CustomBusinessDay frequency #7222, maybe solved by ENH: Cleanup backend for Offsets and Period #5148)
• Better logic to find common divisor frequency (try to use tsplot as much)
• When plotting with x_compat to the ax which already holds tsplot lines, tsplot lines must be redrawn on x_compat coordinates.
  • Check whether to_timestamp(how='e') always revert PeriodIndex back to original DatetimeIndex
• If target ax already holds x_compat lines, continuous plot should be drawn on x_compat coordinates (current version already have a logic, but cannot work if ax once have freq property).

Other refactoring:

• Do not re-plot every row in tsplot
• Simplify LinePlot flow (separated as CLN: Simplify LinePlot flow #7717)
• Store plot_func for line and area mixed time-series plot (separated as BUG: Repeated timeseries plot may result in incorrect kind #7733)

Result using current PR (modified #6608 a little to show legend).

s1 = pd.Series([1, 2, 3], index=[datetime.datetime(1995, 12, 31),
                                 datetime.datetime(2000, 12, 31),
                                 datetime.datetime(2005, 12, 31)], name='idx1')
s2 = pd.Series([1, 2, 3], index=[datetime.datetime(1997, 12, 31),
                                 datetime.datetime(2003, 12, 31),
                                 datetime.datetime(2008, 12, 31)], name='idx2')

ax = s1.plot(legend=True)
ax = s2.plot(legend=True)
s1.plot(ax=ax, legend=True)

One question is whether tsplot is categorized as public function? If so, I'll prepare separate func.

[jreback]

@sinhrks status?

[sinhrks]

@jreback As #5148 was once closed, I'll skip top 2 items and add a test for this in this weekend. Top 2 items are not required to solve the #6608.

[jreback]

@sinhrks status on this?

[jreback]

@sinhrks status?

[jreback]

@TomAugspurger can you pick this up for 0.15? or can defer to 0.15.1

[TomAugspurger]

Let me look at it today.

[TomAugspurger]

This is an improvement (just a couple of fixups first). But still not entirely there.

The following two plots should be identical I think:

In [28]: import datetime

In [29]: pidx = pd.PeriodIndex(start='2010-01-01', freq='m', periods=5)

In [30]: didx = pd.DatetimeIndex(['2010-01-01', '2010-02-05', '2010-03-10', '2010-04-15', '2010-05-25'])

In [31]: x1 = pd.Series(np.arange(5), index=pidx)

In [32]: x2 = pd.Series(np.arange(5), index=didx)

In [33]: ax = x1.plot(marker='o')

In [34]: x2.plot(marker='^', ax=ax)
Out[34]: <matplotlib.axes._subplots.AxesSubplot at 0x11758ae80>

In [35]: ax.set_title("Period then DatetimeIndex")
Out[35]: <matplotlib.text.Text at 0x117b03160>

In [38]: plt.close()

In [39]: ax = x2.plot(marker='^')

In [40]: x1.plot(ax=ax, marker='o')
Out[40]: <matplotlib.axes._subplots.AxesSubplot at 0x117af5a58>

In [41]: ax.set_title("Datetime then PeriodIndex")
Out[41]: <matplotlib.text.Text at 0x117b88828>

[jreback]

@TomAugspurger push?

[jreback]

@TomAugspurger status?

[jreback]

@TomAugspurger what are we oing with this (and #6608) ?

[TomAugspurger]

Push I think. Right now it's only fixed in one direction iirc.

[jreback]

@sinhrks, @TomAugspurger what is status on this?

[sinhrks]

I'm revisiting this, and want to change tsplot as below to make the fix simple:

• Allow to accept DataFrame.
• Remove legend handling.
• Change _maybe_resample to allow Period-DatetimeIndex roundtrip in a single rule.

Let me confirm tsplot is considered to be a public function. If so, direct-call of tsplot can be deprecated because now Series.plot should output the time series plotting?

[jreback]

@TomAugspurger can you have a look

[TomAugspurger]

Yeah, I'll see where we are at.

[sinhrks]

I couldn't finish it yet because of the following (the same one as the @TomAugspurger pointed):

Check whether to_timestamp(how='e') always revert PeriodIndex back to original DatetimeIndex

Based on the following code, there could be no single logic to revert converted PeriodIndex to original. Thus I think we should change the below logic to cover all the cases.

https://github.com/pydata/pandas/blob/master/pandas/tseries/plotting.py#L78

Though this PR cannot cover all the issues, it can fix original one. Thus, one option is move this forward once leaving above as it is.

[jreback]

@sinhrks status?

[sinhrks]

I couldn't investigate the detail above yet. Would like to finish #9814 first which also has tsplot related changes.

[jreback]

@sinhrks want to update

[sinhrks]

Revisiting this... I believe we have to clarify the plotting rules first. This should cause current behavior changes.

• Repeated time series plot should output the same result regardless of plotting order.
• DatetimeIndex should be regarded as PeriodIndex (and converted to PeriodIndex) if it has freq/inferred_freq
• PeriodIndex is regarded as "regular" time series. DatetimeIndex doesn't have freq/inferred_freq is regarded as "irregular" time series.
• If any of time sereis is "irregular", all the time series should be plotted based on absolute time stamp. No freq adjustment needed.
• If all the time series are "regular", period adjustment should be performed to take highest frequency.
• When PeriodIndex is resampled, the begining time of the "Period" should be taken. e.g. resampling monthly Period "2011-01" with daily freq should result in "2011-01-01", not "2011-01-31".

Based on the rule, above @TomAugspurger 's example should result in latter(bottom) output in both cases.

I've created a demo on Gist, but not able to cover all conditions yet.

https://gist.github.com/sinhrks/e12cc7caf753097a2ef0

[TomAugspurger]

I think those all seem reasonable...

[jreback]

@sinhrks @TomAugspurger status of this?

[jreback]

@sinhrks status?

[jreback]

status?

[jreback]

status?

[jreback]

closing, but @sinhrks pls reopen if you want / can fix.