BUG: PeriodIndex._shallow_copy doesn't roundtrip asi8 values correctly

[jschendel]

On master:

In [1]: import pandas as pd; pd.__version__
Out[1]: '0.24.0.dev0+1340.g8c58817bd'

In [2]: pi = pd.period_range('2018-01-01', periods=3, freq='2D')

In [3]: pi
Out[3]: PeriodIndex(['2018-01-01', '2018-01-03', '2018-01-05'], dtype='period[2D]', freq='2D')

In [4]: pi._shallow_copy(pi.asi8, freq=pi.freq)
Out[4]: PeriodIndex(['2018-01-02', '2018-01-04', '2018-01-06'], dtype='period[2D]', freq='2D')

It appears that values are shifted forward by freq.n - 1 base frequencies, e.g. if freq='5D' then the above will result in a shift of '2018-01-01' --> '2018-01-05'.

Note that this is working on 0.23.4:

In [1]: import pandas as pd; pd.__version__
Out[1]: '0.23.4'

In [2]: pi = pd.period_range('2018-01-01', periods=3, freq='2D')

In [3]: pi
Out[3]: PeriodIndex(['2018-01-01', '2018-01-03', '2018-01-05'], dtype='period[2D]', freq='2D')

In [4]: pi._shallow_copy(pi.asi8, freq=pi.freq)
Out[4]: PeriodIndex(['2018-01-01', '2018-01-03', '2018-01-05'], dtype='period[2D]', freq='2D')

cc @jbrockmendel : Am I correct in expecting that Out[3] and Out[4] should be the same?

[jbrockmendel]

Definitely weird. @TomAugspurger has worked on this more recently than me. I'll try to take a look over the weekend.

[jschendel]

FYI there is a test that's being xfailed due to this issue, so can remove it when someone has a change to look into this:

pandas/pandas/tests/indexes/period/test_period.py

Lines 41 to 47 in 159772d

	@pytest.mark.parametrize('index', [
	pd.period_range('2000-01-01', periods=3, freq='D'),
	pytest.param(
	pd.period_range('2001-01-01', periods=3, freq='2D'),
	marks=pytest.mark.xfail(reason='GH 24391')),
	pd.PeriodIndex(['2001-01', 'NaT', '2003-01'], freq='M')])
	def test_repeat_freqstr(self, index, use_numpy):

[TomAugspurger]

The bug seems to be in asfreq

(Pdb) values
<PeriodArray>
['2018-01-01', '2018-01-03', '2018-01-05']
Length: 3, dtype: period[2D]
(Pdb) values.asfreq(values.freq)
<PeriodArray>
['2018-01-02', '2018-01-04', '2018-01-06']
Length: 3, dtype: period[2D]

Not so much a bug, more how we're calling it. The how='E' means we align the end, but I guess we want start in this case.

[TomAugspurger]

I don't really understand why _shallow_copy would be calling asfreq in the first place. Ideally the PeriodArray would have the correct freq.

[TomAugspurger]

I think the solution is to not pass freq to _shallow_copy.

I don't plan to work on this until DatetimeArray / TimedeltaArray (#24024) are in. I think it'll be easier to fix once those are all consistent.