DEPR: remove pd.TimeSeries & Series.is_time_series #15098

jreback · 2017-01-10T15:33:20Z

codecov-io · 2017-01-10T18:03:25Z

Codecov Report

Merging #15098 into master will increase coverage by 0.67%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #15098      +/-   ##
==========================================
+ Coverage   90.36%   91.04%   +0.67%     
==========================================
  Files         136      136              
  Lines       49552    49091     -461     
==========================================
- Hits        44780    44694      -86     
+ Misses       4772     4397     -375

Impacted Files	Coverage Δ
pandas/compat/pickle_compat.py	`68.29% <ø> (ø)`	✅
pandas/io/pytables.py	`93.05% <ø> (-0.86%)`	❌
pandas/core/series.py	`94.93% <ø> (-0.04%)`	❌
pandas/core/api.py	`100% <100%> (ø)`	✅
pandas/util/decorators.py	`67.32% <0%> (-1.28%)`	❌
pandas/core/frame.py	`97.73% <0%> (-0.1%)`	❌
pandas/util/print_versions.py	`15.71% <0%> (ø)`	✅
pandas/core/groupby.py	`95% <0%> (+0.01%)`	✅
pandas/types/cast.py	`85.65% <0%> (+0.2%)`	✅
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e15de4d...d9101bc. Read the comment docs.

jorisvandenbossche

+1!

Just some comments about the wording on pickle compat.

You removed all legacy pickles for versions < 0.13. Is that because just all files we were testing contained a time series? (I suppose that pickles files that do not have a TimeSeries in it can still be read in?)

jorisvandenbossche · 2017-01-10T19:28:50Z

doc/source/whatsnew/v0.20.0.txt

+``pd.TimeSeries`` was deprecated officially in 0.17.0, though has only been an alias since 0.13.0. It has
+been dropped in favor of ``pd.Series``.
+
+This *may* cause pickles / HDF5 files that were created with the ``pd.TimeSeries`` to become unreadable. (:issue:``15098).


Can you add how to solve this? (I suppose reading it in with older pandas version, and saving it again? Or is that not enough to remove the 'TimeSeries' aspect)

yes that's exactly how to solve it. I will update.

jorisvandenbossche · 2017-01-10T19:30:38Z

doc/source/whatsnew/v0.20.0.txt

@@ -122,6 +122,16 @@ Other enhancements
 Backwards incompatible API changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+.. _whatsnew.api_breaking.io_compat
+
+Pickle and HDF5 compat requires >= 0.13.0


This title is not fully clear. I would at least add 'requires pandas >= 0.13.0'. But you mean that pickle and HDF5 files created with pandas versions older than 0.13.0 may not be readable anymore?

no the issue is that if a TimeSeries is used it won't work (and that's why I deleted the pickle files < 0.13.0). To be honest I don't have a problem with just saying we are pickle back-compat to >= 0.13.

It still will work if TimeSeries is not present though.

yes, that is what I thought, but the title makes that not very clear

see my update on the doc if any more clear.

Yes, that's certainly better!
My comment here was mainly about the title (which people scan, so it would be nice if this is clear). The too long version would be something like "Reading pickle and HDF5 files requires them being saved with pandas >= 0.13" or "Possible incompatibility with pickle and HDF5 files created with pandas < 0.13.0", but thinking on more condensed version but that is still clear

jorisvandenbossche · 2017-01-10T19:43:47Z

Hmm, the statsmodels import is crashing on this ..
https://travis-ci.org/pandas-dev/pandas/jobs/190645844

And seems this is already fixed on master (statsmodels/statsmodels#2807), but AFAIK not yet in a released version of statsmodels.

jreback · 2017-01-10T21:39:29Z

@jorisvandenbossche on statsmodels. hmm, I think 0.8.0 will be out before we release 0.20.0, so this is mitigated. But this has been deprecated for quite a long time (and they haven't done releases in a while).

cc @josef-pkt

jorisvandenbossche · 2017-01-10T21:56:07Z

0.17 was released in October 2015, and they removed it in Feb 2016, so they were not that much behind :-)
It's indeed just a problem they didn't have a release for some time, but the problem is also that, although we think/hope it will be, we are not 100% sure statsmodels 0.8 will be released before pandas 0.20

jorisvandenbossche · 2017-01-10T22:02:56Z

doc/source/whatsnew/v0.20.0.txt

+was used. If you find yourself in this situation. You can use a recent prior version of pandas to read in
+your pickle / HDF5 files, then write them out again after applying the procedure below.
+
+  .. code-block:: ipython


this should have no indentation

jreback · 2017-01-11T19:00:38Z

@jorisvandenbossche

jorisvandenbossche · 2017-01-11T19:14:23Z

The PR is certainly fine for me, but I think merging now is possibly a bit problematic. As all users of pandas master that also use statsmodels (or eg seaborn which uses statsmodels for certain functionality) will get import errors. Not sure how many people it would impact though.
Maybe wait until a reaction of statsmodels on their envisioned timeline for a release?

jreback · 2017-01-11T19:18:23Z

@jorisvandenbossche that's fair, will leave it for now then.

jreback · 2017-02-09T00:53:29Z

ok, seems statsmodels 0.8.0 is out (I put our 2.7 build on it via pip install).

So this all checks out and everything runs with this PR (of course deprecation warnings for .ix, but that is another matter).

jreback · 2017-02-27T19:49:22Z

@jorisvandenbossche any more comments here?

jorisvandenbossche · 2017-02-27T20:33:08Z

Just to come back on the legacy pickles you removed. I tried to read those with this branch, and only half of them contain a TimeSeries (and thus fail, it are the 0.12 ones and the 0.11.0_x86_64_linux_3.3.0.pickle one).
Shouldn't we just keep the other ones as tests until we have a more pressing issue to declare we don't read in pickles < 0.13 ? They are currently working, so I don't see a reason to remove them

jreback · 2017-02-27T20:44:58Z

@jorisvandenbossche

used my new renaming feature in pickle_compat :>

87804b4

so these are read/converted just fine. Still I think we should 'drop' compat with < 0.13 anyhow. Its about time.

jorisvandenbossche · 2017-02-27T20:48:55Z

That also means that the whatsnew notice can be largely (the part on pickle compat) removed?

jorisvandenbossche · 2017-02-27T20:50:29Z

Let's have a separate issue about pickle compat. I don't really object dropping older versions, but as long it is not that much of effort to keep it (but I don't know how much effort you have to do in other cases to keep the compat), and we have tests that are passing, I also don't see a strong reason to drop it.

jreback · 2017-02-27T20:50:32Z

@jorisvandenbossche yes, though I am +1 on saying we are no longer < 0.13 compat :)

jreback · 2017-02-27T20:51:16Z

and this works for pickle. It doesn't for HDF5 (yes again may be able to fix it, but that's harder).

jorisvandenbossche · 2017-02-27T20:51:22Z

though I am +1 on saying we are no longer < 0.13 compat :)

OK, but that should not be tied to TimeSeries depr (in the whatsnew explanation)

jreback · 2017-02-27T20:51:43Z

see my prior (so I will revert on the pickle compat in whatsnew).

xref pandas-dev#10890 DEPR: remove Series.is_time_series in favor of Series.index.is_all_dates

jorisvandenbossche · 2017-02-27T20:52:02Z

Yep, just saw it, fine!

jreback · 2017-02-27T20:53:26Z

ok, lmk if you have any other comments. will merge on green.

xref pandas-dev#10890 Author: Jeff Reback <[email protected]> Closes pandas-dev#15098 from jreback/time_series and squashes the following commits: d9101bc [Jeff Reback] fix back-compat for < 0.13 ed57bd5 [Jeff Reback] DEPR: remove legacy pd.TimeSeries class in favor of pd.Series

jreback added the Deprecate Functionality to remove in pandas label Jan 10, 2017

jreback added this to the 0.20.0 milestone Jan 10, 2017

jreback force-pushed the time_series branch 2 times, most recently from 69f9441 to 6adcf80 Compare January 10, 2017 15:35

This was referenced Jan 10, 2017

DEPR: deprecations log for removed issues #13777

Closed

DEPR: 0.21 deprecations master issue #14220

Closed

jorisvandenbossche reviewed Jan 10, 2017

View reviewed changes

jreback force-pushed the time_series branch from 6adcf80 to 9b33228 Compare January 10, 2017 21:46

jorisvandenbossche reviewed Jan 10, 2017

View reviewed changes

jreback force-pushed the time_series branch from 9b33228 to 4a6f484 Compare January 11, 2017 00:11

jreback force-pushed the time_series branch 3 times, most recently from 680f964 to 56080ee Compare February 9, 2017 00:51

jreback force-pushed the time_series branch 3 times, most recently from 194b5c6 to 780a92c Compare February 12, 2017 17:39

jreback force-pushed the time_series branch from 780a92c to 411b170 Compare February 23, 2017 19:48

jreback force-pushed the time_series branch from 411b170 to 87804b4 Compare February 27, 2017 20:44

DEPR: remove legacy pd.TimeSeries class in favor of pd.Series

ed57bd5

xref pandas-dev#10890 DEPR: remove Series.is_time_series in favor of Series.index.is_all_dates

fix back-compat for < 0.13

d9101bc

jreback force-pushed the time_series branch from 87804b4 to d9101bc Compare February 27, 2017 20:52

jreback closed this in dd368eb Feb 28, 2017

jorisvandenbossche mentioned this pull request Mar 4, 2017

DEPR/CLN: remove SparseTimeSeries class (follow-up GH15098) #15567

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPR: remove pd.TimeSeries & Series.is_time_series #15098

DEPR: remove pd.TimeSeries & Series.is_time_series #15098

jreback commented Jan 10, 2017

codecov-io commented Jan 10, 2017 •

edited

Loading

jorisvandenbossche left a comment

jorisvandenbossche Jan 10, 2017

jreback Jan 10, 2017

jorisvandenbossche Jan 10, 2017

jreback Jan 10, 2017

jorisvandenbossche Jan 10, 2017

jreback Jan 10, 2017

jorisvandenbossche Jan 10, 2017

jorisvandenbossche commented Jan 10, 2017

jreback commented Jan 10, 2017

jorisvandenbossche commented Jan 10, 2017

jorisvandenbossche Jan 10, 2017

jreback commented Jan 11, 2017

jorisvandenbossche commented Jan 11, 2017

jreback commented Jan 11, 2017

jreback commented Feb 9, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

DEPR: remove pd.TimeSeries & Series.is_time_series #15098

DEPR: remove pd.TimeSeries & Series.is_time_series #15098

Conversation

jreback commented Jan 10, 2017

codecov-io commented Jan 10, 2017 • edited Loading

Codecov Report

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche commented Jan 10, 2017

jreback commented Jan 10, 2017

jorisvandenbossche commented Jan 10, 2017

Choose a reason for hiding this comment

jreback commented Jan 11, 2017

jorisvandenbossche commented Jan 11, 2017

jreback commented Jan 11, 2017

jreback commented Feb 9, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

jorisvandenbossche commented Feb 27, 2017

jreback commented Feb 27, 2017

codecov-io commented Jan 10, 2017 •

edited

Loading