BUG: yield correct Series subclass in df.iterrows() (#13977)

[kernc]

• closes df.iterrows() constructs Series instead of its proper subclass #13977
• tests added / passed
• passes git diff upstream/master | flake8 --diff
• whatsnew entry

[jreback]

pls read the instructions. we explicity call for tests.

[kernc]

Ah, sorry. I made sure existing tests passed. Now I also added a new one.

[codecov-io]

Current coverage is 85.27% (diff: 100%)

Merging #13978 into master will decrease coverage by <.01%

@@             master     #13978   diff @@
==========================================
  Files           139        139          
  Lines         50455      50447     -8   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43026      43018     -8   
  Misses         7429       7429          
  Partials          0          0          

Powered by Codecov. Last update 6645b2b...2820315

[jreback]

call this klass

[jreback]

minor changes. ping when green.

[kernc]

Revised. Hope it's ok.

I did notice some more Series and DataFrame calls where self._constructor_sliced or self._constructor might make sense (e.g. in DataFrame.apply() and submethods). What is the rationale for ever using Series instead of self._constructor_sliced?

[sinhrks]

can u also add assertIsInstance explicitly to check it is SubclassedSeries? assert_series_equal only checks class identity, not check what it is.

[kernc]

Even though this is already tested in test_indexing_sliced which tests df.loc?

[sinhrks]

Thx for the update. Yes fixing relevant cases are appreciated (OK for separate PR). The remaining normal constructor is mostly because of historical reason, and can be replaced to constructor....

[kernc]

But what are relevant cases? If DataFrame._constructor must always be a subtype of DataFrame (and so on for _constructor_sliced and _constructor_expanddim), why not just do a mass renaming of DataFrame into self._constructor (etc.) in all methods?

Using Python's inspect module, one could even write a simple test ensuring no references to Series, DataFrame and Panel are hardcoded. 💡

[sinhrks]

It's simply because no one can take a time to do it (add tests). Note that sigatures must be changed sometimes, see #13208.

[kernc]

Thanks, will look into it.

[kernc]

Adding just __finalize__() made tests no longer pass.

I added a RFC commit with one proposed solution. It basically makes only DataFrame inherit DataFrame's _metadata properties, Series Series', and so on. I don't know whether this opposes any established behavior / documentation. Doesn't seem to. The tests will pass. 🙏

[jreback]

the inheriting of metadata should be on a like object. However this would belong in another PR.

[sinhrks]

@kernc thx for theck. I've thought finalize has been generally called, sorry.

[kernc]

So this appears to be it for this PR. I'll be opening the other one shortly. Thanks.

[jreback]

lgtm. ping on green.

[kernc]

So this is green. I'm not sure why Xcode build is marked as failed.

[jreback]

thanks!