ENH: show numbers on .info() with verbose flag

[Roymprog]

• closes feature wanted: pd.DataFrame.info() should show line numbers #17304
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[WillAyd]

This is a duplicate of #28696 which was a continuation of #17332 - did you address comments from those? @Alexandreae were you still working on that?

[Roymprog]

Apologies, I completely missed this entire discussion. I was not aware someone else was working on it. I did see the comment left in #17304, but I started working on it as that was months ago.

Let me have a look at the comments, then I'll finish up. Still, you can close this if @Alexandreae responds.

[pep8speaks]

Hello @Roymprog! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-01-02 20:42:37 UTC

[Roymprog]

All the suggestions from #17332 have been implemented. Please let me know what you think

[WillAyd]

Cool thanks for picking this up. cc @bashtage who had some thoughts on original design

[bashtage]

Output looks pretty good and better than current. Does it work well with very long variable names? Or other long info properties? Wondering since the ----- bits are new.

[Roymprog]

Sample for long column name variables.
Script:

col_count = 10
max_col_name = 80
df = pd.DataFrame(np.random.rand(4, col_count), columns=[
          '%s' % (np.random.randint(2, max_col_name)*'a') for x in range(col_count)])
df.info(verbose=True)

Output:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4 entries, 0 to 3
Data columns (total 10 columns):
 #   Column                                                                         Non-Null Count  Dtype  
---  ------                                                                         --------------  -----  
 0   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                               4 non-null      float64
 1   aaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                   4 non-null      float64
 2   aaaaaaaa                                                                       4 non-null      float64
 3   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa  4 non-null      float64
 4   aaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                  4 non-null      float64
 5   aaaaaaaaaaaaaaaaaaaaaaaaaaa                                                    4 non-null      float64
 6   aaaaaaaaaaaaaaa                                                                4 non-null      float64
 7   aaaaaaaaa                                                                      4 non-null      float64
 8   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                     4 non-null      float64
 9   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa   4 non-null      float64

Regarding dtypes, there's a test, and for non-null count there is a docstring test. For the numbering I added an assertion in the verbosity test, as I did not feel like posting a thousand line output here.

[jreback]

this needs some eye balls in it from @pandas-dev/pandas-core

[bashtage]

Look readable in this extreme case.

[WillAyd]

I think this is nice and improves readability

[WillAyd]

@pandas-dev/pandas-core anyone else care to take a look?

[WillAyd]

ping @pandas-dev/pandas-core for any other comments

[mroeschke]

Overall an improvement from the prior output. One small comment.

[WillAyd]

@jreback care to take a look or merge?

[jreback]

yep will again

[jreback]

can you make a small sub-section showing these changes. move to the api breaking section as this is a change in the output repr.

[Roymprog]

I added the difference to the Backwards incompatible API changes section. Hope that is what you meant

[WillAyd]

@Roymprog looks like some merge conflicts - can you clean up and re-push?

[WillAyd]

@Roymprog can you fix up merge conflict?

[Roymprog]

I will in the coming week

[WillAyd]

@jreback any outstanding thoughts on this?

[jreback]

lgtm. small doc-presentation change. ping on green.

[jreback]

can you add a title and underline here (like other sub-sections).

[Roymprog]

Added an underlined title for the new feature, thanks for the feedback

[TomAugspurger]

@Roymprog CI failure:

Check for use of not concatenated strings
[error]./pandas/tests/frame/test_repr_info.py:216:String unnecessarily split in two by black. Please merge them manually.
Check for use of not concatenated strings DONE

[jreback]

thanks @Roymprog very . nice!

[Roymprog]

Thanks for the fix @TomAugspurger.

Thank you all for all the feedback I received. This was my first open source pull request, thanks for making it happen 😃