Undesired UX behavior of DataFrame output in IPython Notebook

[danieljl]

From my previous line comment at be80898#commitcomment-11338872. cc @takluyver @jorisvandenbossche

Because DataFrame's _repr_html_ returns hardcoded div tag with style max-height:1000px, if the height of DataFrame's output exceeds 1000px, it will generate a scroll bar.

First, it is unnecessary because IPython Notebook has "toggle output scrolling" feature. Second, it causes UX problem if the height of browser resolution is below 1000px (user must scroll twice: the whole page and the generated div). Third, it can't make use of "toggle output scrolling" feature (if user turn it on, it will generate nested scroll bars (worse UX)). For more detailed explanations and examples, see the steps to reproduce below.

Pandas version: 0.16.1.

Steps to reproduce:

1. Open an IPython Notebook and paste this code. It will generate an inner scroll bar because the output's height exceeds 1000px.

   import pandas as pd
   pd.DataFrame(list(range(100)))

2. If you want to see all rows, first you should scroll the page.
   

3. After you reach the bottom of the generated ouput's div, you should scroll again the inner scroll bar.
   

4. If you want to toggle the IPython Notebook's "toggle output scrolling" feature, you get this.
   

Desired behaviors:

• When "output scrolling" feature is off
  
• When "output scrolling" feature is on
  

Solution:
Remove style property from div tag that DataFrame's _repr_html_ returns.
Old: <div style="max-height:1000px;max-width:1500px;overflow:auto;">\n
New: <div>\n

Why should max-width be removed as well? Because it also generates far worse UX. If you have so many columns that the horizontal scroll bar generated, you cannot scroll it. You must refresh the page to be able to scroll it.

[shoyer]

Thanks for the clear documentation of the issue with the screenshots. I agree, we should fix this.

[shoyer]

Does IPython notebook cleanly handle horizontal scrolling when necessary if max-width is removed?

[takluyver]

IIRC, in IPython 3 we put the output in its own scrollable container if it is wider than the notebook. In earlier versions, the page would just get wider, so the whole page would get a horizontal scrollbar.

[shoyer]

Indeed, I verified that I get the proper horizontal scrolling behavior in IPython 3.

My only concern here is that IPython 3 may still be a little too new (3 months old) for us to change this -- we might break some folks still using earlier versions. On the other hand, upgrading IPython is pretty easy/seamless (in my experience).

Any other thoughts here?

[danieljl]

@shoyer I tested using IPython 2.4.1 and pandas with DataFrame's max-width removed. As @takluyver said, in IPython 2 the page just gets wider. Yup it is ugly, but it works. Is it acceptable?

[shoyer]

At least it only makes the one output cell wider -- it could be far worst (e.g., the entire page gets stretched).

For things like this, that work either way but are ugly for some users, I think we probably should make things nicer for users who are using the latest version of IPython. Of users who are using the latest version of pandas, a significant fraction are probably also keeping IPython up to date. So I think this change is a good idea.

[jreback]

closed by #10232

[jeremybickerstaffe]

I've encountered a bug which may be related which results in very strange multiple scrollbar behaviour with a particular csv dataset. This is with up to data Anaconda distribution on Windows - pandas 0.16.2 and Python 2.7.9 Unfortunately I wasn't able to generate the problem with a simple script, it occurs with the (anonymised) dataset at this link:

https://www.dropbox.com/s/gdoow6ro8u7yn70/pandas%20multiple%20scrollbars%20bug.csv?dl=0

Steps to reproduce: open a new iPython notebook and paste the following code

import pandas as pd
df = pd.read_csv('pandas multiple scrollbars bug.csv')
df

[jreback]

@jeremybickerstaffe you didn't specify the ipython version. This is fixed in 0.16.2 for ipython >= 3.0.0.

[jeremybickerstaffe]

Thanks for responding @jreback
My iPython is version 3.2.1.

[jorisvandenbossche]

I can't reproduce this with your file. Can you show the output of pd.show_versions()

[jeremybickerstaffe]

Sure, the output is:

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.9.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.16.2
nose: 1.3.4
Cython: 0.22
numpy: 1.9.2
scipy: 0.15.1
statsmodels: 0.6.1
IPython: 3.2.1
sphinx: 1.3.1
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.4
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.6.7
lxml: 3.4.2
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 0.9.9
pymysql: 0.6.6.None
psycopg2: None

[jorisvandenbossche]

Strange, I can't reproduce it with the same version of pandas and IPython. Which browser are you using?

[jeremybickerstaffe]

Chrome Version 44.0.2403.125 m (64-bit).

It does appear to be very much an edge case (but very much repeatable for me - it affects lots of my files in this format). I found that bizarre changes, like changing the name of the file I put on Dropbox or changing the reading code to remove the assignment to 'df' this below can remove the rendering problem.

import pandas as pd
pd.read_csv('pandas multiple scrollbars bug.csv')

[SoundBoySelecta]

Ive had to use df.style to get scroll bars (vert/horz)