Skip to content

Inconsistent display of numbers in dataframe #6842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
markrichardson opened this issue Apr 8, 2014 · 5 comments
Closed

Inconsistent display of numbers in dataframe #6842

markrichardson opened this issue Apr 8, 2014 · 5 comments
Labels
Output-Formatting __repr__ of pandas objects, to_string

Comments

@markrichardson
Copy link

Using pandas 0.13.1, I see see that there is an apparent inconsistency in the format in which floating-point numbers in my dataframe are displayed. In particular, the numbers in the first column appear to be displayed in the form '%1.8e', whereas the other numbers are displayed as '%1.6f'.

                   col_0       col_1       col_2       col_3
2013-01-18  3.132256e+00         NaN         NaN         NaN
2013-01-21  3.513191e+00         NaN         NaN         NaN
2013-01-22  2.581368e+00         NaN         NaN         NaN
2013-01-23  1.751014e+00         NaN         NaN         NaN
2013-01-24  2.508899e+00         NaN         NaN         NaN
2013-01-25  4.085701e+00         NaN         NaN         NaN
2013-01-28  3.400867e+00         NaN         NaN         NaN
2013-01-29  3.030346e+00         NaN         NaN         NaN
2013-01-30  2.934522e+00         NaN         NaN         NaN
2013-01-31  6.548878e-01         NaN         NaN         NaN
2013-02-01  4.743263e-01         NaN         NaN         NaN
2013-02-04  5.372891e-02         NaN         NaN         NaN
2013-02-05  8.271173e-02         NaN         NaN         NaN
2013-02-06  9.247646e-03         NaN         NaN         NaN
2013-02-07  1.499508e-03         NaN         NaN         NaN
2013-02-08  6.927766e-04         NaN         NaN         NaN
2013-02-11  7.232857e-05         NaN         NaN         NaN
2013-02-12  3.274444e-05         NaN         NaN         NaN
2013-02-13  4.520291e-07         NaN         NaN         NaN
2013-02-14  2.092958e-16         NaN         NaN         NaN
2013-02-15  0.000000e+00    6.815180         NaN         NaN
2013-02-18           NaN    7.581717         NaN         NaN
2013-02-19           NaN   15.039587         NaN         NaN
2013-02-20           NaN   10.740931         NaN         NaN
2013-02-21           NaN    4.101203         NaN         NaN
2013-02-22           NaN    9.078457         NaN         NaN
2013-02-25           NaN   13.372686         NaN         NaN
2013-02-26           NaN    4.394059         NaN         NaN
2013-02-27           NaN    4.621631         NaN         NaN
2013-02-28           NaN    5.737833         NaN         NaN
2013-03-01           NaN    3.454591         NaN         NaN
2013-03-04           NaN    2.521326         NaN         NaN
2013-03-05           NaN    9.589268         NaN         NaN
2013-03-06           NaN    8.321083         NaN         NaN
2013-03-07           NaN    8.069279         NaN         NaN
2013-03-08           NaN   16.680768         NaN         NaN
2013-03-11           NaN    9.940089         NaN         NaN
2013-03-12           NaN    5.922898         NaN         NaN
2013-03-13           NaN    2.206807         NaN         NaN
2013-03-14           NaN    9.449342         NaN         NaN
2013-03-15           NaN    0.000000    5.331617         NaN
2013-03-18           NaN         NaN    3.320303         NaN
2013-03-19           NaN         NaN    2.480900         NaN
2013-03-20           NaN         NaN    2.787095         NaN
2013-03-21           NaN         NaN    2.157647         NaN
2013-03-22           NaN         NaN    2.795969         NaN
2013-03-25           NaN         NaN    0.884890         NaN
2013-03-26           NaN         NaN    0.425347         NaN
2013-03-27           NaN         NaN    0.376083         NaN
2013-03-28           NaN         NaN    0.332580         NaN
2013-04-02           NaN         NaN    0.689754         NaN
2013-04-03           NaN         NaN    0.256095         NaN
2013-04-04           NaN         NaN    0.157584         NaN
2013-04-05           NaN         NaN    0.084014         NaN
2013-04-08           NaN         NaN    0.023838         NaN
2013-04-09           NaN         NaN    0.007898         NaN
2013-04-10           NaN         NaN    0.064029         NaN
2013-04-11           NaN         NaN    0.047283         NaN
2013-04-12           NaN         NaN    0.010981         NaN
2013-04-15           NaN         NaN    0.000285         NaN
                     ...         ...         ...         ...

[64 rows x 4 columns]

df.values
array([[  3.13225571e+00,              nan,              nan,
                     nan],
       [  3.51319052e+00,              nan,              nan,
                     nan],
       [  2.58136815e+00,              nan,              nan,
                     nan],
       [  1.75101397e+00,              nan,              nan,
                     nan],
       [  2.50889891e+00,              nan,              nan,
                     nan],
       [  4.08570093e+00,              nan,              nan,
                     nan],
       [  3.40086744e+00,              nan,              nan,
                     nan],
       [  3.03034635e+00,              nan,              nan,
                     nan],
       [  2.93452229e+00,              nan,              nan,
                     nan],
       [  6.54887773e-01,              nan,              nan,
                     nan],
       [  4.74326267e-01,              nan,              nan,
                     nan],
       [  5.37289144e-02,              nan,              nan,
                     nan],
       [  8.27117286e-02,              nan,              nan,
                     nan],
       [  9.24764587e-03,              nan,              nan,
                     nan],
       [  1.49950795e-03,              nan,              nan,
                     nan],
       [  6.92776578e-04,              nan,              nan,
                     nan],
       [  7.23285714e-05,              nan,              nan,
                     nan],
       [  3.27444365e-05,              nan,              nan,
                     nan],
       [  4.52029103e-07,              nan,              nan,
                     nan],
       [  2.09295814e-16,              nan,              nan,
                     nan],
       [  0.00000000e+00,   6.81517955e+00,              nan,
                     nan],
       [             nan,   7.58171652e+00,              nan,
                     nan],
       [             nan,   1.50395868e+01,              nan,
                     nan],
       [             nan,   1.07409307e+01,              nan,
                     nan],
       [             nan,   4.10120346e+00,              nan,
                     nan],
       [             nan,   9.07845736e+00,              nan,
                     nan],
       [             nan,   1.33726856e+01,              nan,
                     nan],
       [             nan,   4.39405879e+00,              nan,
                     nan],
       [             nan,   4.62163060e+00,              nan,
                     nan],
       [             nan,   5.73783295e+00,              nan,
                     nan],
       [             nan,   3.45459132e+00,              nan,
                     nan],
       [             nan,   2.52132635e+00,              nan,
                     nan],
       [             nan,   9.58926847e+00,              nan,
                     nan],
       [             nan,   8.32108299e+00,              nan,
                     nan],
       [             nan,   8.06927868e+00,              nan,
                     nan],
       [             nan,   1.66807682e+01,              nan,
                     nan],
       [             nan,   9.94008926e+00,              nan,
                     nan],
       [             nan,   5.92289775e+00,              nan,
                     nan],
       [             nan,   2.20680680e+00,              nan,
                     nan],
       [             nan,   9.44934225e+00,              nan,
                     nan],
       [             nan,   0.00000000e+00,   5.33161661e+00,
                     nan],
       [             nan,              nan,   3.32030302e+00,
                     nan],
       [             nan,              nan,   2.48089964e+00,
                     nan],
       [             nan,              nan,   2.78709474e+00,
                     nan],
       [             nan,              nan,   2.15764709e+00,
                     nan],
       [             nan,              nan,   2.79596877e+00,
                     nan],
       [             nan,              nan,   8.84890216e-01,
                     nan],
       [             nan,              nan,   4.25347224e-01,
                     nan],
       [             nan,              nan,   3.76083458e-01,
                     nan],
       [             nan,              nan,   3.32579876e-01,
                     nan],
       [             nan,              nan,   6.89754124e-01,
                     nan],
       [             nan,              nan,   2.56095287e-01,
                     nan],
       [             nan,              nan,   1.57583997e-01,
                     nan],
       [             nan,              nan,   8.40144066e-02,
                     nan],
       [             nan,              nan,   2.38379017e-02,
                     nan],
       [             nan,              nan,   7.89829707e-03,
                     nan],
       [             nan,              nan,   6.40289210e-02,
                     nan],
       [             nan,              nan,   4.72825870e-02,
                     nan],
       [             nan,              nan,   1.09810537e-02,
                     nan],
       [             nan,              nan,   2.84832649e-04,
                     nan],
       [             nan,              nan,   1.58283327e-06,
                     nan],
       [             nan,              nan,   3.91397716e-12,
                     nan],
       [             nan,              nan,   6.65980697e-23,
                     nan],
       [             nan,              nan,   0.00000000e+00,
          4.83556226e+00]])

df.index.values
array(['2013-01-18T00:00:00.000000000+0000',
       '2013-01-21T00:00:00.000000000+0000',
       '2013-01-22T00:00:00.000000000+0000',
       '2013-01-23T00:00:00.000000000+0000',
       '2013-01-24T00:00:00.000000000+0000',
       '2013-01-25T00:00:00.000000000+0000',
       '2013-01-28T00:00:00.000000000+0000',
       '2013-01-29T00:00:00.000000000+0000',
       '2013-01-30T00:00:00.000000000+0000',
       '2013-01-31T00:00:00.000000000+0000',
       '2013-02-01T00:00:00.000000000+0000',
       '2013-02-04T00:00:00.000000000+0000',
       '2013-02-05T00:00:00.000000000+0000',
       '2013-02-06T00:00:00.000000000+0000',
       '2013-02-07T00:00:00.000000000+0000',
       '2013-02-08T00:00:00.000000000+0000',
       '2013-02-11T00:00:00.000000000+0000',
       '2013-02-12T00:00:00.000000000+0000',
       '2013-02-13T00:00:00.000000000+0000',
       '2013-02-14T00:00:00.000000000+0000',
       '2013-02-15T00:00:00.000000000+0000',
       '2013-02-18T00:00:00.000000000+0000',
       '2013-02-19T00:00:00.000000000+0000',
       '2013-02-20T00:00:00.000000000+0000',
       '2013-02-21T00:00:00.000000000+0000',
       '2013-02-22T00:00:00.000000000+0000',
       '2013-02-25T00:00:00.000000000+0000',
       '2013-02-26T00:00:00.000000000+0000',
       '2013-02-27T00:00:00.000000000+0000',
       '2013-02-28T00:00:00.000000000+0000',
       '2013-03-01T00:00:00.000000000+0000',
       '2013-03-04T00:00:00.000000000+0000',
       '2013-03-05T00:00:00.000000000+0000',
       '2013-03-06T00:00:00.000000000+0000',
       '2013-03-07T00:00:00.000000000+0000',
       '2013-03-08T00:00:00.000000000+0000',
       '2013-03-11T00:00:00.000000000+0000',
       '2013-03-12T00:00:00.000000000+0000',
       '2013-03-13T00:00:00.000000000+0000',
       '2013-03-14T00:00:00.000000000+0000',
       '2013-03-15T00:00:00.000000000+0000',
       '2013-03-18T00:00:00.000000000+0000',
       '2013-03-19T00:00:00.000000000+0000',
       '2013-03-20T00:00:00.000000000+0000',
       '2013-03-21T00:00:00.000000000+0000',
       '2013-03-22T00:00:00.000000000+0000',
       '2013-03-25T00:00:00.000000000+0000',
       '2013-03-26T00:00:00.000000000+0000',
       '2013-03-27T00:00:00.000000000+0000',
       '2013-03-28T00:00:00.000000000+0000',
       '2013-04-02T01:00:00.000000000+0100',
       '2013-04-03T01:00:00.000000000+0100',
       '2013-04-04T01:00:00.000000000+0100',
       '2013-04-05T01:00:00.000000000+0100',
       '2013-04-08T01:00:00.000000000+0100',
       '2013-04-09T01:00:00.000000000+0100',
       '2013-04-10T01:00:00.000000000+0100',
       '2013-04-11T01:00:00.000000000+0100',
       '2013-04-12T01:00:00.000000000+0100',
       '2013-04-15T01:00:00.000000000+0100',
       '2013-04-16T01:00:00.000000000+0100',
       '2013-04-17T01:00:00.000000000+0100',
       '2013-04-18T01:00:00.000000000+0100',
       '2013-04-19T01:00:00.000000000+0100'], dtype='datetime64[ns]')
@jreback
Copy link
Contributor

jreback commented Apr 8, 2014

its the tiny value (1e-16). you can try display.chop_threshold I think

@jreback
Copy link
Contributor

jreback commented Apr 8, 2014

@gdirichlet how would you see this displayed? IIRC this is on purpose

@markrichardson
Copy link
Author

The trade-off here seems to be retention of visual information versus loss of aesthetic appeal. Personally I value the second enough to be prepared to extract the column in question in order to examine further if I see "0.000000".

@jreback
Copy link
Contributor

jreback commented Apr 8, 2014

Here's a couple of options

not sure if this is a bug, or just what,
see also #6839

In [1]: df = DataFrame({ 'A' : [1.0,1e-16], 'B' : [np.nan,5.0] })

In [2]: df
Out[2]: 
              A   B
0  1.000000e+00 NaN
1  1.000000e-16   5

[2 rows x 2 columns]

In [4]: pd.describe_option('chop_threshold')
display.chop_threshold: [default: None] [currently: None]
: float or None
        if set to a float value, all float values smaller then the given threshold
        will be displayed as exactly 0 by repr and friends.

In [5]: pd.set_option('chop_threshold',1e-6)

In [6]: df
Out[6]: 
              A   B
0  1.000000e+00 NaN
1             0   5

[2 rows x 2 columns]

In [8]: pd.describe_option('float_format')
display.float_format: [default: None] [currently: None]
: callable
        The callable should accept a floating point number and return
        a string with the desired format of the number. This is used
        in some places like SeriesFormatter.
        See core.format.EngFormatter for an example.

In [11]: pd.set_option('float_format',lambda x: "%.6f" % x)

In [12]: df
Out[12]: 
         A        B
0 1.000000      nan
1 0.000000 5.000000

[2 rows x 2 columns]

@jseabold jseabold mentioned this issue Mar 8, 2015
Closed
@jreback
Copy link
Contributor

jreback commented Mar 8, 2015

closing as xref to #6842

@jreback jreback closed this as completed Mar 8, 2015
@jreback jreback added the Output-Formatting __repr__ of pandas objects, to_string label Mar 8, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Output-Formatting __repr__ of pandas objects, to_string
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jreback @markrichardson