ENH: Removing unnecessary whitespace when formatting to a HTML table. #5012

alefnula · 2013-09-27T21:15:14Z

Closes #4987.

The problem is that all formatters always call _make_fixed_width. So, the only solution I found is just to strip the unnecessary whitespace.

jreback · 2013-09-28T01:43:25Z

pandas/core/format.py

@@ -734,7 +734,7 @@ def _write_body(self, indent):

        fmt_values = {}
        for i in range(len(self.columns)):
-            fmt_values[i] = self.fmt._format_col(i)
+            fmt_values[i] = [item.strip() for item in self.fmt._format_col(i)]


is it possible to pass an argument to _format_col that controls this? e.g. in the HTMLFormatter it will not include the whitespace, all other formatters will be unchanged.

@jreback It uses the format_array function, which uses an appropriate ArrayFormatter. They all call _make_fixed_width which can justify either right or left. A third value can be added justify=None, but then every ArrayFormatter should be changed to support justify=None. I can add that, no problem :), but it will be a lot of changes, and I didn't wanted to do that just for this small enhancement that could be fixed in one line. If you think that the more general solution is better, I'll do that.

@jreback But maybe this isn't such a bad idea since then I can add justify=center too. Currently only left and right are supported.

jreback · 2013-09-28T02:18:38Z

yep

alefnula · 2013-09-28T11:06:55Z

@jreback OK, I'll implement the additional justifications.

alefnula · 2013-09-28T22:28:50Z

It turned out easier than I thought. Hope I haven't missed something though...

cpcloud · 2013-09-28T22:38:39Z

pandas/core/format.py

        justfunc = lambda self, x: self.rjust(x)
+    elif justify == 'center':
+        justfunc = lambda self, x: self.center(x)


can these be of the form str.center, str.rjust etc?

No, because it can also be a unicode.

sorry i meant compat.text_type.center ... or are we allowing str and unicode? if allowing both then disregard my suggestion

That's just unicode in python 2 and will throw an error if receives a str:

>>> pd.compat.text_type.center('test', 30) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: descriptor 'center' requires a 'unicode' object but received a 'str'

And as I said in the previous comment, the same thing applies for the other way round:

>>> pd.compat.binary_type.center(u'test', 30) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: descriptor 'center' requires a 'str' object but received a 'unicode'

okay forget it then :)

cpcloud · 2013-09-28T22:51:29Z

@alefnula can u add tests for the new justification options

alefnula · 2013-09-28T23:01:34Z

@cpcloud I changed the HTML tests so the None option is working... For the other, will something like this be enough?

In [2]: df = pd.DataFrame({'foo': [1, 2, 3], 'bar': [4, 5, 6], 'baz': [7, 8, 9]})
In [3]: print(df.to_string(justify='left'))
   bar  baz  foo
0  4    7    1
1  5    8    2
2  6    9    3
In [4]: print(df.to_string(justify='right'))
   bar  baz  foo
0    4    7    1
1    5    8    2
2    6    9    3
In [5]: print(df.to_string(justify='center'))
   bar  baz  foo
0   4    7    1
1   5    8    2
2   6    9    3

alefnula · 2013-09-29T01:38:25Z

Although, maybe align is a better parameter name than justify :-/

alefnula · 2013-09-29T02:25:57Z

There was some really weird code here... The alignment of the columns was done twice (sometimes even 3 times, which is ridiculous - recursive calls...) I really couldn't find the reason why so I removed that.

Also there is a bug in the current string column formatting. The column header is not aligned. The bug is also on the master branch. I'll try to fix it here if this patch can be accepted.

Example of the bug:

import numpy as np
import pandas as pd
ind = pd.date_range('2013-01-01', freq='H', periods=10)
df = pd.DataFrame(np.random.randn(10, 3), index=ind)
df = df.reset_index()
df['test'] = 'foo bar baz'
print(df.to_string(justify='left'))

Produces:

  index                0         1         2        test        
0 2013-01-01 00:00:00 -0.295496 -1.622709 -2.330210  foo bar baz
1 2013-01-01 01:00:00 -0.663977 -0.415800 -1.928071  foo bar baz
2 2013-01-01 02:00:00 -0.468839  0.106345  1.344761  foo bar baz
3 2013-01-01 03:00:00 -0.154564  0.146896 -0.322325  foo bar baz
4 2013-01-01 04:00:00  0.695505  0.900301  0.261876  foo bar baz
5 2013-01-01 05:00:00 -1.142544 -0.089581  2.180599  foo bar baz
6 2013-01-01 06:00:00 -0.806229 -0.766914 -0.852616  foo bar baz
7 2013-01-01 07:00:00  0.037000 -0.093492  1.208734  foo bar baz
8 2013-01-01 08:00:00  1.578699  0.067581  0.855523  foo bar baz
9 2013-01-01 09:00:00  1.707266  1.333138  1.185469  foo bar baz

You can see that the test column header is shifted...

Finally, maybe align is a better parameter name than justify :-/

cpcloud · 2013-09-29T03:07:15Z

@alefnula i would put the column alignment in a separate issue.

alefnula · 2013-09-29T10:33:46Z

@cpcloud The problem is that I don't even know if that's a bug or a feature :D That's why I asked.

jreback · 2013-10-02T21:10:34Z

@cpcloud ok to merge?

jreback · 2013-10-02T21:11:13Z

@alefnula relize no release notes...can you add?

alefnula · 2013-10-02T22:53:29Z

@jreback I don't think this should be merged yet. The justfy=None doesn't actually work on to_string because it will realise that it's None and turn it to right (the default), so maybe it should be justify='none' instead of justify=None? justify=None should probably just revert to whatever is the default... Probably right should be the default as it was before these changes.

jreback · 2013-10-02T22:58:19Z

you can just put justify = 'right' as the default which iirc was what it was

alefnula · 2013-10-02T23:04:45Z

That's fine with me :) I'll do it that way then. I should probably change the docs also. Just to add that the 'center' option is also available.

jreback · 2013-10-03T16:10:04Z

@cpcloud this looks ok....so when you are ready can merge (as I believe you are waiting to merge something else)

alefnula · 2013-10-03T20:38:48Z

@jreback I just rebased it, didn't changed anything yet. :) There is one more problem with this (I'm really sorry to be such a hassle :( but I just don't want to break anything), docs:

justify : {'left', 'right'}, default None
    Left or right-justify the column labels. If None uses the option from
    the print configuration (controlled by set_option), 'right' out
    of the box.

But now justify justifies the whole columns, not just the headers :( And also I cannot use the None and just default to right because if it's None it will read the:

if justify is None:
    self.justify = get_option("display.colheader_justify")

If it's ok to change the functionality to justify the whole columns I need a sentinel for no justification. And probably read get_option("display.column_justify") instead of display.colheader_justify.

display.colheader_justify is mentioned in the docs so it will be a backward incompatible change..

jreback · 2013-10-04T12:49:29Z

@cpcloud what do you think?
I think columns should be left alone and this just should apply to the header

alefnula · 2013-10-04T13:16:32Z

@jreback In that case this whole pull request should be discarded since it justifies the whole columns... I could maybe separate the column and header justification and use justify for header and align parameter for columns?

jreback · 2013-10-04T20:22:32Z

do you want to defer this to 0.14....as templates hopefully will solve these types of things?
@cpcloud ?

jreback · 2013-10-09T19:17:07Z

defering this to 0.14

…(GH4987)

…ting.

ghost · 2014-02-04T09:30:05Z

Stale, Please open an updated PR if the problems mentioned are sorted out.

jreback reviewed Sep 28, 2013
View reviewed changes

cpcloud reviewed Sep 28, 2013
View reviewed changes

jreback mentioned this pull request Sep 30, 2013

Test long string formatting in to_string and to_html #1852

Closed

alefnula added 2 commits November 16, 2013 22:34

ENH: Removing unnecessary whitespace when formatting to a HTML table …

dacea3e

…(GH4987)

ENH: Adding center and None justification options to DataFrame format…

93849a6

…ting.

ghost closed this Feb 4, 2014

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Removing unnecessary whitespace when formatting to a HTML table. #5012

ENH: Removing unnecessary whitespace when formatting to a HTML table. #5012

alefnula commented Sep 27, 2013

jreback Sep 28, 2013

alefnula Sep 28, 2013

alefnula Sep 28, 2013

jreback commented Sep 28, 2013

alefnula commented Sep 28, 2013

alefnula commented Sep 28, 2013

cpcloud Sep 28, 2013

alefnula Sep 28, 2013

cpcloud Sep 28, 2013

alefnula Sep 28, 2013

cpcloud Sep 28, 2013

cpcloud commented Sep 28, 2013

alefnula commented Sep 28, 2013

alefnula commented Sep 29, 2013

alefnula commented Sep 29, 2013

cpcloud commented Sep 29, 2013

alefnula commented Sep 29, 2013

jreback commented Oct 2, 2013

jreback commented Oct 2, 2013

alefnula commented Oct 2, 2013

jreback commented Oct 2, 2013

alefnula commented Oct 2, 2013

jreback commented Oct 3, 2013

alefnula commented Oct 3, 2013

jreback commented Oct 4, 2013

alefnula commented Oct 4, 2013

jreback commented Oct 4, 2013

jreback commented Oct 9, 2013

ghost commented Feb 4, 2014

ENH: Removing unnecessary whitespace when formatting to a HTML table. #5012

ENH: Removing unnecessary whitespace when formatting to a HTML table. #5012

Conversation

alefnula commented Sep 27, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Sep 28, 2013

alefnula commented Sep 28, 2013

alefnula commented Sep 28, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cpcloud commented Sep 28, 2013

alefnula commented Sep 28, 2013

alefnula commented Sep 29, 2013

alefnula commented Sep 29, 2013

cpcloud commented Sep 29, 2013

alefnula commented Sep 29, 2013

jreback commented Oct 2, 2013

jreback commented Oct 2, 2013

alefnula commented Oct 2, 2013

jreback commented Oct 2, 2013

alefnula commented Oct 2, 2013

jreback commented Oct 3, 2013

alefnula commented Oct 3, 2013

jreback commented Oct 4, 2013

alefnula commented Oct 4, 2013

jreback commented Oct 4, 2013

jreback commented Oct 9, 2013

ghost commented Feb 4, 2014