BUG: LatexFormatter.write_result multi-index #17499

MaximilianKoestler · 2017-09-12T12:41:51Z

Fixed GH issue 14484:

LatexFormatter.write_result(...) now does not print blanks if a
higher-order index differs from the previous row.
Also added testcase for this.

closes BUG: to_latex outputs string with missing second index level values #14484
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

codecov · 2017-09-12T13:14:32Z

Codecov Report

Merging #17499 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #17499      +/-   ##
==========================================
- Coverage   91.18%   91.17%   -0.02%     
==========================================
  Files         163      163              
  Lines       49543    49547       +4     
==========================================
- Hits        45177    45172       -5     
- Misses       4366     4375       +9

Flag	Coverage Δ
#multiple	`88.95% <100%> (ø)`	⬆️
#single	`40.21% <0%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/formats/format.py	`96.06% <100%> (+0.01%)`	⬆️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.77% <0%> (-0.1%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f11bbf2...6eadb87. Read the comment docs.

gfyoung · 2017-09-13T08:22:40Z

doc/source/whatsnew/v0.21.0.txt

@@ -415,6 +415,7 @@ I/O
 - Bug in :func:`read_stata` where value labels could not be read when using an iterator (:issue:`16923`)
 - Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
 - Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
+- Bug in ``LatexFormatter.write_result`` where repeated multi-index values were not printed even though a higher level index differed from the previous row (:issue:`14484`)


Reference the function that users will care about, which is "to_latex" instead of "LatexFormatter.write_result" . Also, follow the convention used in the two lines above when referencing "to_latex" with the func notation.

gfyoung · 2017-09-13T08:23:38Z

pandas/tests/io/formats/test_to_latex.py

@@ -221,6 +221,22 @@ def test_to_latex_multiindex(self):

        assert result == expected

+        # GH 14484


Add more comments about what you're testing here.

MaximilianKoestler · 2017-09-13T09:17:47Z

@gfyoung Thanks for the comments, it is fixed and green again

gfyoung · 2017-09-13T09:19:06Z

doc/source/whatsnew/v0.21.0.txt

@@ -415,6 +415,7 @@ I/O
 - Bug in :func:`read_stata` where value labels could not be read when using an iterator (:issue:`16923`)
 - Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
 - Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
+- Bug in :func:`to_latex` where repeated multi-index values were not printed even though a higher level index differed from the previous row (:issue:`14484`)


I think we might have to simplify this, but I'll see what @jreback thinks.

gfyoung

Besides my comment about the whatsnew (wait until @jreback comments), this looks good to go.

jreback · 2017-09-13T10:31:48Z

pandas/io/formats/format.py

@@ -897,11 +897,19 @@ def get_col_type(dtype):
                    lev3 = [blank] * clevels
                if name:
                    lev3.append(lev.name)


I think it might be better to restructure this whole loop.

iterating like this:

for g, grp in table.groupby(level=0): # grp is a DataFrame

would be simpler to print out I think.

You are correct, using df.groupby makes it simpler. Change is implemented and I will ping you once it is green.

I just had to revert my change. My attempt using df.groupby() changed the order of some elements and it didn't format more complex indicies like dates correctly.

The current approach does not have any of these issues, so I think it is the best I can do right now.

ok, neither of these should be true, come back when you can

@jreback The change in the order of some elements is caused by a bug in df.groupby() that I just reported. #17537

@MXKY Hi, I've fixed your issue #17537.
Nothing stands in your way!

Fixed GH issue 14484: `LatexFormatter.write_result`` now does not print blanks if a higher-order index differs from the previous row. Also added testcase for this.

jreback · 2017-11-12T19:16:38Z

doc/source/whatsnew/v0.21.0.txt

@@ -415,6 +415,7 @@ I/O
 - Bug in :func:`read_stata` where value labels could not be read when using an iterator (:issue:`16923`)
 - Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
 - Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
+- Bug in :func:`to_latex` where repeated multi-index values were not printed even though a higher level index differed from the previous row (:issue:`14484`)


can you move to 0.21.1

jreback · 2017-11-12T19:22:49Z

pandas/tests/io/formats/test_to_latex.py

@@ -221,6 +221,28 @@ def test_to_latex_multiindex(self):

        assert result == expected

+        # GH 14484


can you make this a separate test

jreback · 2017-12-08T02:23:08Z

can you update

gfyoung · 2017-12-08T02:29:56Z

@jreback : I can carry this one to the finish line.

jreback · 2017-12-08T02:38:13Z

great!

feel free to pick up any PR that is not responsive from the author and can be finished

Closes pandas-devgh-14484 Closes pandas-devgh-17499

gfyoung · 2017-12-08T05:20:12Z

@jreback : Could not push to the remote (permissions), so I opened #18685 .

* BUG: LatexFormatter.write_result multi-index Fixed GH issue 14484: `LatexFormatter.write_result`` now does not print blanks if a higher-order index differs from the previous row. Also added testcase for this. * MAINT: Address reviewer comments Closes gh-14484 Closes gh-17499

* BUG: LatexFormatter.write_result multi-index Fixed GH issue 14484: `LatexFormatter.write_result`` now does not print blanks if a higher-order index differs from the previous row. Also added testcase for this. * MAINT: Address reviewer comments Closes pandas-devgh-14484 Closes pandas-devgh-17499

* BUG: LatexFormatter.write_result multi-index Fixed GH issue 14484: `LatexFormatter.write_result`` now does not print blanks if a higher-order index differs from the previous row. Also added testcase for this. * MAINT: Address reviewer comments Closes gh-14484 Closes gh-17499

MaximilianKoestler force-pushed the latex_formatter_patch branch from 04c2f82 to cb6aa41 Compare September 12, 2017 13:14

MaximilianKoestler force-pushed the latex_formatter_patch branch from cb6aa41 to f05f3b9 Compare September 13, 2017 06:55

gfyoung added IO LaTeX to_latex Bug MultiIndex labels Sep 13, 2017

gfyoung reviewed Sep 13, 2017

View reviewed changes

MaximilianKoestler force-pushed the latex_formatter_patch branch from f05f3b9 to 586adf8 Compare September 13, 2017 08:41

gfyoung reviewed Sep 13, 2017

View reviewed changes

gfyoung approved these changes Sep 13, 2017

View reviewed changes

jreback requested changes Sep 13, 2017

View reviewed changes

MaximilianKoestler force-pushed the latex_formatter_patch branch 3 times, most recently from e21d2f3 to 1291ead Compare September 13, 2017 13:30

BUG: LatexFormatter.write_result multi-index

6eadb87

Fixed GH issue 14484: `LatexFormatter.write_result`` now does not print blanks if a higher-order index differs from the previous row. Also added testcase for this.

MaximilianKoestler force-pushed the latex_formatter_patch branch from 1291ead to 6eadb87 Compare September 13, 2017 13:31

jreback requested changes Nov 12, 2017

View reviewed changes

gfyoung self-assigned this Dec 8, 2017

gfyoung added a commit to forking-repos/pandas that referenced this pull request Dec 8, 2017

MAINT: Address reviewer comments

3d8275e

Closes pandas-devgh-14484 Closes pandas-devgh-17499

gfyoung added this to the No action milestone Dec 8, 2017

gfyoung mentioned this pull request Dec 8, 2017

BUG: LatexFormatter.write_result multi-index #18685

Merged

gfyoung removed their assignment Dec 8, 2017

jreback closed this Dec 8, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: LatexFormatter.write_result multi-index #17499

BUG: LatexFormatter.write_result multi-index #17499

MaximilianKoestler commented Sep 12, 2017 •

edited

Loading

codecov bot commented Sep 12, 2017 •

edited

Loading

gfyoung Sep 13, 2017

gfyoung Sep 13, 2017

MaximilianKoestler commented Sep 13, 2017 •

edited

Loading

gfyoung Sep 13, 2017

gfyoung left a comment

jreback Sep 13, 2017

MaximilianKoestler Sep 13, 2017

MaximilianKoestler Sep 13, 2017

jreback Sep 14, 2017

MaximilianKoestler Sep 15, 2017

jreback Sep 15, 2017

Licht-T Oct 24, 2017

jreback Nov 12, 2017

jreback Nov 12, 2017

jreback commented Dec 8, 2017

gfyoung commented Dec 8, 2017

jreback commented Dec 8, 2017 •

edited by gfyoung

Loading

gfyoung commented Dec 8, 2017

		@@ -221,6 +221,22 @@ def test_to_latex_multiindex(self):

		assert result == expected

		# GH 14484

		@@ -221,6 +221,28 @@ def test_to_latex_multiindex(self):

		assert result == expected

		# GH 14484

BUG: LatexFormatter.write_result multi-index #17499

BUG: LatexFormatter.write_result multi-index #17499

Conversation

MaximilianKoestler commented Sep 12, 2017 • edited Loading

codecov bot commented Sep 12, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaximilianKoestler commented Sep 13, 2017 • edited Loading

Choose a reason for hiding this comment

gfyoung left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Dec 8, 2017

gfyoung commented Dec 8, 2017

jreback commented Dec 8, 2017 • edited by gfyoung Loading

gfyoung commented Dec 8, 2017

MaximilianKoestler commented Sep 12, 2017 •

edited

Loading

codecov bot commented Sep 12, 2017 •

edited

Loading

MaximilianKoestler commented Sep 13, 2017 •

edited

Loading

jreback commented Dec 8, 2017 •

edited by gfyoung

Loading