Skip to content

BUG: LatexFormatter.write_result multi-index #17499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.21.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,7 @@ I/O
- Bug in :func:`read_stata` where value labels could not be read when using an iterator (:issue:`16923`)
- Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
- Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
- Bug in :func:`to_latex` where repeated multi-index values were not printed even though a higher level index differed from the previous row (:issue:`14484`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might have to simplify this, but I'll see what @jreback thinks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move to 0.21.1


Plotting
^^^^^^^^
Expand Down
18 changes: 13 additions & 5 deletions pandas/io/formats/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@
import pandas as pd
import numpy as np

import itertools
import csv
from functools import partial

Expand Down Expand Up @@ -887,6 +886,7 @@ def get_col_type(dtype):
name = any(self.frame.index.names)
cname = any(self.frame.columns.names)
lastcol = self.frame.index.nlevels - 1
previous_lev3 = None
for i, lev in enumerate(self.frame.index.levels):
lev2 = lev.format()
blank = ' ' * len(lev2[0])
Expand All @@ -897,11 +897,19 @@ def get_col_type(dtype):
lev3 = [blank] * clevels
if name:
lev3.append(lev.name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be better to restructure this whole loop.

iterating like this:

for g, grp in table.groupby(level=0):
      # grp is a DataFrame

would be simpler to print out I think.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct, using df.groupby makes it simpler. Change is implemented and I will ping you once it is green.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had to revert my change. My attempt using df.groupby() changed the order of some elements and it didn't format more complex indicies like dates correctly.

The current approach does not have any of these issues, so I think it is the best I can do right now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, neither of these should be true, come back when you can

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback The change in the order of some elements is caused by a bug in df.groupby() that I just reported. #17537

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok thanks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MXKY Hi, I've fixed your issue #17537.
Nothing stands in your way!

for level_idx, group in itertools.groupby(
self.frame.index.labels[i]):
count = len(list(group))
lev3.extend([lev2[level_idx]] + [blank] * (count - 1))
current_idx_val = None
for level_idx in self.frame.index.labels[i]:
if ((previous_lev3 is None or
previous_lev3[len(lev3)].isspace()) and
lev2[level_idx] == current_idx_val):
# same index as above row and left index was the same
lev3.append(blank)
else:
# different value than above or left index different
lev3.append(lev2[level_idx])
current_idx_val = lev2[level_idx]
strcols.insert(i, lev3)
previous_lev3 = lev3

column_format = self.column_format
if column_format is None:
Expand Down
22 changes: 22 additions & 0 deletions pandas/tests/io/formats/test_to_latex.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,28 @@ def test_to_latex_multiindex(self):
1 & 2.0 & 3.5 & 0.707107 & 3.0 & 3.25 & 3.5 & 3.75 & 4.0 \\
\bottomrule
\end{tabular}
"""

assert result == expected

# GH 14484
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add more comments about what you're testing here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make this a separate test

# If an index is repeated in subsequent rows, it should be
# replaced with a blank in the created table.
# This should ONLY happen if all higher order indicies
# (to the left) are equal, too.
# In this test, 'c' has to be printed both times, because
# the higher order index 'A' != 'B'
df = pd.DataFrame(index=pd.MultiIndex.from_tuples(
[('A', 'c'), ('B', 'c')]), columns=['col'])
result = df.to_latex()
expected = r"""\begin{tabular}{lll}
\toprule
& & col \\
\midrule
A & c & NaN \\
B & c & NaN \\
\bottomrule
\end{tabular}
"""

assert result == expected
Expand Down