Skip to content

REGR: object column repr not respecting float format #40850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 11, 2021
Merged
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v1.2.4.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Fixed regressions
- Fixed regression in (in)equality comparison of ``pd.NaT`` with a non-datetimelike numpy array returning a scalar instead of an array (:issue:`40722`)
- Fixed regression in :meth:`DataFrame.where` not returning a copy in the case of an all True condition (:issue:`39595`)
- Fixed regression in :meth:`DataFrame.replace` raising ``IndexError`` when ``regex`` was a multi-key dictionary (:issue:`39338`)
-
- Fixed regression in repr of floats in an ``object`` column not respecting ``float_format`` (:issue:`40024`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this only for to_html? or more generally?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess also console repr (so mention these 2). doesn't affect, csv for example, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep it affects to_html and console repr.

In theory we might want this to affect to_csv, but this doesn't change behavior right now because formatting is handled based on column dtype, such that float_format doesn't get used for object dtype columns even if some values in that column are floats.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also affects to_latex, but not to_json

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok pls list those and add a test for to_latex if you can

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have updated whatsnew and added to_latex test


.. ---------------------------------------------------------------------------

Expand Down
6 changes: 4 additions & 2 deletions pandas/io/formats/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -1275,7 +1275,9 @@ def _format_strings(self) -> list[str]:
float_format = get_option("display.float_format")
if float_format is None:
precision = get_option("display.precision")
float_format = lambda x: f"{x: .{precision:d}f}"
float_format = lambda x: _trim_zeros_single_float(
f"{x: .{precision:d}f}"
)
else:
float_format = self.float_format

Expand Down Expand Up @@ -1331,7 +1333,7 @@ def _format(x):
if not is_float_type[i] and leading_space:
fmt_values.append(f" {_format(v)}")
elif is_float_type[i]:
fmt_values.append(_trim_zeros_single_float(float_format(v)))
fmt_values.append(float_format(v))
else:
if leading_space is False:
# False specifically, so that the default is
Expand Down
15 changes: 15 additions & 0 deletions pandas/tests/io/formats/test_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -2025,6 +2025,21 @@ def test_repr_str_float_truncation(self, data, expected):
result = repr(series)
assert result == expected

@pytest.mark.parametrize(
"float_format,expected",
[
("{:,.0f}".format, "0 1,000\n1 test\ndtype: object"),
("{:.4f}".format, "0 1000.0000\n1 test\ndtype: object"),
],
)
def test_repr_float_format_in_object_col(self, float_format, expected):
# GH#40024
df = Series([1000.0, "test"])
with option_context("display.float_format", float_format):
result = repr(df)

assert result == expected

def test_dict_entries(self):
df = DataFrame({"A": [{"a": 1, "b": 2}]})

Expand Down
26 changes: 26 additions & 0 deletions pandas/tests/io/formats/test_to_html.py
Original file line number Diff line number Diff line change
Expand Up @@ -883,3 +883,29 @@ def test_to_html_na_rep_and_float_format(na_rep):
</tbody>
</table>"""
assert result == expected


def test_to_html_float_format_object_col():
# GH#40024
df = DataFrame(data={"x": [1000.0, "test"]})
result = df.to_html(float_format=lambda x: f"{x:,.0f}")
expected = """<table border="1" class="dataframe">
<thead>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the expected html should be in a file, but I see #36690 slipped through #36690 (comment) so OK to leave as follow-up

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, thanks that's good to know. Will make a follow-up fixing that

<tr style="text-align: right;">
<th></th>
<th>x</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>1,000</td>
</tr>
<tr>
<th>1</th>
<td>test</td>
</tr>
</tbody>
</table>"""

assert result == expected