Skip to content

ENH: Styler.background_gradient to accept vmin vmax and dtype Int64 #29245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v1.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ Other enhancements
- :meth:`read_stata` can read Stata 119 dta files. (:issue:`28250`)
- Added ``encoding`` argument to :meth:`DataFrame.to_string` for non-ascii text (:issue:`28766`)
- Added ``encoding`` argument to :func:`DataFrame.to_html` for non-ascii text (:issue:`28663`)
- :meth:`Styler.background_gradient` now accepts ``vmin`` and ``vmax`` arguments (:issue:`12145`)

Build Changes
^^^^^^^^^^^^^
Expand Down Expand Up @@ -385,6 +386,7 @@ I/O
- Bug in :meth:`DataFrame.read_excel` with ``engine='ods'`` when ``sheet_name`` argument references a non-existent sheet (:issue:`27676`)
- Bug in :meth:`pandas.io.formats.style.Styler` formatting for floating values not displaying decimals correctly (:issue:`13257`)
- Bug in :meth:`DataFrame.to_html` when using ``formatters=<list>`` and ``max_cols`` together. (:issue:`25955`)
- Bug in :meth:`Styler.background_gradient` not able to work with dtype ``Int64`` (:issue:`28869`)

Plotting
^^^^^^^^
Expand Down
32 changes: 27 additions & 5 deletions pandas/io/formats/style.py
Original file line number Diff line number Diff line change
Expand Up @@ -958,6 +958,8 @@ def background_gradient(
axis=0,
subset=None,
text_color_threshold=0.408,
vmin=None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you annotate the new parameters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your review @WillAyd, but I don't quite get it what do you mean by "annotate", can you elaborate a little more? (I have docstring already so I suppose you don't mean docstring right?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got it! you mean the type-hints annotation right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that is correct

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done! 😄

vmax=None,
):
"""
Color the background in a gradient according to
Expand Down Expand Up @@ -986,6 +988,18 @@ def background_gradient(

.. versionadded:: 0.24.0

vmin : float, optional
Minimum data value that corresponds to colormap minimum value.
When None (default): the minimum value of the data will be used.

.. versionadded:: 1.0.0

vmax : float, optional
Maximum data value that corresponds to colormap maximum value.
When None (default): the maximum value of the data will be used.

.. versionadded:: 1.0.0

Returns
-------
self : Styler
Expand All @@ -1012,11 +1026,15 @@ def background_gradient(
low=low,
high=high,
text_color_threshold=text_color_threshold,
vmin=vmin,
vmax=vmax,
)
return self

@staticmethod
def _background_gradient(s, cmap="PuBu", low=0, high=0, text_color_threshold=0.408):
def _background_gradient(
s, cmap="PuBu", low=0, high=0, text_color_threshold=0.408, vmin=None, vmax=None
):
"""
Color background in a range according to the data.
"""
Expand All @@ -1028,14 +1046,18 @@ def _background_gradient(s, cmap="PuBu", low=0, high=0, text_color_threshold=0.4
raise ValueError(msg)

with _mpl(Styler.background_gradient) as (plt, colors):
smin = s.values.min()
smax = s.values.max()
smin = s.min() if vmin is None else vmin
if isinstance(smin, ABCSeries):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not already covered by the line directly preceding it?

Copy link
Contributor Author

@immaxchen immaxchen Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here depends on the axis argument provided to Styler.background_gradient, s can be a DataFrame or a Series. if s is a DataFrame and vmin not provided, smin will first become a Series then become the min value of the full data, the code here is consistent with (aka. direct copy from) Styler.bar method, do we want to change it?

Copy link
Contributor Author

@immaxchen immaxchen Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side note: try to use s.values.min() to get overall minima will produce AttributeError: 'IntegerArray' object has no attribute 'min' as described in #28869. OTOH, use np.min(s.values) will also produce NotImplementedError: The 'reduce' method is not supported. I thought IntegerArray might be a WIP and .min().min() is the workaround.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this into a helper function? I think this affects highlight_extrema as well but seems to tackle slightly differently; should keep unified

squeeze might also be more appropriate to use here

Copy link
Contributor Author

@immaxchen immaxchen Nov 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! didn't notice the highlight_extrema, they really should be consistent. turned-out a fairly simple approach can be used: np.nanmin(s.to_numpy()), s can be any shape

smin = smin.min()
smax = s.max() if vmax is None else vmax
if isinstance(smax, ABCSeries):
smax = smax.max()
rng = smax - smin
# extend lower / upper bounds, compresses color range
norm = colors.Normalize(smin - (rng * low), smax + (rng * high))
# matplotlib colors.Normalize modifies inplace?
# https://github.com/matplotlib/matplotlib/issues/5427
rgbas = plt.cm.get_cmap(cmap)(norm(s.values))
rgbas = plt.cm.get_cmap(cmap)(norm(s.to_numpy(dtype=float)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK with this but can just use default constructor without dtype argument I think? As it it seems like this is trying to coerce, which I'm not sure why that would be needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately, .to_numpy() alone will produce dtype=object and rejected by matplotlib.
OTOH, it will immediately convert to float anyway since cmap require normalizing to [0,1], nothing to loose for coercing in the first place.


def relative_luminance(rgba):
"""
Expand Down Expand Up @@ -1121,7 +1143,7 @@ def _bar(s, align, colors, width=100, vmin=None, vmax=None):
smax = max(abs(smin), abs(smax))
smin = -smax
# Transform to percent-range of linear-gradient
normed = width * (s.values - smin) / (smax - smin + 1e-12)
normed = width * (s.to_numpy(dtype=float) - smin) / (smax - smin + 1e-12)
zero = -width * smin / (smax - smin + 1e-12)

def css_bar(start, end, color):
Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/io/formats/test_style.py
Original file line number Diff line number Diff line change
Expand Up @@ -1648,6 +1648,23 @@ def test_background_gradient_axis(self):
assert result[(1, 0)] == mid
assert result[(1, 1)] == high

def test_background_gradient_vmin_vmax(self):
# GH 12145
df = pd.DataFrame(range(5))
ctx = df.style.background_gradient(vmin=1, vmax=3)._compute().ctx
assert ctx[(0, 0)] == ctx[(1, 0)]
assert ctx[(4, 0)] == ctx[(3, 0)]

def test_background_gradient_int64(self):
# GH 28869
df1 = pd.Series(range(3)).to_frame()
df2 = pd.Series(range(3), dtype="Int64").to_frame()
ctx1 = df1.style.background_gradient()._compute().ctx
ctx2 = df2.style.background_gradient()._compute().ctx
assert ctx2[(0, 0)] == ctx1[(0, 0)]
assert ctx2[(1, 0)] == ctx1[(1, 0)]
assert ctx2[(2, 0)] == ctx1[(2, 0)]


def test_block_names():
# catch accidental removal of a block
Expand Down