Skip to content

ENH: Styler.apply_index and Styler.applymap_index for conditional formatting of column/index headers #41893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Aug 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
ece8855
add apply across index
attack68 Jun 9, 2021
a3a88e5
add applymap across index
attack68 Jun 9, 2021
066e4f3
improve docs
attack68 Jun 9, 2021
50dbcc7
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jun 11, 2021
ef5839f
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jun 12, 2021
5e4c1c0
add column header styling and amend tests
attack68 Jun 12, 2021
20ac7e0
doc sharing
attack68 Jun 13, 2021
26c5340
doc fix
attack68 Jun 13, 2021
2437b72
doc fix
attack68 Jun 13, 2021
312a6e6
collapse the cellstyle maps
attack68 Jun 13, 2021
553426f
add basic test
attack68 Jun 13, 2021
f01dfee
add basic test
attack68 Jun 13, 2021
f940165
parametrise tests
attack68 Jun 13, 2021
6f5b46c
test for raises ValueError
attack68 Jun 13, 2021
75cf6ca
test html working
attack68 Jun 13, 2021
d654139
test html working
attack68 Jun 13, 2021
728091a
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jun 14, 2021
17787ef
whats new 1.4.0
attack68 Jun 14, 2021
248de0d
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jun 20, 2021
321458e
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jun 29, 2021
ed58798
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jul 13, 2021
8d8e88f
update tests
attack68 Jul 14, 2021
e17d766
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Jul 30, 2021
f70bf65
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Aug 6, 2021
3dfe6d5
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Aug 9, 2021
81f3a3c
rename: applymap_header -->> applymap_index
attack68 Aug 9, 2021
e709777
skip doctests
attack68 Aug 9, 2021
1e6efea
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Aug 9, 2021
ab237b0
rename levels -->> level
attack68 Aug 9, 2021
9b7ec06
update user guide
attack68 Aug 9, 2021
1c7ef8b
Merge remote-tracking branch 'upstream/master' into styler_apply_inde…
attack68 Aug 11, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added doc/source/_static/style/appmaphead1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/_static/style/appmaphead2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions doc/source/reference/style.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ Style application

Styler.apply
Styler.applymap
Styler.apply_index
Styler.applymap_index
Styler.format
Styler.hide_index
Styler.hide_columns
Expand Down
46 changes: 45 additions & 1 deletion doc/source/user_guide/style.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -225,13 +225,15 @@
"\n",
"- Using [.set_table_styles()][table] to control broader areas of the table with specified internal CSS. Although table styles allow the flexibility to add CSS selectors and properties controlling all individual parts of the table, they are unwieldy for individual cell specifications. Also, note that table styles cannot be exported to Excel. \n",
"- Using [.set_td_classes()][td_class] to directly link either external CSS classes to your data cells or link the internal CSS classes created by [.set_table_styles()][table]. See [here](#Setting-Classes-and-Linking-to-External-CSS). These cannot be used on column header rows or indexes, and also won't export to Excel. \n",
"- Using the [.apply()][apply] and [.applymap()][applymap] functions to add direct internal CSS to specific data cells. See [here](#Styler-Functions). These cannot be used on column header rows or indexes, but only these methods add styles that will export to Excel. These methods work in a similar way to [DataFrame.apply()][dfapply] and [DataFrame.applymap()][dfapplymap].\n",
"- Using the [.apply()][apply] and [.applymap()][applymap] functions to add direct internal CSS to specific data cells. See [here](#Styler-Functions). As of v1.4.0 there are also methods that work directly on column header rows or indexes; [.apply_index()][applyindex] and [.applymap_index()][applymapindex]. Note that only these methods add styles that will export to Excel. These methods work in a similar way to [DataFrame.apply()][dfapply] and [DataFrame.applymap()][dfapplymap].\n",
"\n",
"[table]: ../reference/api/pandas.io.formats.style.Styler.set_table_styles.rst\n",
"[styler]: ../reference/api/pandas.io.formats.style.Styler.rst\n",
"[td_class]: ../reference/api/pandas.io.formats.style.Styler.set_td_classes.rst\n",
"[apply]: ../reference/api/pandas.io.formats.style.Styler.apply.rst\n",
"[applymap]: ../reference/api/pandas.io.formats.style.Styler.applymap.rst\n",
"[applyindex]: ../reference/api/pandas.io.formats.style.Styler.apply_index.rst\n",
"[applymapindex]: ../reference/api/pandas.io.formats.style.Styler.applymap_index.rst\n",
"[dfapply]: ../reference/api/pandas.DataFrame.apply.rst\n",
"[dfapplymap]: ../reference/api/pandas.DataFrame.applymap.rst"
]
Expand Down Expand Up @@ -432,6 +434,8 @@
"source": [
"## Styler Functions\n",
"\n",
"### Acting on Data\n",
"\n",
"We use the following methods to pass your style functions. Both of those methods take a function (and some other keyword arguments) and apply it to the DataFrame in a certain way, rendering CSS styles.\n",
"\n",
"- [.applymap()][applymap] (elementwise): accepts a function that takes a single value and returns a string with the CSS attribute-value pair.\n",
Expand Down Expand Up @@ -533,6 +537,18 @@
" .apply(highlight_max, props='color:white;background-color:purple', axis=None)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"nbsphinx": "hidden"
},
"outputs": [],
"source": [
"# Hidden cell to avoid CSS clashes and latter code upcoding previous formatting \n",
"s2.set_uuid('after_apply_again')"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -548,6 +564,33 @@
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Acting on the Index and Column Headers\n",
"\n",
"Similar application is acheived for headers by using:\n",
" \n",
"- [.applymap_index()][applymapindex] (elementwise): accepts a function that takes a single value and returns a string with the CSS attribute-value pair.\n",
"- [.apply_index()][applyindex] (level-wise): accepts a function that takes a Series and returns a Series, or numpy array with an identical shape where each element is a string with a CSS attribute-value pair. This method passes each level of your Index one-at-a-time. To style the index use `axis=0` and to style the column headers use `axis=1`.\n",
"\n",
"You can select a `level` of a `MultiIndex` but currently no similar `subset` application is available for these methods.\n",
"\n",
"[applyindex]: ../reference/api/pandas.io.formats.style.Styler.apply_index.rst\n",
"[applymapindex]: ../reference/api/pandas.io.formats.style.Styler.applymap_index.rst"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s2.applymap_index(lambda v: \"color:pink;\" if v>4 else \"color:darkblue;\", axis=0)\n",
"s2.apply_index(lambda s: np.where(s.isin([\"A\", \"B\"]), \"color:pink;\", \"color:darkblue;\"), axis=1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -1931,6 +1974,7 @@
}
],
"metadata": {
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.4.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Other enhancements
- :meth:`Series.ewm`, :meth:`DataFrame.ewm`, now support a ``method`` argument with a ``'table'`` option that performs the windowing operation over an entire :class:`DataFrame`. See :ref:`Window Overview <window.overview>` for performance and functional benefits (:issue:`42273`)
- Added ``sparse_index`` and ``sparse_columns`` keyword arguments to :meth:`.Styler.to_html` (:issue:`41946`)
- Added keyword argument ``environment`` to :meth:`.Styler.to_latex` also allowing a specific "longtable" entry with a separate jinja2 template (:issue:`41866`)
- :meth:`.Styler.apply_index` and :meth:`.Styler.applymap_index` added to allow conditional styling of index and column header values (:issue:`41893`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be worth creating a styler sub-section in other enhancements (followon)

- :meth:`.GroupBy.cummin` and :meth:`.GroupBy.cummax` now support the argument ``skipna`` (:issue:`34047`)
-

Expand Down
177 changes: 176 additions & 1 deletion pandas/io/formats/style.py
Original file line number Diff line number Diff line change
Expand Up @@ -1012,6 +1012,32 @@ def _update_ctx(self, attrs: DataFrame) -> None:
i, j = self.index.get_loc(rn), self.columns.get_loc(cn)
self.ctx[(i, j)].extend(css_list)

def _update_ctx_header(self, attrs: DataFrame, axis: str) -> None:
"""
Update the state of the ``Styler`` for header cells.

Collects a mapping of {index_label: [('<property>', '<value>'), ..]}.

Parameters
----------
attrs : Series
Should contain strings of '<property>: <value>;<prop2>: <val2>', and an
integer index.
Whitespace shouldn't matter and the final trailing ';' shouldn't
matter.
axis : str
Identifies whether the ctx object being updated is the index or columns
"""
for j in attrs.columns:
for i, c in attrs[[j]].itertuples():
if not c:
continue
css_list = maybe_convert_css_to_tuples(c)
if axis == "index":
self.ctx_index[(i, j)].extend(css_list)
else:
self.ctx_columns[(j, i)].extend(css_list)

def _copy(self, deepcopy: bool = False) -> Styler:
"""
Copies a Styler, allowing for deepcopy or shallow copy
Expand Down Expand Up @@ -1051,6 +1077,8 @@ def _copy(self, deepcopy: bool = False) -> Styler:
"hidden_rows",
"hidden_columns",
"ctx",
"ctx_index",
"ctx_columns",
"cell_context",
"_todo",
"table_styles",
Expand Down Expand Up @@ -1172,6 +1200,8 @@ def apply(

See Also
--------
Styler.applymap_index: Apply a CSS-styling function to headers elementwise.
Styler.apply_index: Apply a CSS-styling function to headers level-wise.
Styler.applymap: Apply a CSS-styling function elementwise.

Notes
Expand Down Expand Up @@ -1215,6 +1245,149 @@ def apply(
)
return self

def _apply_index(
self,
func: Callable[..., Styler],
axis: int | str = 0,
level: Level | list[Level] | None = None,
method: str = "apply",
**kwargs,
) -> Styler:
if axis in [0, "index"]:
obj, axis = self.index, "index"
elif axis in [1, "columns"]:
obj, axis = self.columns, "columns"
else:
raise ValueError(
f"`axis` must be one of 0, 1, 'index', 'columns', got {axis}"
)

levels_ = _refactor_levels(level, obj)
data = DataFrame(obj.to_list()).loc[:, levels_]

if method == "apply":
result = data.apply(func, axis=0, **kwargs)
elif method == "applymap":
result = data.applymap(func, **kwargs)

self._update_ctx_header(result, axis)
return self

@doc(
this="apply",
wise="level-wise",
alt="applymap",
altwise="elementwise",
func="take a Series and return a string array of the same length",
axis='{0, 1, "index", "columns"}',
input_note="the index as a Series, if an Index, or a level of a MultiIndex",
output_note="an identically sized array of CSS styles as strings",
var="s",
ret='np.where(s == "B", "background-color: yellow;", "")',
ret2='["background-color: yellow;" if "x" in v else "" for v in s]',
)
def apply_index(
self,
func: Callable[..., Styler],
axis: int | str = 0,
level: Level | list[Level] | None = None,
**kwargs,
) -> Styler:
"""
Apply a CSS-styling function to the index or column headers, {wise}.

Updates the HTML representation with the result.

.. versionadded:: 1.4.0

Parameters
----------
func : function
``func`` should {func}.
axis : {axis}
The headers over which to apply the function.
level : int, str, list, optional
If index is MultiIndex the level(s) over which to apply the function.
**kwargs : dict
Pass along to ``func``.

Returns
-------
self : Styler

See Also
--------
Styler.{alt}_index: Apply a CSS-styling function to headers {altwise}.
Styler.apply: Apply a CSS-styling function column-wise, row-wise, or table-wise.
Styler.applymap: Apply a CSS-styling function elementwise.

Notes
-----
Each input to ``func`` will be {input_note}. The output of ``func`` should be
{output_note}, in the format 'attribute: value; attribute2: value2; ...'
or, if nothing is to be applied to that element, an empty string or ``None``.

Examples
--------
Basic usage to conditionally highlight values in the index.

>>> df = pd.DataFrame([[1,2], [3,4]], index=["A", "B"])
>>> def color_b(s):
... return {ret}
>>> df.style.{this}_index(color_b) # doctest: +SKIP

.. figure:: ../../_static/style/appmaphead1.png

Selectively applying to specific levels of MultiIndex columns.

>>> midx = pd.MultiIndex.from_product([['ix', 'jy'], [0, 1], ['x3', 'z4']])
>>> df = pd.DataFrame([np.arange(8)], columns=midx)
>>> def highlight_x({var}):
... return {ret2}
>>> df.style.{this}_index(highlight_x, axis="columns", level=[0, 2])
... # doctest: +SKIP

.. figure:: ../../_static/style/appmaphead2.png
"""
self._todo.append(
(
lambda instance: getattr(instance, "_apply_index"),
(func, axis, level, "apply"),
kwargs,
)
)
return self

@doc(
apply_index,
this="applymap",
wise="elementwise",
alt="apply",
altwise="level-wise",
func="take a scalar and return a string",
axis='{0, 1, "index", "columns"}',
input_note="an index value, if an Index, or a level value of a MultiIndex",
output_note="CSS styles as a string",
var="v",
ret='"background-color: yellow;" if v == "B" else None',
ret2='"background-color: yellow;" if "x" in v else None',
)
def applymap_index(
self,
func: Callable[..., Styler],
axis: int | str = 0,
level: Level | list[Level] | None = None,
**kwargs,
) -> Styler:
self._todo.append(
(
lambda instance: getattr(instance, "_apply_index"),
(func, axis, level, "applymap"),
kwargs,
)
)
return self

def _applymap(
self, func: Callable, subset: Subset | None = None, **kwargs
) -> Styler:
Expand All @@ -1237,7 +1410,7 @@ def applymap(
Parameters
----------
func : function
``func`` should take a scalar and return a scalar.
``func`` should take a scalar and return a string.
subset : label, array-like, IndexSlice, optional
A valid 2d input to `DataFrame.loc[<subset>]`, or, in the case of a 1d input
or single key, to `DataFrame.loc[:, <subset>]` where the columns are
Expand All @@ -1251,6 +1424,8 @@ def applymap(

See Also
--------
Styler.applymap_index: Apply a CSS-styling function to headers elementwise.
Styler.apply_index: Apply a CSS-styling function to headers level-wise.
Styler.apply: Apply a CSS-styling function column-wise, row-wise, or table-wise.

Notes
Expand Down
Loading