Skip to content

Fixed Metadata Propogation in DataFrame #37381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 4, 2020
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -558,7 +558,7 @@ Other
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` incorrectly raising ``AssertionError`` instead of ``ValueError`` when invalid parameter combinations are passed (:issue:`36045`)
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` with numeric values and string ``to_replace`` (:issue:`34789`)
- Fixed bug in metadata propagation incorrectly copying DataFrame columns as metadata when the column name overlaps with the metadata name (:issue:`37037`)
- Fixed metadata propagation in the :class:`Series.dt` and :class:`Series.str` accessors and :class:`DataFrame.duplicated` and :class:`DataFrame.stack` and :class:`DataFrame.unstack` and :class:`DataFrame.pivot` methods (:issue:`28283`)
- Fixed metadata propagation in the :class:`Series.dt`, :class:`Series.str` accessors, :class:`DataFrame.duplicated`, :class:`DataFrame.stack`, :class:`DataFrame.unstack`, :class:`DataFrame.pivot`, :class:`DataFrame.append`, :class:`DataFrame.diff`, :class:`DataFrame.applymap` and :class:`DataFrame.update` methods (:issue:`28283`) (:issue:`37381`)
- Bug in :meth:`Index.union` behaving differently depending on whether operand is a :class:`Index` or other list-like (:issue:`36384`)
- Passing an array with 2 or more dimensions to the :class:`Series` constructor now raises the more specific ``ValueError``, from a bare ``Exception`` previously (:issue:`35744`)
- Bug in ``accessor.DirNamesMixin``, where ``dir(obj)`` wouldn't show attributes defined on the instance (:issue:`37173`).
Expand Down
18 changes: 10 additions & 8 deletions pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -7401,7 +7401,7 @@ def diff(self, periods: int = 1, axis: Axis = 0) -> DataFrame:
return self - self.shift(periods, axis=axis)

new_data = self._mgr.diff(n=periods, axis=bm_axis)
return self._constructor(new_data)
return self._constructor(new_data).__finalize__(self, "diff")

# ----------------------------------------------------------------------
# Function application
Expand Down Expand Up @@ -7780,7 +7780,7 @@ def infer(x):
return lib.map_infer(x, func, ignore_na=ignore_na)
return lib.map_infer(x.astype(object)._values, func, ignore_na=ignore_na)

return self.apply(infer)
return self.apply(infer).__finalize__(self, "applymap")

# ----------------------------------------------------------------------
# Merging / joining methods
Expand Down Expand Up @@ -7917,12 +7917,14 @@ def append(
to_concat = [self, *other]
else:
to_concat = [self, other]
return concat(
to_concat,
ignore_index=ignore_index,
verify_integrity=verify_integrity,
sort=sort,
)
return (
concat(
to_concat,
ignore_index=ignore_index,
verify_integrity=verify_integrity,
sort=sort,
)
).__finalize__(self, method="append")

def join(
self, other, on=None, how="left", lsuffix="", rsuffix="", sort=False
Expand Down
4 changes: 2 additions & 2 deletions pandas/core/reshape/merge.py
Original file line number Diff line number Diff line change
Expand Up @@ -689,7 +689,7 @@ def get_result(self):

self._maybe_restore_index_levels(result)

return result
return result.__finalize__(self, method="merge")

def _indicator_pre_merge(
self, left: "DataFrame", right: "DataFrame"
Expand Down Expand Up @@ -1505,7 +1505,7 @@ def get_result(self):
)

typ = self.left._constructor
result = typ(result_data).__finalize__(self, method=self._merge_type)
result = typ(result_data)

self._maybe_add_join_keys(result, left_indexer, right_indexer)

Expand Down
11 changes: 2 additions & 9 deletions pandas/tests/generic/test_finalize.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,28 +178,21 @@
marks=not_implemented_mark,
),
pytest.param(
(pd.DataFrame, frame_data, operator.methodcaller("diff")),
marks=not_implemented_mark,
),
pytest.param(
(pd.DataFrame, frame_data, operator.methodcaller("applymap", lambda x: x)),
marks=not_implemented_mark,
(pd.DataFrame, frame_data, operator.methodcaller("applymap", lambda x: x))
),
pytest.param(
(
pd.DataFrame,
frame_data,
operator.methodcaller("append", pd.DataFrame({"A": [1]})),
),
marks=not_implemented_mark,
)
),
pytest.param(
(
pd.DataFrame,
frame_data,
operator.methodcaller("append", pd.DataFrame({"B": [1]})),
),
marks=not_implemented_mark,
),
pytest.param(
(
Expand Down