Skip to content

BUG: DataFrame.melt gives unexpected result when column "value" already exists #35003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Jul 2, 2020
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5a84593
Merge pull request #1 from pandas-dev/master
pizzathief Jun 29, 2018
c00363e
Merge remote-tracking branch 'upstream/master'
pizzathief Jun 24, 2020
d243026
warn if value column name already exists
pizzathief Jun 26, 2020
968fc5b
enblackenated my patch
pizzathief Jun 26, 2020
12bdaf7
added entry in whatsnew file
pizzathief Jun 26, 2020
f20e90f
flake8ified my patch
pizzathief Jun 26, 2020
ff84c46
enacted isortification on patch
pizzathief Jun 26, 2020
7267b59
Merge remote-tracking branch 'upstream/master'
pizzathief Jun 26, 2020
98acc00
Merge remote-tracking branch 'upstream/master'
pizzathief Jun 27, 2020
4a52ccd
warn if value column name already exists
pizzathief Jun 26, 2020
e8821d3
enblackenated my patch
pizzathief Jun 26, 2020
c7dd8de
added entry in whatsnew file
pizzathief Jun 26, 2020
0dc55f3
flake8ified my patch
pizzathief Jun 26, 2020
4b8cbc0
enacted isortification on patch
pizzathief Jun 26, 2020
720713d
warn if value column name already exists
pizzathief Jun 26, 2020
f68e322
enblackenated my patch
pizzathief Jun 26, 2020
77003a9
flake8ified my patch
pizzathief Jun 26, 2020
c9efcee
enacted isortification on patch
pizzathief Jun 26, 2020
e120f23
replaced pytest.warns with assert_produces_warning as requested
pizzathief Jun 27, 2020
a9da61e
had to set stacklevel
pizzathief Jun 27, 2020
6a7ad2e
Merge branch 'issue34731-1' of https://github.com/pizzathief/pandas i…
pizzathief Jun 27, 2020
56d2acc
Gwqot a warning about not removing warnings after I removed
pizzathief Jun 27, 2020
9ff17c4
reenblackenatified file.
pizzathief Jun 27, 2020
2d55f27
requested changes
pizzathief Jun 30, 2020
d118a90
black says "no" to single quote strings.
pizzathief Jun 30, 2020
ffaa00c
made further requested changes.
pizzathief Jul 1, 2020
d2b9fdd
E231 formatting problems fixed
pizzathief Jul 1, 2020
b8d79f4
ok black, you can have the comma.
pizzathief Jul 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1105,6 +1105,7 @@ Reshaping
- Fixed bug in :func:`melt` where melting MultiIndex columns with ``col_level`` > 0 would raise a ``KeyError`` on ``id_vars`` (:issue:`34129`)
- Bug in :meth:`Series.where` with an empty Series and empty ``cond`` having non-bool dtype (:issue:`34592`)
- Fixed regression where :meth:`DataFrame.apply` would raise ``ValueError`` for elements whth ``S`` dtype (:issue:`34529`)
- Issue warning if value column name already exists when using :meth:`DataFrame.melt` (:issue:`34731`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this to the Deprecations section?


Sparse
^^^^^^
Expand Down
9 changes: 9 additions & 0 deletions pandas/core/reshape/melt.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import re
from typing import TYPE_CHECKING, List, cast
import warnings

import numpy as np

Expand Down Expand Up @@ -40,6 +41,14 @@ def melt(
else:
cols = list(frame.columns)

if value_name in frame.columns:
warnings.warn(
'The value column name "%s" conflicts with an existing '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use f-string syntax? We don't use the Py2 syntax any more

"column in the dataframe." % (value_name),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally we want to give users insight as to what they need to do before the deprecated behavior gets changed - can you try to clarify the message a bit?

DeprecationWarning,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change this to a FutureWarning?

stacklevel=3,
)

if id_vars is not None:
if not is_list_like(id_vars):
id_vars = [id_vars]
Expand Down
9 changes: 9 additions & 0 deletions pandas/tests/reshape/test_melt.py
Original file line number Diff line number Diff line change
Expand Up @@ -1014,3 +1014,12 @@ def test_col_substring_of_stubname(self):
)
result = pd.wide_to_long(wide_df, stubnames="PA", i=["node_id", "A"], j="time")
tm.assert_frame_equal(result, expected)

def test_warn_of_column_name_value(self):
# GH34731
# raise a warning if the resultant value column name matches
# a name in the dataframe already (default name is "value")
df = pd.DataFrame({"col": list("ABC"), "value": range(10, 16, 2)})

with tm.assert_produces_warning(DeprecationWarning):
dfm = df.melt(id_vars="value") # noqa F841
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you create an expected value and use assert_frame_equal

also instead of dfm, call this result.