Skip to content

BUG: drop_duplicates raises FutureWarning after resetting index #49657

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
tobiscode opened this issue Nov 11, 2022 · 4 comments
Closed
2 of 3 tasks

BUG: drop_duplicates raises FutureWarning after resetting index #49657

tobiscode opened this issue Nov 11, 2022 · 4 comments
Labels
Closing Candidate May be closeable, needs more eyeballs Warnings Warnings that appear or should be added to pandas

Comments

@tobiscode
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
df = pd.DataFrame(data={0: [0, 0, 3], 1: [1, 1, 0], "time": [0, 1, 2]})
df.set_index("time", inplace=True)
df.drop_duplicates(keep="last")

Issue Description

I get the following warning:

/home/tkoehne/apps/miniconda3/envs/testpd/lib/python3.10/site-packages/pandas/core/frame.py:6257: FutureWarning: In a future version, the Index constructor will not infer numeric dtypes when passed object-dtype sequences (matching Series behavior)
  diff = Index(subset).difference(self.columns)

Expected Behavior

Internally, the changing of the index and subsequent checks for dropping shouldn't raise a FutureWarning, but rather adopt the future behavior by default.

Installed Versions

/home/tkoehne/apps/miniconda3/envs/testpd/lib/python3.10/site-packages/_distutils_hack/init.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")

INSTALLED VERSIONS

commit : ca60aab
python : 3.10.6.final.0
python-bits : 64
OS : Linux
OS-release : 4.18.0-348.2.1.el8_5.x86_64
Version : #1 SMP Mon Nov 8 13:30:15 EST 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.4.4
numpy : 1.23.3
pytz : 2022.1
dateutil : 2.8.2
setuptools : 65.5.0
pip : 22.2.2
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : 1.3.5
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
markupsafe : None
matplotlib : None
numba : None
numexpr : 2.8.3
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None

@tobiscode tobiscode added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Nov 11, 2022
@dannyi96
Copy link
Contributor

dannyi96 commented Nov 13, 2022

Have confirmed that this issue exists in latest released version 1.5 but is fixed in main ( for next 2.0 release ) via changes made in PR #49458 (in file pandas/core/indexes/base.py ).

[ CC: @mroeschke ]
would this be a change which would need to be backported to 1.5 as well ? or its fine for fix to be in next 2.0 release ?

@mroeschke
Copy link
Member

Looks like #49458 fixed this because FutureWarning was removed to enforce the deprecation; therefore, it can't be backported to 1.5.2 since it would be a breaking change.

Overall this will be fixed in 2.0. A backport PR would be accepted, but it needs to be based on the 1.5.x branch.

@phofl
Copy link
Member

phofl commented Nov 14, 2022

I can see this on 1.4.4 but neither on 1.5.0 nor on 1.5.1,

@phofl phofl added Warnings Warnings that appear or should be added to pandas Closing Candidate May be closeable, needs more eyeballs and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Nov 14, 2022
@dannyi96
Copy link
Contributor

@phofl yes, you're right ( my bad, the issue is present on older versions 1.4.x & is handled in 1.5.x )
I believe we can close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closing Candidate May be closeable, needs more eyeballs Warnings Warnings that appear or should be added to pandas
Projects
None yet
Development

No branches or pull requests

4 participants