Skip to content

BUG: Using .loc raises SettingWithCopyWarning #41309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
ad12 opened this issue May 4, 2021 · 3 comments
Closed
2 of 3 tasks

BUG: Using .loc raises SettingWithCopyWarning #41309

ad12 opened this issue May 4, 2021 · 3 comments
Labels
Closing Candidate May be closeable, needs more eyeballs

Comments

@ad12
Copy link

ad12 commented May 4, 2021

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

# Your code here
res = pd.DataFrame({"Method": ["Alpha1", "Beta1", "Beta2", "Alpha2", "Alpha3"], "Value": [5, 6, 7, 8, 10]})
df = res[res["Value"] <= 8]
df.loc[df["Method"].str.contains("Alpha"), "Method"] = "Gamma"

display(res)
display(df)

Problem description

In the code above, we are:

  1. Creating a new dataframe res
  2. Getting a copy of the dataframe where values are less than 8 and assigning that to df
  3. Setting values in df using the .loc indexing

Step 3 raises the SettingWithCopyWarning. I don't believe we are setting the value of a copy there as we are using the .loc indexing method, which should set values in df. The values are being set, but the warning is being raised. If I'm understand the warning correctly, it should not be raised.

Expected Output

No warning should be raised

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 2cb9652
python : 3.7.9.final.0
python-bits : 64
OS : Linux
OS-release : 5.8.0-50-generic
Version : #56~20.04.1-Ubuntu SMP Mon Apr 12 21:46:35 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.4
numpy : 1.19.1
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.2
setuptools : 49.6.0.post20200814
Cython : None
pytest : 6.2.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.18.1
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.5.2
sqlalchemy : None
tables : None
tabulate : 0.8.7
xarray : None
xlrd : None
xlwt : None
numba : 0.51.1

@ad12 ad12 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 4, 2021
@Liam3851
Copy link
Contributor

Liam3851 commented May 4, 2021

df = res[res["Value"] <= 8]

does not (necessarily) create a copy, it (at least can) create a view on res, hence the warning. SettingWithCopyWarning is in place specifically to catch this use case.

@attack68
Copy link
Contributor

attack68 commented May 4, 2021

if you remove df from your code and just run:

res = pd.DataFrame({"Method": ["Alpha1", "Beta1", "Beta2", "Alpha2", "Alpha3"], "Value": [5, 6, 7, 8, 10]})
res[res["Value"] <= 8].loc[res[res["Value"] <= 8]["Method"].str.contains("Alpha"), "Method"] = "Gamma"

you get the same warning.

As @Liam3851 states you are not necessarily creating a copy.

The error goes away if you explicitly create a copy.

df = res[res["Value"] <= 8].copy()

@attack68 attack68 added Closing Candidate May be closeable, needs more eyeballs and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 4, 2021
@jreback
Copy link
Contributor

jreback commented May 4, 2021

pls read the https://pandas.pydata.org/docs/user_guide/indexing.html#indexing-view-versus-copy

this is virtually the same example.

@jreback jreback closed this as completed May 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closing Candidate May be closeable, needs more eyeballs
Projects
None yet
Development

No branches or pull requests

4 participants