Skip to content

BUG: DataFrame from query result should not give SettingWithCopyWarning #55451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
limwz01 opened this issue Oct 9, 2023 · 0 comments · Fixed by #56614
Closed
2 of 3 tasks

BUG: DataFrame from query result should not give SettingWithCopyWarning #55451

limwz01 opened this issue Oct 9, 2023 · 0 comments · Fixed by #56614
Labels
Bug Copy / view semantics Needs Triage Issue that has not been reviewed by a pandas team member

Comments

@limwz01
Copy link

limwz01 commented Oct 9, 2023

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
def main():
    df = pd.DataFrame({"v": [0, 1, 2]})
    df2 = df.query("v < 2")
    df2["v"] = 0
    print(df)
    df = pd.DataFrame({"v": [0, 1, 2]})
    df = df.query("v < 2")
    df["v"] = 0
    print(df)
main()

Issue Description

The query method already always seems to return a copy, and that should be expected. Writing to the result should not trigger a SettingWithCopyWarning, especially if the original DataFrame is already out of scope.

The first example shows this unexpected behaviour where the original DataFrame is still in scope, and the assignment statement causes a SettingWithCopyWarning. But as seen, the changes to the resulting DataFrame does not affect the original anyway, and the behaviour is consistent, so why should there be a warning?

In the second example, the reference count does not seem to work properly and the weakref to the original DataFrame is still alive, and hence we still get a SettingWithCopyWarning. This is definitely a bug, but I am proposing that we solve it by setting the _is_copy to None for all query() results.

Expected Behavior

No SettingWithCopyWarning should be produced.

Installed Versions

INSTALLED VERSIONS

commit : e86ed37
python : 3.11.2.final.0
python-bits : 64
OS : Linux

pandas : 2.1.1
numpy : 1.26.0

@limwz01 limwz01 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Copy / view semantics Needs Triage Issue that has not been reviewed by a pandas team member
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants