Skip to content

CLN: Use to_numpy directly in cov / corr #34031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
May 23, 2020
Merged

CLN: Use to_numpy directly in cov / corr #34031

Changes from 37 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
054d74e
ENH: Add na_value argument to DataFrame.to_numpy
dsaxton Apr 29, 2020
cafbf5f
Add some tests
dsaxton Apr 29, 2020
34b9b9f
Issue num
dsaxton Apr 30, 2020
9a2bbd6
A little better
dsaxton Apr 30, 2020
f7dc246
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton Apr 30, 2020
5eb8bb2
as_array
dsaxton May 1, 2020
ec2f729
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 1, 2020
b48b6c9
Black
dsaxton May 1, 2020
d1a60e8
More black
dsaxton May 1, 2020
09fdf51
to_numpy for ExtensionBlock
dsaxton May 2, 2020
f5db15a
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 2, 2020
89e8930
dtype hack
dsaxton May 2, 2020
d24b976
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 2, 2020
bec3889
reshape
dsaxton May 3, 2020
a20f116
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 3, 2020
02405a1
Check for NA first
dsaxton May 3, 2020
055413f
Cast non-extension single block
dsaxton May 3, 2020
ae088e4
Test nit
dsaxton May 3, 2020
d78ba29
Update pandas/core/frame.py
dsaxton May 4, 2020
9c87e00
side effect test
dsaxton May 4, 2020
df5b683
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 4, 2020
ae2b34a
Update pandas/tests/base/test_conversion.py
dsaxton May 4, 2020
bcb69c5
Copy
dsaxton May 4, 2020
c3a7a55
Copy less
dsaxton May 4, 2020
b5ec43f
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 4, 2020
f3e45d7
should_copy
dsaxton May 5, 2020
e54cc28
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 5, 2020
491a5ae
Update pandas/core/internals/managers.py
dsaxton May 5, 2020
addfdf8
CLN: Use to_numpy directly in cov / corr
dsaxton May 3, 2020
2ca95cd
Merge remote-tracking branch 'upstream/master' into to_numpy-corr
dsaxton May 9, 2020
4ecccff
Rename and comment
dsaxton May 9, 2020
142c808
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 9, 2020
8d42fd4
Don't special case datetimetz
dsaxton May 9, 2020
c2228bf
Merge remote-tracking branch 'upstream/master' into dataframe-to-numpy
dsaxton May 12, 2020
4ddda12
Merge remote-tracking branch 'upstream/master' into to_numpy-corr
dsaxton May 12, 2020
c593fb4
Merge branch 'dataframe-to-numpy' into to_numpy-corr
dsaxton May 12, 2020
3446131
Merge remote-tracking branch 'upstream/master' into to_numpy-corr
dsaxton May 13, 2020
b61087f
copy=False
dsaxton May 20, 2020
b3237b1
Merge remote-tracking branch 'upstream/master' into to_numpy-corr
dsaxton May 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pandas/core/frame.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7973,7 +7973,7 @@ def corr(self, method="pearson", min_periods=1) -> "DataFrame":
numeric_df = self._get_numeric_data()
cols = numeric_df.columns
idx = cols.copy()
mat = numeric_df.astype(float, copy=False).to_numpy()
mat = numeric_df.to_numpy(dtype=float, na_value=np.nan)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have copy=False by default in .to_numpy()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes but I can make this explicit


if method == "pearson":
correl = libalgos.nancorr(mat, minp=min_periods)
Expand Down Expand Up @@ -8108,7 +8108,7 @@ def cov(self, min_periods=None) -> "DataFrame":
numeric_df = self._get_numeric_data()
cols = numeric_df.columns
idx = cols.copy()
mat = numeric_df.astype(float, copy=False).to_numpy()
mat = numeric_df.to_numpy(dtype=float, na_value=np.nan)

if notna(mat).all():
if min_periods is not None and min_periods > len(mat):
Expand Down