DOC: Add clearer info when copy is False but memory shared only for certain objects #41514

mdhsieh · 2021-05-16T23:07:46Z

closes DOC: pandas.Series(data=None, index=None, dtype=None, name=None, copy=False, fastpath=False) #41423
tests added / passed
Ensure all linting tests pass, see here for how to run them
whatsnew entry

Added to Series section more info in copy constructor parameter and 2 examples.

Ran validation script:

python scripts/validate_docstrings.py pandas.Series

Output:

################################################################################
########################## Docstring (pandas.Series)  ##########################
################################################################################

One-dimensional ndarray with axis labels (including time series).

Labels need not be unique but must be a hashable type. The object
supports both integer- and label-based indexing and provides a host of
methods for performing operations involving the index. Statistical
methods from ndarray have been overridden to automatically exclude
missing data (currently represented as NaN).

Operations between Series (+, -, /, *, **) align values based on their
associated index values-- they need not be the same length. The result
index will be the sorted union of the two indexes.

Parameters
----------
data : array-like, Iterable, dict, or scalar value
    Contains data stored in Series. If data is a dict, argument order is
    maintained.
index : array-like or Index (1d)
    Values must be hashable and have the same length as `data`.
    Non-unique index values are allowed. Will default to
    RangeIndex (0, 1, 2, ..., n) if not provided. If data is dict-like
    and index is None, then the keys in the data are used as the index. If the
    index is not None, the resulting Series is reindexed with the index values.
dtype : str, numpy.dtype, or ExtensionDtype, optional
    Data type for the output Series. If not specified, this will be
    inferred from `data`.
    See the :ref:`user guide <basics.dtypes>` for more usages.
name : str, optional
    The name to give to the Series.
copy : bool, default False
    Copy input data. If False and the Series returns a `copy`
    of the data, the memory location for the values is not shared.
    If False and the Series returns a `view` on the data,
    the memory location for the values is shared.

Examples
--------
Constructing Series from a dictionary with an Index specified

>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> ser = pd.Series(data=d, index=['a', 'b', 'c'])
>>> ser
a   1
b   2
c   3
dtype: int64

The keys of the dictionary match with the Index values, hence the Index
values have no effect.

>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> ser = pd.Series(data=d, index=['x', 'y', 'z'])
>>> ser
x   NaN
y   NaN
z   NaN
dtype: float64

Note that the Index is first build with the keys from the dictionary.
After this the Series is reindexed with the given Index values, hence we
get all NaN as a result.

Constructing Series from an array with `copy=False`.

>>> r = [1,2]
>>> ser = pd.Series(r, copy=False)
>>> ser.iloc[0] = 999
>>> r
[1, 2]
>>> ser
0    999
1      2
dtype: int64

The Series returns a `copy` of the original data, so
the original data is unchanged.

Constructing Series from a `numpy.array` with `copy=False`.

>>> r = np.array([1,2])
>>> ser = pd.Series(r, copy=False)
>>> ser.iloc[0] = 999
>>> r
array([999,   2])
>>> ser
0    999
1      2
dtype: int32

The Series returns a `view` on the original data, so
the original data is changed as well.

################################################################################
################################## Validation ##################################
################################################################################

3 Errors found:
        Parameters {'fastpath'} not documented
        See Also section not found
        flake8 error: E902 PermissionError: [Errno 13] Permission denied: 'C:\\Users\\mdhsi\\AppData\\Local\\Temp\\tmpxz50buet'

…41423). Update is on main Series documentation.

…v#41423). Original data unchanged when Series is a Copy, but original data is changed when Series is a View.

…v#41423). After building documentation, found the code examples were not displayed correctly. Added newlines and rebuilt to fix.

pep8speaks · 2021-05-16T23:07:49Z

Hello @mdhsieh! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-05-20 00:41:26 UTC

mdhsieh · 2021-05-16T23:38:09Z

@github-actions pre-commit

mzeitlin11

Thanks for helping to clarify this behavior @mdhsieh! Left some comments below

mzeitlin11 · 2021-05-17T03:17:05Z

pandas/core/series.py

@@ -251,6 +254,36 @@ class Series(base.IndexOpsMixin, generic.NDFrame):
    Note that the Index is first build with the keys from the dictionary.
    After this the Series is reindexed with the given Index values, hence we
    get all NaN as a result.
+
+    Constructing Series from an array with `copy=False`.


I'd call this a list, since the difference between an array and list in this context is important in terms of ability to share memory.

mzeitlin11 · 2021-05-17T03:17:46Z

pandas/core/series.py

+    1      2
+    dtype: int64
+
+    The Series returns a `copy` of the original data, so


What do you mean by returns here? It's more like the data backing the series does not share the same memory as the input r.

Oh right, by "returns" a copy I meant the Series created had a copy of the data. I guess "returns" is not the correct word to use.

mzeitlin11 · 2021-05-17T03:19:14Z

pandas/core/series.py

+
+    Constructing Series from an array with `copy=False`.
+
+    >>> r = [1,2]


Suggested change

>>> r = [1,2]

>>> r = [1, 2]

mzeitlin11 · 2021-05-17T03:19:23Z

pandas/core/series.py

+
+    Constructing Series from a `numpy.array` with `copy=False`.
+
+    >>> r = np.array([1,2])


Suggested change

>>> r = np.array([1,2])

>>> r = np.array([1, 2])

mzeitlin11 · 2021-05-17T03:23:59Z

pandas/core/series.py

@@ -223,7 +223,10 @@ class Series(base.IndexOpsMixin, generic.NDFrame):
    name : str, optional
        The name to give to the Series.
    copy : bool, default False
-        Copy input data.
+        Copy input data. If False and the Series returns a `copy`


Same comment on usage of "returns" as below.

Generally I think this description might be too verbose for a parameter description and could be shortened to basically just get the idea across that a copy may still be made if necessary (as in the case of a list where we cannot share memory). Then the examples below should suffice to explain the different possible cases.

(https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html has a good example of this)

mzeitlin11 · 2021-05-17T03:24:35Z

pandas/core/series.py

+    dtype: int32
+
+    The Series returns a `view` on the original data, so
+    the original data is changed as well.


Same comment as above on wording of returns

attack68

good start, very similar comments to @mzeitlin11 and suggetsed changes.

attack68 · 2021-05-17T05:09:10Z

pandas/core/series.py

+        Copy input data. If False and the Series returns a `copy`
+        of the data, the memory location for the values is not shared.
+        If False and the Series returns a `view` on the data,
+        the memory location for the values is shared.


Suggested change

Copy input data. If False and the Series returns a `copy`

of the data, the memory location for the values is not shared.

If False and the Series returns a `view` on the data,

the memory location for the values is shared.

Copy input data. Only affects Series or 1d ndarray input. See examples.

attack68 · 2021-05-17T05:13:31Z

pandas/core/series.py

@@ -251,6 +254,36 @@ class Series(base.IndexOpsMixin, generic.NDFrame):
    Note that the Index is first build with the keys from the dictionary.
    After this the Series is reindexed with the given Index values, hence we
    get all NaN as a result.
+
+    Constructing Series from an array with `copy=False`.


Suggested change

Constructing Series from an array with `copy=False`.

Constructing Series from a list with `copy=False`.

attack68 · 2021-05-17T05:15:00Z

pandas/core/series.py

+    1      2
+    dtype: int64
+
+    The Series returns a `copy` of the original data, so


Suggested change

The Series returns a `copy` of the original data, so

Due to input data type the Series is created with a `copy` even though `copy=False`, so

attack68 · 2021-05-17T05:16:26Z

pandas/core/series.py

+    1      2
+    dtype: int32
+
+    The Series returns a `view` on the original data, so


Suggested change

The Series returns a `view` on the original data, so

Due to input data type here the Series represents a `view` on the original data, so

…1423). Remove return, replace with has. Remove array, replace with list. Fix array spacing to match other Series examples. Replace long param info with only affects sentence, based off DataFrame documentation. Reword example explanation. Change numpy.array to 1d ndarray to match other documentation, like DataFrame.

…copy

…-series-copy" This reverts commit 5eab8f9, reversing changes made to 671cf86. Need to revert merge commit to find out why build is failing after pushing branches. These were the commands run to do the revert, at previous commit: check parent branches of the merge commit git cat-file -p 5eab8f9 then revert commit git revert --no-commit -m 2 5eab8f9 Then accepted incoming change in series.py, which removed my info and examples added.

…#41423). Remove whitespace on blank lines and line break sentences to avoid style errors. Previous commit had 1 failing check.

MarcoGorelli · 2021-05-19T06:52:57Z

pandas/core/series.py

+    >>> r = np.array([1, 2])
+    >>> ser = pd.Series(r, copy=False)
+    >>> ser.iloc[0] = 999
+    >>> r
+    array([999,   2])
+    >>> ser
+    0    999
+    1      2
+    dtype: int32


looks like this test is failing in CI:

=================================== FAILURES =================================== _____________________ [doctest] pandas.core.series.Series ______________________ 269 the data is unchanged. 270 271 Constructing Series from a 1d ndarray with `copy=False`. 272 273 >>> r = np.array([1, 2]) 274 >>> ser = pd.Series(r, copy=False) 275 >>> ser.iloc[0] = 999 276 >>> r 277 array([999, 2]) 278 >>> ser Differences (unified diff with -expected +actual): @@ -1,3 +1,3 @@ 0 999 1 2 -dtype: int32 +dtype: int64

Oh got it, I didn't see that. Appreciate the help.

…de example.

mdhsieh · 2021-05-20T02:11:29Z

Thanks @mzeitlin11 @attack68 . I've tried rewording as suggested.

attack68

looks clearer to me. thanks.

jreback · 2021-05-21T16:12:08Z

thanks @mdhsieh

…ertain objects (pandas-dev#41514)

mdhsieh added 3 commits May 14, 2021 15:46

DOC: Add info on memory location sharing when copy=False (pandas-dev#…

f9efd2a

…41423). Update is on main Series documentation.

DOC: Add 2 examples when creating Series and copy is False (pandas-de…

0c70b9f

…v#41423). Original data unchanged when Series is a Copy, but original data is changed when Series is a View.

DOC: Fix code format of 2 examples added to Series section (pandas-de…

2ed2b8d

…v#41423). After building documentation, found the code examples were not displayed correctly. Added newlines and rebuilt to fix.

mzeitlin11 reviewed May 17, 2021

View reviewed changes

mzeitlin11 added Constructors Series/DataFrame/Index/pd.array Constructors Copy / view semantics Docs labels May 17, 2021

attack68 requested changes May 17, 2021

View reviewed changes

mdhsieh added 6 commits May 17, 2021 23:24

DOC: Try fix PEP8 spacing issue.

ac82b6b

Merge remote-tracking branch 'upstream/master' into 41423-doc-series-…

5eab8f9

…copy

DOC: Try fix one line which is too long.

0e147a1

DOC: Add reworded parameter info and and 2 examples again (pandas-dev…

52ce3ba

…#41423). Remove whitespace on blank lines and line break sentences to avoid style errors. Previous commit had 1 failing check.

mdhsieh force-pushed the 41423-doc-series-copy branch from a4f512e to 52ce3ba Compare May 18, 2021 21:05

MarcoGorelli reviewed May 19, 2021

View reviewed changes

DOC: Change type output from int32 to int64 in ndarray data Series co…

818d5d6

…de example.

mdhsieh requested review from MarcoGorelli, mzeitlin11 and attack68 May 20, 2021 02:11

attack68 approved these changes May 20, 2021

View reviewed changes

jreback added this to the 1.3 milestone May 21, 2021

jreback merged commit 3980696 into pandas-dev:master May 21, 2021

TLouf pushed a commit to TLouf/pandas that referenced this pull request Jun 1, 2021

DOC: Add clearer info when copy is False but memory shared only for c…

4d70dc7

…ertain objects (pandas-dev#41514)

JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021

DOC: Add clearer info when copy is False but memory shared only for c…

1b7bbb6

…ertain objects (pandas-dev#41514)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: Add clearer info when copy is False but memory shared only for certain objects #41514

DOC: Add clearer info when copy is False but memory shared only for certain objects #41514

mdhsieh commented May 16, 2021

pep8speaks commented May 16, 2021 •

edited

Loading

mdhsieh commented May 16, 2021

mzeitlin11 left a comment

mzeitlin11 May 17, 2021

mzeitlin11 May 17, 2021

mdhsieh May 18, 2021

mzeitlin11 May 17, 2021

mzeitlin11 May 17, 2021

mzeitlin11 May 17, 2021

mzeitlin11 May 17, 2021

mzeitlin11 May 17, 2021

attack68 left a comment

attack68 May 17, 2021

attack68 May 17, 2021

attack68 May 17, 2021

attack68 May 17, 2021

MarcoGorelli May 19, 2021

mdhsieh May 20, 2021

mdhsieh commented May 20, 2021

attack68 left a comment

jreback commented May 21, 2021


		Constructing Series from an array with `copy=False`.

		>>> r = [1,2]


		Constructing Series from a `numpy.array` with `copy=False`.

		>>> r = np.array([1,2])

	Constructing Series from an array with `copy=False`.
	Constructing Series from a list with `copy=False`.

	The Series returns a `copy` of the original data, so
	Due to input data type the Series is created with a `copy` even though `copy=False`, so

	The Series returns a `view` on the original data, so
	Due to input data type here the Series represents a `view` on the original data, so

DOC: Add clearer info when copy is False but memory shared only for certain objects #41514

DOC: Add clearer info when copy is False but memory shared only for certain objects #41514

Conversation

mdhsieh commented May 16, 2021

pep8speaks commented May 16, 2021 • edited Loading

Comment last updated at 2021-05-20 00:41:26 UTC

mdhsieh commented May 16, 2021

mzeitlin11 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

attack68 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdhsieh commented May 20, 2021

attack68 left a comment

Choose a reason for hiding this comment

jreback commented May 21, 2021

pep8speaks commented May 16, 2021 •

edited

Loading