Skip to content

Commit 9451999

Browse files
rushabh-vrushabh-v
authored
Merge branch 'master' into pre-head
2 parents 5fc40bd + 64336ff commit 9451999

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+3001
-2881
lines changed

‎README.md

+10-76
Original file line numberDiff line numberDiff line change
@@ -5,82 +5,16 @@
55
-----------------
66

77
# pandas: powerful Python data analysis toolkit
8-
9-
<table>
10-
<tr>
11-
<td>Latest Release</td>
12-
<td>
13-
<a href="https://pypi.org/project/pandas/">
14-
<img src="https://img.shields.io/pypi/v/pandas.svg" alt="latest release" />
15-
</a>
16-
</td>
17-
</tr>
18-
<td></td>
19-
<td>
20-
<a href="https://anaconda.org/anaconda/pandas/">
21-
<img src="https://anaconda.org/conda-forge/pandas/badges/version.svg" alt="latest release" />
22-
</a>
23-
</td>
24-
</tr>
25-
<tr>
26-
<td>Package Status</td>
27-
<td>
28-
<a href="https://pypi.org/project/pandas/">
29-
<img src="https://img.shields.io/pypi/status/pandas.svg" alt="status" />
30-
</a>
31-
</td>
32-
</tr>
33-
<tr>
34-
<td>License</td>
35-
<td>
36-
<a href="https://github.com/pandas-dev/pandas/blob/master/LICENSE">
37-
<img src="https://img.shields.io/pypi/l/pandas.svg" alt="license" />
38-
</a>
39-
</td>
40-
</tr>
41-
<tr>
42-
<td>Build Status</td>
43-
<td>
44-
<a href="https://travis-ci.org/pandas-dev/pandas">
45-
<img src="https://travis-ci.org/pandas-dev/pandas.svg?branch=master" alt="travis build status" />
46-
</a>
47-
</td>
48-
</tr>
49-
<tr>
50-
<td></td>
51-
<td>
52-
<a href="https://dev.azure.com/pandas-dev/pandas/_build/latest?definitionId=1&branch=master">
53-
<img src="https://dev.azure.com/pandas-dev/pandas/_apis/build/status/pandas-dev.pandas?branch=master" alt="Azure Pipelines build status" />
54-
</a>
55-
</td>
56-
</tr>
57-
<tr>
58-
<td>Coverage</td>
59-
 <td>
60-
<a href="https://codecov.io/gh/pandas-dev/pandas">
61-
<img src="https://codecov.io/github/pandas-dev/pandas/coverage.svg?branch=master" alt="coverage" />
62-
</a>
63-
</td>
64-
</tr>
65-
<tr>
66-
<td>Downloads</td>
67-
<td>
68-
<a href="https://pandas.pydata.org">
69-
<img src="https://anaconda.org/conda-forge/pandas/badges/downloads.svg" alt="conda-forge downloads" />
70-
</a>
71-
</td>
72-
</tr>
73-
<tr>
74-
<td>Gitter</td>
75-
<td>
76-
<a href="https://gitter.im/pydata/pandas">
77-
<img src="https://badges.gitter.im/Join%20Chat.svg" />
78-
</a>
79-
</td>
80-
</tr>
81-
</table>
82-
83-
8+
[![PyPI Latest Release](https://img.shields.io/pypi/v/pandas.svg)](https://pypi.org/project/pandas/)
9+
[![Conda Latest Release](https://anaconda.org/conda-forge/pandas/badges/version.svg)](https://anaconda.org/anaconda/pandas/)
10+
[![Package Status](https://img.shields.io/pypi/status/pandas.svg)](https://pypi.org/project/pandas/)
11+
[![License](https://img.shields.io/pypi/l/pandas.svg)](https://github.com/pandas-dev/pandas/blob/master/LICENSE)
12+
[![Travis Build Status](https://travis-ci.org/pandas-dev/pandas.svg?branch=master)](https://travis-ci.org/pandas-dev/pandas)
13+
[![Azure Build Status](https://dev.azure.com/pandas-dev/pandas/_apis/build/status/pandas-dev.pandas?branch=master)](https://dev.azure.com/pandas-dev/pandas/_build/latest?definitionId=1&branch=master)
14+
[![Coverage](https://codecov.io/github/pandas-dev/pandas/coverage.svg?branch=master)](https://codecov.io/gh/pandas-dev/pandas)
15+
[![Downloads](https://anaconda.org/conda-forge/pandas/badges/downloads.svg)](https://pandas.pydata.org)
16+
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/pydata/pandas)
17+
[![Powered by NumFOCUS](https://img.shields.io/badge/powered%20by-NumFOCUS-orange.svg?style=flat&colorA=E1523D&colorB=007D8A)](https://numfocus.org)
8418

8519
## What is it?
8620

‎ci/deps/travis-36-cov.yaml

+1-2
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,7 @@ dependencies:
2727
- numexpr
2828
- numpy=1.15.*
2929
- odfpy
30-
- openpyxl<=3.0.1
31-
# https://github.com/pandas-dev/pandas/pull/30009 openpyxl 3.0.2 broke
30+
- openpyxl
3231
- pandas-gbq
3332
- psycopg2
3433
- pyarrow>=0.13.0

‎doc/source/whatsnew/v1.0.1.rst

+11-1
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,14 @@ including other versions of pandas.
1010

1111
.. ---------------------------------------------------------------------------
1212
13+
.. _whatsnew_101.deprecations:
14+
15+
Deprecations
16+
~~~~~~~~~~~~
17+
18+
- Support for negative integer for :attr:`pd.options.display.max_colwidth` is deprecated in favor of using ``None`` (:issue:`31532`)
19+
20+
.. ---------------------------------------------------------------------------
1321
1422
.. _whatsnew_101.bug_fixes:
1523

@@ -123,12 +131,14 @@ ExtensionArray
123131
^^^^^^^^^^^^^^
124132

125133
- Bug in dtype being lost in ``__invert__`` (``~`` operator) for extension-array backed ``Series`` and ``DataFrame`` (:issue:`23087`)
134+
- Bug where :meth:`qcut` would raise when passed a nullable integer. (:issue:`31389`)
126135
-
127136

128137

129138
Other
130139
^^^^^
131-
-
140+
- Regression fixed in objTOJSON.c fix return-type warning (:issue:`31463`)
141+
- Fixed a regression where setting :attr:`pd.options.display.max_colwidth` was not accepting negative integer. In addition, this behavior has been deprecated in favor of using ``None`` (:issue:`31532`)
132142
-
133143

134144
.. ---------------------------------------------------------------------------

‎doc/source/whatsnew/v1.1.0.rst

+14-4
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,9 @@ Backwards incompatible API changes
6060
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6161
- :meth:`DataFrame.swaplevels` now raises a ``TypeError`` if the axis is not a :class:`MultiIndex`.
6262
Previously a ``AttributeError`` was raised (:issue:`31126`)
63+
- :meth:`DataFrameGroupby.mean` and :meth:`SeriesGroupby.mean` (and similarly for :meth:`~DataFrameGroupby.median`, :meth:`~DataFrameGroupby.std`` and :meth:`~DataFrameGroupby.var``)
64+
now raise a ``TypeError`` if a not-accepted keyword argument is passed into it.
65+
Previously a ``UnsupportedFunctionCall`` was raised (``AssertionError`` if ``min_count`` passed into :meth:`~DataFrameGroupby.median``) (:issue:`31485`)
6366

6467

6568
.. ---------------------------------------------------------------------------
@@ -105,11 +108,12 @@ Datetimelike
105108
- Bug in :class:`Timestamp` where constructing :class:`Timestamp` from ambiguous epoch time and calling constructor again changed :meth:`Timestamp.value` property (:issue:`24329`)
106109
- :meth:`DatetimeArray.searchsorted`, :meth:`TimedeltaArray.searchsorted`, :meth:`PeriodArray.searchsorted` not recognizing non-pandas scalars and incorrectly raising ``ValueError`` instead of ``TypeError`` (:issue:`30950`)
107110
- Bug in :class:`Timestamp` where constructing :class:`Timestamp` with dateutil timezone less than 128 nanoseconds before daylight saving time switch from winter to summer would result in nonexistent time (:issue:`31043`)
111+
- Bug in :meth:`DataFrame.reindex` and :meth:`Series.reindex` when reindexing with a tz-aware index (:issue:`26683`)
108112

109113
Timedelta
110114
^^^^^^^^^
111115

112-
-
116+
- Bug in constructing a :class:`Timedelta` with a high precision integer that would round the :class:`Timedelta` components (:issue:`31354`)
113117
-
114118

115119
Timezones
@@ -151,7 +155,6 @@ Indexing
151155
- Bug in :meth:`Series.at` and :meth:`DataFrame.at` not matching ``.loc`` behavior when looking up an integer in a :class:`Float64Index` (:issue:`31329`)
152156
- Bug in :meth:`PeriodIndex.is_monotonic` incorrectly returning ``True`` when containing leading ``NaT`` entries (:issue:`31437`)
153157
- Bug in :meth:`DatetimeIndex.get_loc` raising ``KeyError`` with converted-integer key instead of the user-passed key (:issue:`31425`)
154-
-
155158

156159
Missing
157160
^^^^^^^
@@ -161,14 +164,21 @@ Missing
161164

162165
MultiIndex
163166
^^^^^^^^^^
167+
- Bug in :meth:`Dataframe.loc` when used with a :class:`MultiIndex`. The returned values were not in the same order as the given inputs (:issue:`22797`)
164168

165-
-
169+
.. ipython:: python
170+
171+
df = pd.DataFrame(np.arange(4),
172+
index=[["a", "a", "b", "b"], [1, 2, 1, 2]])
173+
# Rows are now ordered as the requested keys
174+
df.loc[(['b', 'a'], [2, 1]), :]
166175
-
167176

168177
I/O
169178
^^^
170179
- Bug in :meth:`read_json` where integer overflow was occuring when json contains big number strings. (:issue:`30320`)
171180
- `read_csv` will now raise a ``ValueError`` when arguments `header` and `prefix` both are not None. (:issue:`27394`)
181+
- Bug in :meth:`DataFrame.to_json` was raising ``NotFoundError`` when ``path_or_buf`` was an S3 URI (:issue:`28375`)
172182
-
173183

174184
Plotting
@@ -211,7 +221,7 @@ Other
211221
^^^^^
212222
- Appending a dictionary to a :class:`DataFrame` without passing ``ignore_index=True`` will raise ``TypeError: Can only append a dict if ignore_index=True``
213223
instead of ``TypeError: Can only append a Series if ignore_index=True or if the Series has a name`` (:issue:`30871`)
214-
-
224+
- Set operations on an object-dtype :class:`Index` now always return object-dtype results (:issue:`31401`)
215225

216226
.. ---------------------------------------------------------------------------
217227

‎pandas/_libs/index.pyx

+13-20
Original file line numberDiff line numberDiff line change
@@ -87,20 +87,6 @@ cdef class IndexEngine:
8787
else:
8888
return get_value_at(arr, loc, tz=tz)
8989

90-
cpdef set_value(self, ndarray arr, object key, object value):
91-
"""
92-
Parameters
93-
----------
94-
arr : 1-dimensional ndarray
95-
"""
96-
cdef:
97-
object loc
98-
99-
loc = self.get_loc(key)
100-
value = convert_scalar(arr, value)
101-
102-
arr[loc] = value
103-
10490
cpdef get_loc(self, object val):
10591
cdef:
10692
Py_ssize_t loc
@@ -585,16 +571,23 @@ cpdef convert_scalar(ndarray arr, object value):
585571
raise ValueError("cannot set a Timedelta with a non-timedelta "
586572
f"{type(value).__name__}")
587573

588-
if (issubclass(arr.dtype.type, (np.integer, np.floating, np.complex)) and
589-
not issubclass(arr.dtype.type, np.bool_)):
590-
if util.is_bool_object(value):
591-
raise ValueError("Cannot assign bool to float/integer series")
574+
else:
575+
validate_numeric_casting(arr.dtype, value)
576+
577+
return value
578+
592579

593-
if issubclass(arr.dtype.type, (np.integer, np.bool_)):
580+
cpdef validate_numeric_casting(dtype, object value):
581+
# Note: we can't annotate dtype as cnp.dtype because that cases dtype.type
582+
# to integer
583+
if issubclass(dtype.type, (np.integer, np.bool_)):
594584
if util.is_float_object(value) and value != value:
595585
raise ValueError("Cannot assign nan to integer series")
596586

597-
return value
587+
if (issubclass(dtype.type, (np.integer, np.floating, np.complex)) and
588+
not issubclass(dtype.type, np.bool_)):
589+
if util.is_bool_object(value):
590+
raise ValueError("Cannot assign bool to float/integer series")
598591

599592

600593
cdef class BaseMultiIndexCodesEngine:

‎pandas/_libs/src/ujson/python/objToJSON.c

+14-26
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,6 @@ typedef struct __PyObjectEncoder {
127127
// pass-through to encode numpy data directly
128128
int npyType;
129129
void *npyValue;
130-
TypeContext basicTypeContext;
131130

132131
int datetimeIso;
133132
NPY_DATETIMEUNIT datetimeUnit;
@@ -177,6 +176,8 @@ void *initObjToJSON(void) {
177176

178177
/* Initialise numpy API */
179178
import_array();
179+
// GH 31463
180+
return NULL;
180181
}
181182

182183
static TypeContext *createTypeContext(void) {
@@ -925,15 +926,15 @@ char *Tuple_iterGetName(JSOBJ Py_UNUSED(obj), JSONTypeContext *Py_UNUSED(tc),
925926
}
926927

927928
//=============================================================================
928-
// Iterator iteration functions
929+
// Set iteration functions
929930
// itemValue is borrowed reference, no ref counting
930931
//=============================================================================
931-
void Iter_iterBegin(JSOBJ obj, JSONTypeContext *tc) {
932+
void Set_iterBegin(JSOBJ obj, JSONTypeContext *tc) {
932933
GET_TC(tc)->itemValue = NULL;
933934
GET_TC(tc)->iterator = PyObject_GetIter(obj);
934935
}
935936

936-
int Iter_iterNext(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
937+
int Set_iterNext(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
937938
PyObject *item;
938939

939940
if (GET_TC(tc)->itemValue) {
@@ -951,7 +952,7 @@ int Iter_iterNext(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
951952
return 1;
952953
}
953954

954-
void Iter_iterEnd(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
955+
void Set_iterEnd(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
955956
if (GET_TC(tc)->itemValue) {
956957
Py_DECREF(GET_TC(tc)->itemValue);
957958
GET_TC(tc)->itemValue = NULL;
@@ -963,11 +964,11 @@ void Iter_iterEnd(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
963964
}
964965
}
965966

966-
JSOBJ Iter_iterGetValue(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
967+
JSOBJ Set_iterGetValue(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
967968
return GET_TC(tc)->itemValue;
968969
}
969970

970-
char *Iter_iterGetName(JSOBJ Py_UNUSED(obj), JSONTypeContext *Py_UNUSED(tc),
971+
char *Set_iterGetName(JSOBJ Py_UNUSED(obj), JSONTypeContext *Py_UNUSED(tc),
971972
size_t *Py_UNUSED(outLen)) {
972973
return NULL;
973974
}
@@ -2040,11 +2041,11 @@ void Object_beginTypeContext(JSOBJ _obj, JSONTypeContext *tc) {
20402041
} else if (PyAnySet_Check(obj)) {
20412042
PRINTMARK();
20422043
tc->type = JT_ARRAY;
2043-
pc->iterBegin = Iter_iterBegin;
2044-
pc->iterEnd = Iter_iterEnd;
2045-
pc->iterNext = Iter_iterNext;
2046-
pc->iterGetValue = Iter_iterGetValue;
2047-
pc->iterGetName = Iter_iterGetName;
2044+
pc->iterBegin = Set_iterBegin;
2045+
pc->iterEnd = Set_iterEnd;
2046+
pc->iterNext = Set_iterNext;
2047+
pc->iterGetValue = Set_iterGetValue;
2048+
pc->iterGetName = Set_iterGetName;
20482049
return;
20492050
}
20502051

@@ -2115,10 +2116,7 @@ void Object_endTypeContext(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
21152116

21162117
PyObject_Free(GET_TC(tc)->cStr);
21172118
GET_TC(tc)->cStr = NULL;
2118-
if (tc->prv !=
2119-
&(((PyObjectEncoder *)tc->encoder)->basicTypeContext)) { // NOLINT
2120-
PyObject_Free(tc->prv);
2121-
}
2119+
PyObject_Free(tc->prv);
21222120
tc->prv = NULL;
21232121
}
21242122
}
@@ -2216,16 +2214,6 @@ PyObject *objToJSON(PyObject *Py_UNUSED(self), PyObject *args,
22162214
pyEncoder.datetimeUnit = NPY_FR_ms;
22172215
pyEncoder.outputFormat = COLUMNS;
22182216
pyEncoder.defaultHandler = 0;
2219-
pyEncoder.basicTypeContext.newObj = NULL;
2220-
pyEncoder.basicTypeContext.dictObj = NULL;
2221-
pyEncoder.basicTypeContext.itemValue = NULL;
2222-
pyEncoder.basicTypeContext.itemName = NULL;
2223-
pyEncoder.basicTypeContext.attrList = NULL;
2224-
pyEncoder.basicTypeContext.iterator = NULL;
2225-
pyEncoder.basicTypeContext.cStr = NULL;
2226-
pyEncoder.basicTypeContext.npyarr = NULL;
2227-
pyEncoder.basicTypeContext.rowLabels = NULL;
2228-
pyEncoder.basicTypeContext.columnLabels = NULL;
22292217

22302218
PRINTMARK();
22312219

‎pandas/_libs/tslibs/conversion.pyx

+1-9
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ from pandas._libs.tslibs.util cimport (
2929
from pandas._libs.tslibs.timedeltas cimport cast_from_unit
3030
from pandas._libs.tslibs.timezones cimport (
3131
is_utc, is_tzlocal, is_fixed_offset, get_utcoffset, get_dst_info,
32-
get_timezone, maybe_get_tz, tz_compare, treat_tz_as_dateutil)
32+
get_timezone, maybe_get_tz, tz_compare)
3333
from pandas._libs.tslibs.timezones import UTC
3434
from pandas._libs.tslibs.parsing import parse_datetime_string
3535

@@ -341,14 +341,6 @@ cdef _TSObject convert_datetime_to_tsobject(datetime ts, object tz,
341341
obj.tzinfo = tz
342342
else:
343343
obj.value = pydatetime_to_dt64(ts, &obj.dts)
344-
# GH 24329 When datetime is ambiguous,
345-
# pydatetime_to_dt64 doesn't take DST into account
346-
# but with dateutil timezone, get_utcoffset does
347-
# so we need to correct for it
348-
if treat_tz_as_dateutil(ts.tzinfo):
349-
if ts.tzinfo.is_ambiguous(ts):
350-
dst_offset = ts.tzinfo.dst(ts)
351-
obj.value += int(dst_offset.total_seconds() * 1e9)
352344
obj.tzinfo = ts.tzinfo
353345

354346
if obj.tzinfo is not None and not is_utc(obj.tzinfo):

0 commit comments

Comments
 (0)