Skip to content

Commit dd77772

Browse files
Merge branch 'master' into ea-agg-fallback-fix-38980
2 parents ec4bdc3 + 6918b54 commit dd77772

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+626
-532
lines changed

doc/source/whatsnew/v1.2.1.rst

+12-11
Original file line numberDiff line numberDiff line change
@@ -14,22 +14,23 @@ including other versions of pandas.
1414

1515
Fixed regressions
1616
~~~~~~~~~~~~~~~~~
17-
- The deprecated attributes ``_AXIS_NAMES`` and ``_AXIS_NUMBERS`` of :class:`DataFrame` and :class:`Series` will no longer show up in ``dir`` or ``inspect.getmembers`` calls (:issue:`38740`)
1817
- Fixed regression in :meth:`to_csv` that created corrupted zip files when there were more rows than ``chunksize`` (:issue:`38714`)
19-
- Fixed a regression in ``groupby().rolling()`` where :class:`MultiIndex` levels were dropped (:issue:`38523`)
18+
- Fixed regression in ``groupby().rolling()`` where :class:`MultiIndex` levels were dropped (:issue:`38523`)
2019
- Fixed regression in repr of float-like strings of an ``object`` dtype having trailing 0's truncated after the decimal (:issue:`38708`)
2120
- Fixed regression in :meth:`DataFrame.groupby()` with :class:`Categorical` grouping column not showing unused categories for ``grouped.indices`` (:issue:`38642`)
2221
- Fixed regression in :meth:`DataFrame.any` and :meth:`DataFrame.all` not returning a result for tz-aware ``datetime64`` columns (:issue:`38723`)
2322
- Fixed regression in :meth:`DataFrame.__setitem__` raising ``ValueError`` when expanding :class:`DataFrame` and new column is from type ``"0 - name"`` (:issue:`39010`)
2423
- Fixed regression in :meth:`.GroupBy.sem` where the presence of non-numeric columns would cause an error instead of being dropped (:issue:`38774`)
2524
- Fixed regression in :meth:`DataFrame.loc.__setitem__` raising ``ValueError`` when :class:`DataFrame` has unsorted :class:`MultiIndex` columns and indexer is a scalar (:issue:`38601`)
2625
- Fixed regression in :func:`read_excel` with non-rawbyte file handles (:issue:`38788`)
27-
- Bug in :meth:`read_csv` with ``float_precision="high"`` caused segfault or wrong parsing of long exponent strings. This resulted in a regression in some cases as the default for ``float_precision`` was changed in pandas 1.2.0 (:issue:`38753`)
2826
- Fixed regression in :meth:`Rolling.skew` and :meth:`Rolling.kurt` modifying the object inplace (:issue:`38908`)
2927
- Fixed regression in :meth:`read_csv` and other read functions were the encoding error policy (``errors``) did not default to ``"replace"`` when no encoding was specified (:issue:`38989`)
30-
- Fixed regression in :meth:`DataFrame.replace` raising ValueError when :class:`DataFrame` has dtype ``bytes`` (:issue:`38900`)
28+
- Fixed regression in :meth:`DataFrame.replace` raising ``ValueError`` when :class:`DataFrame` has dtype ``bytes`` (:issue:`38900`)
3129
- Fixed regression in :meth:`DataFrameGroupBy.diff` raising for ``int8`` and ``int16`` columns (:issue:`39050`)
3230
- Fixed regression in :meth:`DataFrame.groupby()` when aggregating an :class:`ExtensionDType` that could fail for non-numeric values (:issue:`38980`)
31+
- Fixed regression that raised ``AttributeError`` with PyArrow versions [0.16.0, 1.0.0) (:issue:`38801`)
32+
-
33+
-
3334

3435
.. ---------------------------------------------------------------------------
3536
@@ -38,13 +39,7 @@ Fixed regressions
3839
Bug fixes
3940
~~~~~~~~~
4041

41-
I/O
42-
^^^
43-
44-
- Bumped minimum fastparquet version to 0.4.0 to avoid ``AttributeError`` from numba (:issue:`38344`)
45-
- Bumped minimum pymysql version to 0.8.1 to avoid test failures (:issue:`38344`)
46-
- Fixed ``AttributeError`` with PyArrow versions [0.16.0, 1.0.0) (:issue:`38801`)
47-
42+
- Bug in :meth:`read_csv` with ``float_precision="high"`` caused segfault or wrong parsing of long exponent strings. This resulted in a regression in some cases as the default for ``float_precision`` was changed in pandas 1.2.0 (:issue:`38753`)
4843
-
4944
-
5045

@@ -54,8 +49,14 @@ I/O
5449

5550
Other
5651
~~~~~
52+
53+
- The deprecated attributes ``_AXIS_NAMES`` and ``_AXIS_NUMBERS`` of :class:`DataFrame` and :class:`Series` will no longer show up in ``dir`` or ``inspect.getmembers`` calls (:issue:`38740`)
54+
- Bumped minimum fastparquet version to 0.4.0 to avoid ``AttributeError`` from numba (:issue:`38344`)
55+
- Bumped minimum pymysql version to 0.8.1 to avoid test failures (:issue:`38344`)
5756
- Fixed build failure on MacOS 11 in Python 3.9.1 (:issue:`38766`)
5857
- Added reference to backwards incompatible ``check_freq`` arg of :func:`testing.assert_frame_equal` and :func:`testing.assert_series_equal` in :ref:`pandas 1.1.0 whats new <whatsnew_110.api_breaking.testing.check_freq>` (:issue:`34050`)
58+
-
59+
-
5960

6061
.. ---------------------------------------------------------------------------
6162

doc/source/whatsnew/v1.3.0.rst

+2-1
Original file line numberDiff line numberDiff line change
@@ -247,10 +247,11 @@ Indexing
247247
- Bug in :meth:`CategoricalIndex.get_indexer` failing to raise ``InvalidIndexError`` when non-unique (:issue:`38372`)
248248
- Bug in inserting many new columns into a :class:`DataFrame` causing incorrect subsequent indexing behavior (:issue:`38380`)
249249
- Bug in :meth:`DataFrame.loc`, :meth:`Series.loc`, :meth:`DataFrame.__getitem__` and :meth:`Series.__getitem__` returning incorrect elements for non-monotonic :class:`DatetimeIndex` for string slices (:issue:`33146`)
250+
- Bug in :meth:`DataFrame.reindex` and :meth:`Series.reindex` with timezone aware indexes raising ``TypeError`` for ``method="ffill"`` and ``method="bfill"`` and specified ``tolerance`` (:issue:`38566`)
250251
- Bug in :meth:`DataFrame.__setitem__` raising ``ValueError`` with empty :class:`DataFrame` and specified columns for string indexer and non empty :class:`DataFrame` to set (:issue:`38831`)
251252
- Bug in :meth:`DataFrame.iloc.__setitem__` and :meth:`DataFrame.loc.__setitem__` with mixed dtypes when setting with a dictionary value (:issue:`38335`)
252253
- Bug in :meth:`DataFrame.loc` dropping levels of :class:`MultiIndex` when :class:`DataFrame` used as input has only one row (:issue:`10521`)
253-
-
254+
- Bug in setting ``timedelta64`` values into numeric :class:`Series` failing to cast to object dtype (:issue:`39086`)
254255

255256
Missing
256257
^^^^^^^

pandas/_typing.py

+7-8
Original file line numberDiff line numberDiff line change
@@ -84,9 +84,8 @@
8484
FrameOrSeries = TypeVar("FrameOrSeries", bound="NDFrame")
8585

8686
Axis = Union[str, int]
87-
Label = Optional[Hashable]
88-
IndexLabel = Union[Label, Sequence[Label]]
89-
Level = Union[Label, int]
87+
IndexLabel = Union[Hashable, Sequence[Hashable]]
88+
Level = Union[Hashable, int]
9089
Shape = Tuple[int, ...]
9190
Suffixes = Tuple[str, str]
9291
Ordered = Optional[bool]
@@ -99,11 +98,11 @@
9998
"ExtensionDtype", NpDtype, Type[Union[str, float, int, complex, bool, object]]
10099
]
101100
# DtypeArg specifies all allowable dtypes in a functions its dtype argument
102-
DtypeArg = Union[Dtype, Dict[Label, Dtype]]
101+
DtypeArg = Union[Dtype, Dict[Hashable, Dtype]]
103102
DtypeObj = Union[np.dtype, "ExtensionDtype"]
104103

105104
# For functions like rename that convert one label to another
106-
Renamer = Union[Mapping[Label, Any], Callable[[Label], Label]]
105+
Renamer = Union[Mapping[Hashable, Any], Callable[[Hashable], Hashable]]
107106

108107
# to maintain type information across generic functions and parametrization
109108
T = TypeVar("T")
@@ -120,7 +119,7 @@
120119

121120
# types of `func` kwarg for DataFrame.aggregate and Series.aggregate
122121
AggFuncTypeBase = Union[Callable, str]
123-
AggFuncTypeDict = Dict[Label, Union[AggFuncTypeBase, List[AggFuncTypeBase]]]
122+
AggFuncTypeDict = Dict[Hashable, Union[AggFuncTypeBase, List[AggFuncTypeBase]]]
124123
AggFuncType = Union[
125124
AggFuncTypeBase,
126125
List[AggFuncTypeBase],
@@ -155,8 +154,8 @@
155154
FormattersType = Union[
156155
List[Callable], Tuple[Callable, ...], Mapping[Union[str, int], Callable]
157156
]
158-
ColspaceType = Mapping[Label, Union[str, int]]
157+
ColspaceType = Mapping[Hashable, Union[str, int]]
159158
FloatFormatType = Union[str, Callable, "EngFormatter"]
160159
ColspaceArgType = Union[
161-
str, int, Sequence[Union[str, int]], Mapping[Label, Union[str, int]]
160+
str, int, Sequence[Union[str, int]], Mapping[Hashable, Union[str, int]]
162161
]

pandas/compat/_optional.py

+15-16
Original file line numberDiff line numberDiff line change
@@ -62,8 +62,7 @@ def _get_version(module: types.ModuleType) -> str:
6262
def import_optional_dependency(
6363
name: str,
6464
extra: str = "",
65-
raise_on_missing: bool = True,
66-
on_version: str = "raise",
65+
errors: str = "raise",
6766
min_version: Optional[str] = None,
6867
):
6968
"""
@@ -79,29 +78,30 @@ def import_optional_dependency(
7978
The module name.
8079
extra : str
8180
Additional text to include in the ImportError message.
82-
raise_on_missing : bool, default True
83-
Whether to raise if the optional dependency is not found.
84-
When False and the module is not present, None is returned.
85-
on_version : str {'raise', 'warn'}
86-
What to do when a dependency's version is too old.
81+
errors : str {'raise', 'warn', 'ignore'}
82+
What to do when a dependency is not found or its version is too old.
8783
8884
* raise : Raise an ImportError
89-
* warn : Warn that the version is too old. Returns None
90-
* ignore: Return the module, even if the version is too old.
85+
* warn : Only applicable when a module's version is to old.
86+
Warns that the version is too old and returns None
87+
* ignore: If the module is not installed, return None, otherwise,
88+
return the module, even if the version is too old.
9189
It's expected that users validate the version locally when
92-
using ``on_version="ignore"`` (see. ``io/html.py``)
90+
using ``errors="ignore"`` (see. ``io/html.py``)
9391
min_version : str, default None
9492
Specify a minimum version that is different from the global pandas
9593
minimum version required.
9694
Returns
9795
-------
9896
maybe_module : Optional[ModuleType]
9997
The imported module, when found and the version is correct.
100-
None is returned when the package is not found and `raise_on_missing`
101-
is False, or when the package's version is too old and `on_version`
98+
None is returned when the package is not found and `errors`
99+
is False, or when the package's version is too old and `errors`
102100
is ``'warn'``.
103101
"""
104102

103+
assert errors in {"warn", "raise", "ignore"}
104+
105105
package_name = INSTALL_MAPPING.get(name)
106106
install_name = package_name if package_name is not None else name
107107

@@ -112,7 +112,7 @@ def import_optional_dependency(
112112
try:
113113
module = importlib.import_module(name)
114114
except ImportError:
115-
if raise_on_missing:
115+
if errors == "raise":
116116
raise ImportError(msg) from None
117117
else:
118118
return None
@@ -128,15 +128,14 @@ def import_optional_dependency(
128128
if minimum_version:
129129
version = _get_version(module_to_get)
130130
if distutils.version.LooseVersion(version) < minimum_version:
131-
assert on_version in {"warn", "raise", "ignore"}
132131
msg = (
133132
f"Pandas requires version '{minimum_version}' or newer of '{parent}' "
134133
f"(version '{version}' currently installed)."
135134
)
136-
if on_version == "warn":
135+
if errors == "warn":
137136
warnings.warn(msg, UserWarning)
138137
return None
139-
elif on_version == "raise":
138+
elif errors == "raise":
140139
raise ImportError(msg)
141140

142141
return module

pandas/core/aggregation.py

+5-5
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
Callable,
1212
DefaultDict,
1313
Dict,
14+
Hashable,
1415
Iterable,
1516
List,
1617
Optional,
@@ -28,7 +29,6 @@
2829
Axis,
2930
FrameOrSeries,
3031
FrameOrSeriesUnion,
31-
Label,
3232
)
3333

3434
from pandas.core.dtypes.cast import is_nested_object
@@ -294,9 +294,9 @@ def maybe_mangle_lambdas(agg_spec: Any) -> Any:
294294
def relabel_result(
295295
result: FrameOrSeries,
296296
func: Dict[str, List[Union[Callable, str]]],
297-
columns: Iterable[Label],
297+
columns: Iterable[Hashable],
298298
order: Iterable[int],
299-
) -> Dict[Label, "Series"]:
299+
) -> Dict[Hashable, "Series"]:
300300
"""
301301
Internal function to reorder result if relabelling is True for
302302
dataframe.agg, and return the reordered result in dict.
@@ -323,7 +323,7 @@ def relabel_result(
323323
reordered_indexes = [
324324
pair[0] for pair in sorted(zip(columns, order), key=lambda t: t[1])
325325
]
326-
reordered_result_in_dict: Dict[Label, "Series"] = {}
326+
reordered_result_in_dict: Dict[Hashable, "Series"] = {}
327327
idx = 0
328328

329329
reorder_mask = not isinstance(result, ABCSeries) and len(result.columns) > 1
@@ -493,7 +493,7 @@ def transform_dict_like(
493493
# GH 15931 - deprecation of renaming keys
494494
raise SpecificationError("nested renamer is not supported")
495495

496-
results: Dict[Label, FrameOrSeriesUnion] = {}
496+
results: Dict[Hashable, FrameOrSeriesUnion] = {}
497497
for name, how in func.items():
498498
colg = obj._gotitem(name, ndim=1)
499499
try:

pandas/core/apply.py

-21
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,6 @@
4040

4141
def frame_apply(
4242
obj: DataFrame,
43-
how: str,
4443
func: AggFuncType,
4544
axis: Axis = 0,
4645
raw: bool = False,
@@ -58,7 +57,6 @@ def frame_apply(
5857

5958
return klass(
6059
obj,
61-
how,
6260
func,
6361
raw=raw,
6462
result_type=result_type,
@@ -69,15 +67,13 @@ def frame_apply(
6967

7068
def series_apply(
7169
obj: Series,
72-
how: str,
7370
func: AggFuncType,
7471
convert_dtype: bool = True,
7572
args=None,
7673
kwds=None,
7774
) -> SeriesApply:
7875
return SeriesApply(
7976
obj,
80-
how,
8177
func,
8278
convert_dtype,
8379
args,
@@ -91,16 +87,13 @@ class Apply(metaclass=abc.ABCMeta):
9187
def __init__(
9288
self,
9389
obj: FrameOrSeriesUnion,
94-
how: str,
9590
func,
9691
raw: bool,
9792
result_type: Optional[str],
9893
args,
9994
kwds,
10095
):
101-
assert how in ("apply", "agg")
10296
self.obj = obj
103-
self.how = how
10497
self.raw = raw
10598
self.args = args or ()
10699
self.kwds = kwds or {}
@@ -132,12 +125,6 @@ def f(x):
132125
def index(self) -> Index:
133126
return self.obj.index
134127

135-
def get_result(self):
136-
if self.how == "apply":
137-
return self.apply()
138-
else:
139-
return self.agg()
140-
141128
@abc.abstractmethod
142129
def apply(self) -> FrameOrSeriesUnion:
143130
pass
@@ -234,12 +221,6 @@ def dtypes(self) -> Series:
234221
def agg_axis(self) -> Index:
235222
return self.obj._get_agg_axis(self.axis)
236223

237-
def get_result(self):
238-
if self.how == "apply":
239-
return self.apply()
240-
else:
241-
return self.agg()
242-
243224
def apply(self) -> FrameOrSeriesUnion:
244225
""" compute the results """
245226
# dispatch to agg
@@ -570,7 +551,6 @@ class SeriesApply(Apply):
570551
def __init__(
571552
self,
572553
obj: Series,
573-
how: str,
574554
func: AggFuncType,
575555
convert_dtype: bool,
576556
args,
@@ -580,7 +560,6 @@ def __init__(
580560

581561
super().__init__(
582562
obj,
583-
how,
584563
func,
585564
raw=False,
586565
result_type=None,

0 commit comments

Comments
 (0)