These are the changes in pandas 2.1.0. See :ref:`release` for a full changelog including other versions of pandas.
{{ header }}
- Implemented
__pandas_priority__to allow custom types to take precedence over :class:`DataFrame`, :class:`Series`, :class:`Index`, or :class:`ExtensionArray` for arithmetic operations, :ref:`see the developer guide <extending.pandas_priority>` (:issue:`48347`) - :meth:`MultiIndex.sort_values` now supports
na_position(:issue:`51612`) - :meth:`MultiIndex.sortlevel` and :meth:`Index.sortlevel` gained a new keyword
na_position(:issue:`51612`) - Improve error message when setting :class:`DataFrame` with wrong number of columns through :meth:`DataFrame.isetitem` (:issue:`51701`)
- Let :meth:`DataFrame.to_feather` accept a non-default :class:`Index` and non-string column names (:issue:`51787`)
- :class:`api.extensions.ExtensionArray` now has a :meth:`~api.extensions.ExtensionArray.map` method (:issue:`51809`)
- Improve error message when having incompatible columns using :meth:`DataFrame.merge` (:issue:`51861`)
- Improved error message when creating a DataFrame with empty data (0 rows), no index and an incorrect number of columns. (:issue:`52084`)
- :meth:`arrays.SparseArray.map` now supports
na_action(:issue:`52096`).
These are bug fixes that might have notable behavior changes.
Some minimum supported versions of dependencies were updated. If installed, we now require:
| Package | Minimum Version | Required | Changed |
|---|---|---|---|
| X | X |
For optional libraries the general recommendation is to use the latest version. The following table lists the lowest version per library that is currently being tested throughout the development of pandas. Optional libraries below the lowest tested version may still work, but are not considered supported.
| Package | Minimum Version | Changed |
|---|---|---|
| X |
See :ref:`install.dependencies` and :ref:`install.optional_dependencies` for more.
- Deprecated silently dropping unrecognized timezones when parsing strings to datetimes (:issue:`18702`)
- Deprecated :meth:`DataFrame._data` and :meth:`Series._data`, use public APIs instead (:issue:`33333`)
- Deprecating pinning
group.nameto each group in :meth:`SeriesGroupBy.aggregate` aggregations; if your operation requires utilizing the groupby keys, iterate over the groupby object instead (:issue:`41090`) - Deprecated the default of
observed=Falsein :meth:`DataFrame.groupby` and :meth:`Series.groupby`; this will default toTruein a future version (:issue:`43999`) - Deprecated :meth:`DataFrameGroupBy.dtypes`, check
dtypeson the underlying object instead (:issue:`51045`) - Deprecated
axis=1in :meth:`DataFrame.groupby` and in :class:`Grouper` constructor, doframe.T.groupby(...)instead (:issue:`51203`) - Deprecated :meth:`Categorical.to_list`, use
obj.tolist()instead (:issue:`51254`) - Deprecated passing a :class:`DataFrame` to :meth:`DataFrame.from_records`, use :meth:`DataFrame.set_index` or :meth:`DataFrame.drop` instead (:issue:`51353`)
- Deprecated accepting slices in :meth:`DataFrame.take`, call
obj[slicer]or pass a sequence of integers instead (:issue:`51539`) - Deprecated
axis=1in :meth:`DataFrame.ewm`, :meth:`DataFrame.rolling`, :meth:`DataFrame.expanding`, transpose before calling the method instead (:issue:`51778`) - Deprecated the
axiskeyword in :meth:`DataFrame.ewm`, :meth:`Series.ewm`, :meth:`DataFrame.rolling`, :meth:`Series.rolling`, :meth:`DataFrame.expanding`, :meth:`Series.expanding` (:issue:`51778`) - Deprecated 'method', 'limit', and 'fill_axis' keywords in :meth:`DataFrame.align` and :meth:`Series.align`, explicitly call
fillnaon the alignment results instead (:issue:`51856`) - Deprecated 'broadcast_axis' keyword in :meth:`Series.align` and :meth:`DataFrame.align`, upcast before calling
alignwithleft = DataFrame({col: left for col in right.columns}, index=right.index)(:issue:`51856`) - Deprecated the 'axis' keyword in :meth:`.GroupBy.idxmax`, :meth:`.GroupBy.idxmin`, :meth:`.GroupBy.fillna`, :meth:`.GroupBy.take`, :meth:`.GroupBy.skew`, :meth:`.GroupBy.rank`, :meth:`.GroupBy.cumprod`, :meth:`.GroupBy.cumsum`, :meth:`.GroupBy.cummax`, :meth:`.GroupBy.cummin`, :meth:`.GroupBy.pct_change`, :meth:`GroupBy.diff`, :meth:`.GroupBy.shift`, and :meth:`DataFrameGroupBy.corrwith`; for
axis=1operate on the underlying :class:`DataFrame` instead (:issue:`50405`, :issue:`51046`)
- Performance improvement in :func:`read_parquet` on string columns when using
use_nullable_dtypes=True(:issue:`47345`) - Performance improvement in :meth:`DataFrame.clip` and :meth:`Series.clip` (:issue:`51472`)
- Performance improvement in :meth:`DataFrame.first_valid_index` and :meth:`DataFrame.last_valid_index` for extension array dtypes (:issue:`51549`)
- Performance improvement in :meth:`DataFrame.where` when
condis backed by an extension dtype (:issue:`51574`) - Performance improvement in :func:`read_orc` when reading a remote URI file path. (:issue:`51609`)
- Performance improvement in :func:`read_parquet` and :meth:`DataFrame.to_parquet` when reading a remote file with
engine="pyarrow"(:issue:`51609`) - Performance improvement in :meth:`MultiIndex.sortlevel` when
ascendingis a list (:issue:`51612`) - Performance improvement in :meth:`~arrays.ArrowExtensionArray.isna` when array has zero nulls or is all nulls (:issue:`51630`)
- Performance improvement in :meth:`~arrays.ArrowExtensionArray.fillna` when array does not contain nulls (:issue:`51635`)
- Performance improvement when parsing strings to
boolean[pyarrow]dtype (:issue:`51730`) - Performance improvement when searching an :class:`Index` sliced from other indexes (:issue:`51738`)
- Performance improvement in :meth:`Series.combine_first` (:issue:`51777`)
- Performance improvement in :meth:`MultiIndex.set_levels` and :meth:`MultiIndex.set_codes` when
verify_integrity=True(:issue:`51873`) - Performance improvement in :func:`factorize` for object columns not containing strings (:issue:`51921`)
- Bug in :meth:`Timestamp.round` with values close to the implementation bounds returning incorrect results instead of raising
OutOfBoundsDatetime(:issue:`51494`) - :meth:`arrays.DatetimeArray.map` can now take a
na_actionargument. :meth:`DatetimeIndex.map` withna_action="ignore"now works as expected. (:issue:`51644`) - Bug in :meth:`arrays.DatetimeArray.map` and :meth:`DatetimeIndex.map`, where the supplied callable operated array-wise instead of element-wise (:issue:`51977`)
- Bug in :meth:`Timedelta.round` with values close to the implementation bounds returning incorrect results instead of raising
OutOfBoundsTimedelta(:issue:`51494`) - Bug in :class:`TimedeltaIndex` division or multiplication leading to
.freqof "0 Days" instead ofNone(:issue:`51575`) - :meth:`arrays.TimedeltaArray.map` can now take a
na_actionargument. :meth:`TimedeltaIndex.map` withna_action="ignore"now works as expected. (:issue:`51644`) - Bug in :meth:`arrays.TimedeltaArray.map` and :meth:`TimedeltaIndex.map`, where the supplied callable operated array-wise instead of element-wise (:issue:`51977`)
- Bug in :meth:`Series.corr` and :meth:`Series.cov` raising
AttributeErrorfor masked dtypes (:issue:`51422`)
- Bug in :meth:`ArrowDtype.numpy_dtype` returning nanosecond units for non-nanosecond
pyarrow.timestampandpyarrow.durationtypes (:issue:`51800`) - Bug in :meth:`DataFrame.info` raising
ValueErrorwhenuse_numbais set (:issue:`51922`)
- :meth:`DataFrame.to_orc` now raising
ValueErrorwhen non-default :class:`Index` is given (:issue:`51828`)
- Bug in :class:`PeriodDtype` constructor failing to raise
TypeErrorwhen no argument is passed or whenNoneis passed (:issue:`27388`) - :meth:`arrays.PeriodArray.map` can now take a
na_actionargument. :meth:`PeriodIndex.map` withna_action="ignore"now works as expected. (:issue:`51644`) - Bug in :class:`PeriodDtype` constructor raising
ValueErrorinstead ofTypeErrorwhen an invalid type is passed (:issue:`51790`) - Bug in :meth:`arrays.PeriodArray.map` and :meth:`PeriodIndex.map`, where the supplied callable operated array-wise instead of element-wise (:issue:`51977`)
- Bug in :meth:`Series.plot` when invoked with
color=None(:issue:`51953`)
- Bug in :meth:`DataFrameGroupBy.idxmin`, :meth:`SeriesGroupBy.idxmin`, :meth:`DataFrameGroupBy.idxmax`, :meth:`SeriesGroupBy.idxmax` return wrong dtype when used on empty DataFrameGroupBy or SeriesGroupBy (:issue:`51423`)
- Bug in weighted rolling aggregations when specifying
min_periods=0(:issue:`51449`) - Bug in :meth:`DataFrame.resample` and :meth:`Series.resample` in incorrectly allowing non-fixed
freqwhen resampling on a :class:`TimedeltaIndex` (:issue:`51896`) - Bug in :meth:`DataFrame.groupby` and :meth:`Series.groupby`, where, when the index of the
grouped :class:`Series` or :class:`DataFrame` was a :class:`DatetimeIndex`, :class:`TimedeltaIndex`
or :class:`PeriodIndex`, and the
groupbymethod was given a function as its first argument, the function operated on the whole index rather than each element of the index. (:issue:`51979`)
- Bug in :meth:`DataFrame.stack` losing extension dtypes when columns is a :class:`MultiIndex` and frame contains mixed dtypes (:issue:`45740`)
- Bug in :meth:`DataFrame.transpose` inferring dtype for object column (:issue:`51546`)
- Bug in :meth:`Series.combine_first` converting
int64dtype tofloat64and losing precision on very large integers (:issue:`51764`)
- Bug in :meth:`arrays.SparseArray.map` allowed the fill value to be included in the sparse values (:issue:`52095`)
- Bug in :func:`assert_almost_equal` now throwing assertion error for two unequal sets (:issue:`51727`)