Skip to content

Commit ac9f79a

Browse files
author
HH
committed
update
2 parents 66b8e42 + def01cf commit ac9f79a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+1437
-904
lines changed

ci/deps/azure-macos-35.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ dependencies:
2222
- xlrd
2323
- xlsxwriter
2424
- xlwt
25+
- pip
2526
- pip:
2627
- pyreadstat
2728
# universal

ci/run_tests.sh

+6-5
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,10 @@ do
5050
# if no tests are found (the case of "single and slow"), pytest exits with code 5, and would make the script fail, if not for the below code
5151
sh -c "$PYTEST_CMD; ret=\$?; [ \$ret = 5 ] && exit 0 || exit \$ret"
5252

53-
if [[ "$COVERAGE" && $? == 0 ]]; then
54-
echo "uploading coverage for $TYPE tests"
55-
echo "bash <(curl -s https://codecov.io/bash) -Z -c -F $TYPE -f $COVERAGE_FNAME"
56-
bash <(curl -s https://codecov.io/bash) -Z -c -F $TYPE -f $COVERAGE_FNAME
57-
fi
53+
# 2019-08-21 disabling because this is hitting HTTP 400 errors GH#27602
54+
# if [[ "$COVERAGE" && $? == 0 && "$TRAVIS_BRANCH" == "master" ]]; then
55+
# echo "uploading coverage for $TYPE tests"
56+
# echo "bash <(curl -s https://codecov.io/bash) -Z -c -F $TYPE -f $COVERAGE_FNAME"
57+
# bash <(curl -s https://codecov.io/bash) -Z -c -F $TYPE -f $COVERAGE_FNAME
58+
# fi
5859
done

doc/source/user_guide/enhancingperf.rst

+3-3
Original file line numberDiff line numberDiff line change
@@ -243,9 +243,9 @@ We've gotten another big improvement. Let's check again where the time is spent:
243243

244244
.. ipython:: python
245245
246-
%prun -l 4 apply_integrate_f(df['a'].to_numpy(),
247-
df['b'].to_numpy(),
248-
df['N'].to_numpy())
246+
%%prun -l 4 apply_integrate_f(df['a'].to_numpy(),
247+
df['b'].to_numpy(),
248+
df['N'].to_numpy())
249249
250250
As one might expect, the majority of the time is now spent in ``apply_integrate_f``,
251251
so if we wanted to make anymore efficiencies we must continue to concentrate our

doc/source/user_guide/io.rst

+8-6
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ The pandas I/O API is a set of top level ``reader`` functions accessed like
2828
:delim: ;
2929

3030
text;`CSV <https://en.wikipedia.org/wiki/Comma-separated_values>`__;:ref:`read_csv<io.read_csv_table>`;:ref:`to_csv<io.store_in_csv>`
31+
text;Fixed-Width Text File;:ref:`read_fwf<io.fwf_reader>`
3132
text;`JSON <https://www.json.org/>`__;:ref:`read_json<io.json_reader>`;:ref:`to_json<io.json_writer>`
3233
text;`HTML <https://en.wikipedia.org/wiki/HTML>`__;:ref:`read_html<io.read_html>`;:ref:`to_html<io.html>`
3334
text; Local clipboard;:ref:`read_clipboard<io.clipboard>`;:ref:`to_clipboard<io.clipboard>`
@@ -1372,6 +1373,7 @@ should pass the ``escapechar`` option:
13721373
print(data)
13731374
pd.read_csv(StringIO(data), escapechar='\\')
13741375
1376+
.. _io.fwf_reader:
13751377
.. _io.fwf:
13761378

13771379
Files with fixed width columns
@@ -3572,7 +3574,7 @@ Closing a Store and using a context manager:
35723574
Read/write API
35733575
''''''''''''''
35743576

3575-
``HDFStore`` supports an top-level API using ``read_hdf`` for reading and ``to_hdf`` for writing,
3577+
``HDFStore`` supports a top-level API using ``read_hdf`` for reading and ``to_hdf`` for writing,
35763578
similar to how ``read_csv`` and ``to_csv`` work.
35773579

35783580
.. ipython:: python
@@ -3687,7 +3689,7 @@ Hierarchical keys
36873689
Keys to a store can be specified as a string. These can be in a
36883690
hierarchical path-name like format (e.g. ``foo/bar/bah``), which will
36893691
generate a hierarchy of sub-stores (or ``Groups`` in PyTables
3690-
parlance). Keys can be specified with out the leading '/' and are **always**
3692+
parlance). Keys can be specified without the leading '/' and are **always**
36913693
absolute (e.g. 'foo' refers to '/foo'). Removal operations can remove
36923694
everything in the sub-store and **below**, so be *careful*.
36933695

@@ -3825,7 +3827,7 @@ data.
38253827

38263828
A query is specified using the ``Term`` class under the hood, as a boolean expression.
38273829

3828-
* ``index`` and ``columns`` are supported indexers of a ``DataFrames``.
3830+
* ``index`` and ``columns`` are supported indexers of ``DataFrames``.
38293831
* if ``data_columns`` are specified, these can be used as additional indexers.
38303832

38313833
Valid comparison operators are:
@@ -3917,7 +3919,7 @@ Use boolean expressions, with in-line function evaluation.
39173919
39183920
store.select('dfq', "index>pd.Timestamp('20130104') & columns=['A', 'B']")
39193921
3920-
Use and inline column reference
3922+
Use inline column reference.
39213923

39223924
.. ipython:: python
39233925
@@ -4593,8 +4595,8 @@ Performance
45934595
write chunksize (default is 50000). This will significantly lower
45944596
your memory usage on writing.
45954597
* You can pass ``expectedrows=<int>`` to the first ``append``,
4596-
to set the TOTAL number of expected rows that ``PyTables`` will
4597-
expected. This will optimize read/write performance.
4598+
to set the TOTAL number of rows that ``PyTables`` will expect.
4599+
This will optimize read/write performance.
45984600
* Duplicate rows can be written to tables, but are filtered out in
45994601
selection (with the last items being selected; thus a table is
46004602
unique on major, minor pairs)

doc/source/whatsnew/v0.25.1.rst

+33-82
Original file line numberDiff line numberDiff line change
@@ -1,164 +1,115 @@
11
.. _whatsnew_0251:
22

3-
What's new in 0.25.1 (July XX, 2019)
4-
------------------------------------
5-
6-
Enhancements
7-
~~~~~~~~~~~~
3+
What's new in 0.25.1 (August 21, 2019)
4+
--------------------------------------
85

6+
These are the changes in pandas 0.25.1. See :ref:`release` for a full changelog
7+
including other versions of pandas.
98

10-
.. _whatsnew_0251.enhancements.other:
11-
12-
Other enhancements
13-
^^^^^^^^^^^^^^^^^^
9+
I/O and LZMA
10+
~~~~~~~~~~~~
1411

15-
-
16-
-
17-
-
12+
Some users may unknowingly have an incomplete Python installation lacking the `lzma` module from the standard library. In this case, `import pandas` failed due to an `ImportError` (:issue: `27575`).
13+
Pandas will now warn, rather than raising an `ImportError` if the `lzma` module is not present. Any subsequent attempt to use `lzma` methods will raise a `RuntimeError`.
14+
A possible fix for the lack of the `lzma` module is to ensure you have the necessary libraries and then re-install Python.
15+
For example, on MacOS installing Python with `pyenv` may lead to an incomplete Python installation due to unmet system dependencies at compilation time (like `xz`). Compilation will succeed, but Python might fail at run time. The issue can be solved by installing the necessary dependencies and then re-installing Python.
1816

1917
.. _whatsnew_0251.bug_fixes:
2018

2119
Bug fixes
2220
~~~~~~~~~
2321

24-
2522
Categorical
2623
^^^^^^^^^^^
2724

28-
-
29-
-
30-
-
25+
- Bug in :meth:`Categorical.fillna` that would replace all values, not just those that are ``NaN`` (:issue:`26215`)
3126

3227
Datetimelike
3328
^^^^^^^^^^^^
34-
- Bug in :func:`to_datetime` where passing a timezone-naive :class:`DatetimeArray` or :class:`DatetimeIndex` and ``utc=True`` would incorrectly return a timezone-naive result (:issue:`27733`)
35-
-
36-
-
37-
-
3829

39-
Timedelta
40-
^^^^^^^^^
41-
42-
-
43-
-
44-
-
30+
- Bug in :func:`to_datetime` where passing a timezone-naive :class:`DatetimeArray` or :class:`DatetimeIndex` and ``utc=True`` would incorrectly return a timezone-naive result (:issue:`27733`)
31+
- Bug in :meth:`Period.to_timestamp` where a :class:`Period` outside the :class:`Timestamp` implementation bounds (roughly 1677-09-21 to 2262-04-11) would return an incorrect :class:`Timestamp` instead of raising ``OutOfBoundsDatetime`` (:issue:`19643`)
32+
- Bug in iterating over :class:`DatetimeIndex` when the underlying data is read-only (:issue:`28055`)
4533

4634
Timezones
4735
^^^^^^^^^
4836

4937
- Bug in :class:`Index` where a numpy object array with a timezone aware :class:`Timestamp` and ``np.nan`` would not return a :class:`DatetimeIndex` (:issue:`27011`)
50-
-
51-
-
5238

5339
Numeric
5440
^^^^^^^
41+
5542
- Bug in :meth:`Series.interpolate` when using a timezone aware :class:`DatetimeIndex` (:issue:`27548`)
5643
- Bug when printing negative floating point complex numbers would raise an ``IndexError`` (:issue:`27484`)
57-
-
58-
-
44+
- Bug where :class:`DataFrame` arithmetic operators such as :meth:`DataFrame.mul` with a :class:`Series` with axis=1 would raise an ``AttributeError`` on :class:`DataFrame` larger than the minimum threshold to invoke numexpr (:issue:`27636`)
45+
- Bug in :class:`DataFrame` arithmetic where missing values in results were incorrectly masked with ``NaN`` instead of ``Inf`` (:issue:`27464`)
5946

6047
Conversion
6148
^^^^^^^^^^
6249

6350
- Improved the warnings for the deprecated methods :meth:`Series.real` and :meth:`Series.imag` (:issue:`27610`)
64-
-
65-
-
66-
67-
Strings
68-
^^^^^^^
69-
70-
-
71-
-
72-
-
73-
7451

7552
Interval
7653
^^^^^^^^
54+
7755
- Bug in :class:`IntervalIndex` where `dir(obj)` would raise ``ValueError`` (:issue:`27571`)
78-
-
79-
-
80-
-
8156

8257
Indexing
8358
^^^^^^^^
8459

8560
- Bug in partial-string indexing returning a NumPy array rather than a ``Series`` when indexing with a scalar like ``.loc['2015']`` (:issue:`27516`)
8661
- Break reference cycle involving :class:`Index` and other index classes to allow garbage collection of index objects without running the GC. (:issue:`27585`, :issue:`27840`)
8762
- Fix regression in assigning values to a single column of a DataFrame with a ``MultiIndex`` columns (:issue:`27841`).
88-
-
63+
- Fix regression in ``.ix`` fallback with an ``IntervalIndex`` (:issue:`27865`).
8964

9065
Missing
9166
^^^^^^^
9267

93-
-
94-
-
95-
-
96-
97-
MultiIndex
98-
^^^^^^^^^^
99-
100-
-
101-
-
102-
-
68+
- Bug in :func:`pandas.isnull` or :func:`pandas.isna` when the input is a type e.g. ``type(pandas.Series())`` (:issue:`27482`)
10369

10470
I/O
10571
^^^
10672

10773
- Avoid calling ``S3File.s3`` when reading parquet, as this was removed in s3fs version 0.3.0 (:issue:`27756`)
108-
-
109-
-
74+
- Better error message when a negative header is passed in :func:`pandas.read_csv` (:issue:`27779`)
75+
- Follow the ``min_rows`` display option (introduced in v0.25.0) correctly in the HTML repr in the notebook (:issue:`27991`).
11076

11177
Plotting
11278
^^^^^^^^
11379

114-
- Added a pandas_plotting_backends entrypoint group for registering plot backends. See :ref:`extending.plotting-backends` for more (:issue:`26747`).
80+
- Added a ``pandas_plotting_backends`` entrypoint group for registering plot backends. See :ref:`extending.plotting-backends` for more (:issue:`26747`).
81+
- Fixed the re-instatement of Matplotlib datetime converters after calling
82+
:meth:`pandas.plotting.deregister_matplotlib_converters` (:issue:`27481`).
11583
- Fix compatibility issue with matplotlib when passing a pandas ``Index`` to a plot call (:issue:`27775`).
116-
-
11784

11885
Groupby/resample/rolling
11986
^^^^^^^^^^^^^^^^^^^^^^^^
12087

88+
- Fixed regression in :meth:`pands.core.groupby.DataFrameGroupBy.quantile` raising when multiple quantiles are given (:issue:`27526`)
12189
- Bug in :meth:`pandas.core.groupby.DataFrameGroupBy.transform` where applying a timezone conversion lambda function would drop timezone information (:issue:`27496`)
90+
- Bug in :meth:`pandas.core.groupby.GroupBy.nth` where ``observed=False`` was being ignored for Categorical groupers (:issue:`26385`)
12291
- Bug in windowing over read-only arrays (:issue:`27766`)
12392
- Fixed segfault in `pandas.core.groupby.DataFrameGroupBy.quantile` when an invalid quantile was passed (:issue:`27470`)
124-
-
12593

12694
Reshaping
12795
^^^^^^^^^
12896

12997
- A ``KeyError`` is now raised if ``.unstack()`` is called on a :class:`Series` or :class:`DataFrame` with a flat :class:`Index` passing a name which is not the correct one (:issue:`18303`)
130-
- Bug in :meth:`DataFrame.crosstab` when ``margins`` set to ``True`` and ``normalize`` is not ``False``, an error is raised. (:issue:`27500`)
98+
- Bug :meth:`merge_asof` could not merge :class:`Timedelta` objects when passing `tolerance` kwarg (:issue:`27642`)
99+
- Bug in :meth:`DataFrame.crosstab` when ``margins`` set to ``True`` and ``normalize`` is not ``False``, an error is raised. (:issue:`27500`)
131100
- :meth:`DataFrame.join` now suppresses the ``FutureWarning`` when the sort parameter is specified (:issue:`21952`)
132-
-
101+
- Bug in :meth:`DataFrame.join` raising with readonly arrays (:issue:`27943`)
133102

134103
Sparse
135104
^^^^^^
136-
- Bug in reductions for :class:`Series` with Sparse dtypes (:issue:`27080`)
137-
-
138-
-
139-
-
140105

141-
142-
Build Changes
143-
^^^^^^^^^^^^^
144-
145-
-
146-
-
147-
-
148-
149-
ExtensionArray
150-
^^^^^^^^^^^^^^
151-
152-
-
153-
-
154-
-
106+
- Bug in reductions for :class:`Series` with Sparse dtypes (:issue:`27080`)
155107

156108
Other
157109
^^^^^
110+
158111
- Bug in :meth:`Series.replace` and :meth:`DataFrame.replace` when replacing timezone-aware timestamps using a dict-like replacer (:issue:`27720`)
159-
-
160-
-
161-
-
112+
- Bug in :meth:`Series.rename` when using a custom type indexer. Now any value that isn't callable or dict-like is treated as a scalar. (:issue:`27814`)
162113

163114
.. _whatsnew_0.251.contributors:
164115

doc/source/whatsnew/v0.7.3.rst

-6
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,6 @@ New features
2525
from pandas.tools.plotting import scatter_matrix
2626
scatter_matrix(df, alpha=0.2) # noqa F821
2727
28-
.. image:: ../savefig/scatter_matrix_kde.png
29-
:width: 5in
3028
3129
- Add ``stacked`` argument to Series and DataFrame's ``plot`` method for
3230
:ref:`stacked bar plots <visualization.barplot>`.
@@ -35,15 +33,11 @@ New features
3533
3634
df.plot(kind='bar', stacked=True) # noqa F821
3735
38-
.. image:: ../savefig/bar_plot_stacked_ex.png
39-
:width: 4in
4036
4137
.. code-block:: python
4238
4339
df.plot(kind='barh', stacked=True) # noqa F821
4440
45-
.. image:: ../savefig/barh_plot_stacked_ex.png
46-
:width: 4in
4741
4842
- Add log x and y :ref:`scaling options <visualization.basic>` to
4943
``DataFrame.plot`` and ``Series.plot``

doc/source/whatsnew/v1.0.0.rst

+4-1
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ Bug fixes
8787
Categorical
8888
^^^^^^^^^^^
8989

90+
- Added test to assert the :func:`fillna` raises the correct ValueError message when the value isn't a value from categories (:issue:`13628`)
9091
-
9192
-
9293

@@ -157,15 +158,17 @@ MultiIndex
157158
I/O
158159
^^^
159160

160-
-
161+
- :meth:`read_csv` now accepts binary mode file buffers when using the Python csv engine (:issue:`23779`)
161162
-
162163

163164
Plotting
164165
^^^^^^^^
165166

166167
- Bug in :meth:`Series.plot` not able to plot boolean values (:issue:`23719`)
167168
-
169+
- Bug in :meth:`DataFrame.plot` producing incorrect legend markers when plotting multiple series on the same axis (:issue:`18222`)
168170
- Bug in :meth:`DataFrame.plot` when ``kind='box'`` and data contains datetime or timedelta data. These types are now automatically dropped (:issue:`22799`)
171+
- Bug in :meth:`DataFrame.plot.line` and :meth:`DataFrame.plot.area` produce wrong xlim in x-axis (:issue:`27686`, :issue:`25160`, :issue:`24784`)
169172

170173
Groupby/resample/rolling
171174
^^^^^^^^^^^^^^^^^^^^^^^^

pandas/_libs/hashtable.pyx

+1-1
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ cdef class Int64Factorizer:
108108
def get_count(self):
109109
return self.count
110110

111-
def factorize(self, int64_t[:] values, sort=False,
111+
def factorize(self, const int64_t[:] values, sort=False,
112112
na_sentinel=-1, na_value=None):
113113
"""
114114
Factorize values with nans replaced by na_sentinel

pandas/_libs/parsers.pyx

+5-3
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
# See LICENSE for the license
33
import bz2
44
import gzip
5-
import lzma
65
import os
76
import sys
87
import time
@@ -59,9 +58,12 @@ from pandas.core.arrays import Categorical
5958
from pandas.core.dtypes.concat import union_categoricals
6059
import pandas.io.common as icom
6160

61+
from pandas.compat import _import_lzma, _get_lzma_file
6262
from pandas.errors import (ParserError, DtypeWarning,
6363
EmptyDataError, ParserWarning)
6464

65+
lzma = _import_lzma()
66+
6567
# Import CParserError as alias of ParserError for backwards compatibility.
6668
# Ultimately, we want to remove this import. See gh-12665 and gh-14479.
6769
CParserError = ParserError
@@ -645,9 +647,9 @@ cdef class TextReader:
645647
'zip file %s', str(zip_names))
646648
elif self.compression == 'xz':
647649
if isinstance(source, str):
648-
source = lzma.LZMAFile(source, 'rb')
650+
source = _get_lzma_file(lzma)(source, 'rb')
649651
else:
650-
source = lzma.LZMAFile(filename=source)
652+
source = _get_lzma_file(lzma)(filename=source)
651653
else:
652654
raise ValueError('Unrecognized compression type: %s' %
653655
self.compression)

0 commit comments

Comments
 (0)