-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
/
Copy pathv0.11.1.txt
229 lines (171 loc) · 9.03 KB
/
v0.11.1.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
.. _whatsnew_0111:
v0.11.1 (??)
------------------------
This is a minor release from 0.11.0 and includes several new features and
enhancements along with a large number of bug fixes.
API changes
~~~~~~~~~~~
- Fix modulo and integer division on Series,DataFrames to act similary to ``float`` dtypes to return
``np.nan`` or ``np.inf`` as appropriate (GH3590_). This correct a numpy bug that treats ``integer``
and ``float`` dtypes differently.
.. ipython:: python
p = DataFrame({ 'first' : [3,4,5,8], 'second' : [0,0,0,3] })
p % 0
p % p
p / p
p / 0
- Add ``squeeze`` keyword to ``groupby`` to allow reduction from
DataFrame -> Series if groups are unique. This is a Regression from 0.10.1.
We are reverting back to the prior behavior. This means groupby will return the
same shaped objects whether the groups are unique or not. revert on (GH2893_)
with (GH3596_).
.. ipython:: python
df2 = DataFrame([{"val1": 1, "val2" : 20}, {"val1":1, "val2": 19},
{"val1":1, "val2": 27}, {"val1":1, "val2": 12}])
def func(dataf):
return dataf["val2"] - dataf["val2"].mean()
# squeezing the result frame to a series (because we have unique groups)
df2.groupby("val1", squeeze=True).apply(func)
# no squeezing (the default, and behavior in 0.10.1)
df2.groupby("val1").apply(func)
- Raise on ``iloc`` when boolean indexing with a label based indexer mask
e.g. a boolean Series, even with integer labels, will raise. Since ``iloc``
is purely positional based, the labels on the Series are not alignable (GH3631_)
This case is rarely used, and there are plently of alternatives. This preserves the
``iloc`` API to be *purely* positional based.
.. ipython:: python
df = DataFrame(range(5), list('ABCDE'), columns=['a'])
mask = (df.a%2 == 0)
mask
# this is what you should use
df.loc[mask]
# this will work as well
df.iloc[mask.values]
``df.iloc[mask]`` will raise a ``ValueError``
Enhancements
~~~~~~~~~~~~
- ``pd.read_html()`` can now parse HTML strings, files or urls and return
DataFrames, courtesy of @cpcloud. (GH3477_, GH3605_, GH3606_, GH3616_).
It works with a *single* parser backend: BeautifulSoup4 + html5lib
- You can use ``pd.read_html()`` to read the output from ``DataFrame.to_html()`` like so
.. ipython :: python
df = DataFrame({'a': range(3), 'b': list('abc')})
print df
html = df.to_html()
alist = pd.read_html(html, infer_types=True, index_col=0)
print df == alist[0]
Note that ``alist`` here is a Python ``list`` so ``pd.read_html()`` and
``DataFrame.to_html()`` are not inverses.
- ``pd.read_html()`` no longer performs hard conversion of date strings
(GH3656_).
- ``HDFStore``
- will retain index attributes (freq,tz,name) on recreation (GH3499_)
- will warn with a AttributeConflictWarning if you are attempting to append
an index with a different frequency than the existing, or attempting
to append an index with a different name than the existing
- support datelike columns with a timezone as data_columns (GH2852_)
- ``fillna`` methods now raise a ``TypeError`` if the ``value`` parameter is
a list or tuple.
- Added module for reading and writing Stata files: pandas.io.stata (GH1512_)
- ``DataFrame.replace()`` now allows regular expressions on contained
``Series`` with object dtype. See the examples section in the regular docs
:ref:`Replacing via String Expression <missing_data.replace_expression>`
- ``Series.str`` now supports iteration (GH3638_). You can iterate over the
individual elements of each string in the ``Series``. Each iteration yields
yields a ``Series`` with either a single character at each index of the
original ``Series`` or ``NaN``. For example,
.. ipython:: python
strs = 'go', 'bow', 'joe', 'slow'
ds = Series(strs)
for s in ds.str:
print s
s
s.dropna().values.item() == 'w'
The last element yielded by the iterator will be a ``Series`` containing
the last element of the longest string in the ``Series`` with all other
elements being ``NaN``. Here since ``'slow`` is the longest string
and there are no other strings with the same length ``'w'`` is the only
non-null string in the yielded ``Series``.
- Multi-index column support for reading and writing csvs
- The ``header`` option in ``read_csv`` now accepts a
list of the rows from which to read the index.
- The option, ``tupleize_cols`` can now be specified in both ``to_csv`` and
``read_csv``, to provide compatiblity for the pre 0.11.1 behavior of
writing and reading multi-index columns via a list of tuples. The default in
0.11.1 is to write lists of tuples and *not* interpret list of tuples as a
multi-index column.
Note: The default behavior in 0.11.1 remains unchanged, but starting with 0.12,
the default *to* write and read multi-index columns will be in the new
format. (GH3571_, GH1651_, GH3141_)
- If an ``index_col`` is not specified (e.g. you don't have an index, or wrote it
with ``df.to_csv(..., index=False``), then any ``names`` on the columns index will
be *lost*.
.. ipython:: python
from pandas.util.testing import makeCustomDataframe as mkdf
df = mkdf(5,3,r_idx_nlevels=2,c_idx_nlevels=4)
df.to_csv('mi.csv',tupleize_cols=False)
print open('mi.csv').read()
pd.read_csv('mi.csv',header=[0,1,2,3],index_col=[0,1],tupleize_cols=False)
.. ipython:: python
:suppress:
import os
os.remove('mi.csv')
- ``pd.melt()`` now accepts the optional parameters ``var_name`` and ``value_name``
to specify custom column names of the returned DataFrame.
Bug Fixes
~~~~~~~~~
- Non-unique index support clarified (GH3468_).
- Fix assigning a new index to a duplicate index in a DataFrame would fail (GH3468_)
- Fix construction of a DataFrame with a duplicate index
- ref_locs support to allow duplicative indices across dtypes,
allows iget support to always find the index (even across dtypes) (GH2194_)
- applymap on a DataFrame with a non-unique index now works
(removed warning) (GH2786_), and fix (GH3230_)
- Fix to_csv to handle non-unique columns (GH3495_)
- Duplicate indexes with getitem will return items in the correct order (GH3455_, GH3457_)
and handle missing elements like unique indices (GH3561_)
- Duplicate indexes with and empty DataFrame.from_records will return a correct frame (GH3562_)
- Concat to produce a non-unique columns when duplicates are across dtypes is fixed (GH3602_)
For example you can do
.. ipython :: python
df = DataFrame({'a': list('ab..'), 'b': [1, 2, 3, 4]})
df.replace(regex=r'\s*\.\s*', value=nan)
to replace all occurrences of the string ``'.'`` with zero or more
instances of surrounding whitespace with ``NaN``.
Regular string replacement still works as expected. For example, you can do
.. ipython :: python
df.replace('.', nan)
to replace all occurrences of the string ``'.'`` with ``NaN``.
See the `full release notes
<https://github.com/pydata/pandas/blob/master/RELEASE.rst>`__ or issue tracker
on GitHub for a complete list.
.. _GH3468: https://github.com/pydata/pandas/issues/3468
.. _GH2194: https://github.com/pydata/pandas/issues/2194
.. _GH2786: https://github.com/pydata/pandas/issues/2786
.. _GH3230: https://github.com/pydata/pandas/issues/3230
.. _GH3495: https://github.com/pydata/pandas/issues/3495
.. _GH3455: https://github.com/pydata/pandas/issues/3455
.. _GH3457: https://github.com/pydata/pandas/issues/3457
.. _GH3561: https://github.com/pydata/pandas/issues/3561
.. _GH3562: https://github.com/pydata/pandas/issues/3562
.. _GH3602: https://github.com/pydata/pandas/issues/3602
.. _GH2437: https://github.com/pydata/pandas/issues/2437
.. _GH2852: https://github.com/pydata/pandas/issues/2852
.. _GH3477: https://github.com/pydata/pandas/issues/3477
.. _GH3492: https://github.com/pydata/pandas/issues/3492
.. _GH3499: https://github.com/pydata/pandas/issues/3499
.. _GH2893: https://github.com/pydata/pandas/issues/2893
.. _GH3596: https://github.com/pydata/pandas/issues/3596
.. _GH3590: https://github.com/pydata/pandas/issues/3590
.. _GH3435: https://github.com/pydata/pandas/issues/3435
.. _GH1512: https://github.com/pydata/pandas/issues/1512
.. _GH2285: https://github.com/pydata/pandas/issues/2285
.. _GH3631: https://github.com/pydata/pandas/issues/3631
.. _GH3571: https://github.com/pydata/pandas/issues/3571
.. _GH1651: https://github.com/pydata/pandas/issues/1651
.. _GH3141: https://github.com/pydata/pandas/issues/3141
.. _GH3638: https://github.com/pydata/pandas/issues/3638
.. _GH3616: https://github.com/pydata/pandas/issues/3616
.. _GH3605: https://github.com/pydata/pandas/issues/3605
.. _GH3606: https://github.com/pydata/pandas/issues/3606
.. _GH3656: https://github.com/pydata/pandas/issues/3656