Merge tag 'v0.8.1' into debian

Version 0.8.1

* tag 'v0.8.1': (126 commits)
  RLS: Version 0.8.1
  DOC: tweak
  DOC: set_index/reset_index examples
  DOC: doc fixes and what's new in 0.8.1, vectorized string methods
  ENH: better string element access/slicing notation close pandas-dev#1656
  DOC: minor additions to release notes for 0.8.1
  BUG: handle Yahoo! finance returning duplicate dates for prev bus day, doc fixes
  BUG: fix windows/32-bit builds
  BUG: get pandas-dev#1620 fix working on python 3
  ENH: handling of UTF-8 strings in DataFrame columns, close pandas-dev#1620
  TST: span unit test pandas-dev#1635
  TST: skip another @network test if no internet connection
  ENH/BUG: handle tz-aware datetime.datetime in to_datetime, add utc=True option to allow conversion to utc, close pandas-dev#1581
  ENH: hack to not compress single group keys, accelerate single-key and Categorical groupby operations
  BUG: fix merge bug with left joins on length-0 DataFrame, close pandas-dev#1628
  BUG: Series.interpolate bug with method='values' and datetime64[ns], close pandas-dev#1646
  BUG: properly handle None values in dict input to concat, close pandas-dev#1649
  BUG: len-0 Series min/max/describe pandas-dev#1650
  Fix describe() failure for None and empty Series.
  BUG: string date aliases now work with tz-aware time series close pandas-dev#1647
  ...

Author: yarikoptic

RELEASE.rst

@@ -22,6 +22,94 @@ Where to get it
 * Binary installers on PyPI: http://pypi.python.org/pypi/pandas
 * Documentation: http://pandas.pydata.org
 
+pandas 0.8.1
+============
+
+**Release date:** July 22, 2012
+
+**New features**
+
+  - Add vectorized, NA-friendly string methods to Series (#1621, #620)
+  - Can pass dict of per-column line styles to DataFrame.plot (#1559)
+  - Selective plotting to secondary y-axis on same subplot (PR #1640)
+  - Add new ``bootstrap_plot`` plot function
+  - Add new ``parallel_coordinates`` plot function (#1488)
+  - Add ``radviz`` plot function (#1566)
+  - Add ``multi_sparse`` option to ``set_printoptions`` to modify display of
+    hierarchical indexes (#1538)
+  - Add ``dropna`` method to Panel (#171)
+
+**Improvements to existing features**
+
+  - Use moving min/max algorithms from Bottleneck in rolling_min/rolling_max
+    for > 100x speedup. (#1504, #50)
+  - Add Cython group median method for >15x speedup (#1358)
+  - Drastically improve ``to_datetime`` performance on ISO8601 datetime strings
+    (with no time zones) (#1571)
+  - Improve single-key groupby performance on large data sets, accelerate use of
+    groupby with a Categorical variable
+  - Add ability to append hierarchical index levels with ``set_index`` and to
+    drop single levels with ``reset_index`` (#1569, #1577)
+  - Always apply passed functions in ``resample``, even if upsampling (#1596)
+  - Avoid unnecessary copies in DataFrame constructor with explicit dtype (#1572)
+  - Cleaner DatetimeIndex string representation with 1 or 2 elements (#1611)
+  - Improve performance of array-of-Period to PeriodIndex, convert such arrays
+    to PeriodIndex inside Index (#1215)
+  - More informative string representation for weekly Period objects (#1503)
+  - Accelerate 3-axis multi data selection from homogeneous Panel (#979)
+  - Add ``adjust`` option to ewma to disable adjustment factor (#1584)
+  - Add new matplotlib converters for high frequency time series plotting (#1599)
+  - Handling of tz-aware datetime.datetime objects in to_datetime; raise
+    Exception unless utc=True given (#1581)
+
+**Bug fixes**
+
+  - Fix NA handling in DataFrame.to_panel (#1582)
+  - Handle TypeError issues inside PyObject_RichCompareBool calls in khash
+    (#1318)
+  - Fix resampling bug to lower case daily frequency (#1588)
+  - Fix kendall/spearman DataFrame.corr bug with no overlap (#1595)
+  - Fix bug in DataFrame.set_index (#1592)
+  - Don't ignore axes in boxplot if by specified (#1565)
+  - Fix Panel .ix indexing with integers bug (#1603)
+  - Fix Partial indexing bugs (years, months, ...) with PeriodIndex (#1601)
+  - Fix MultiIndex console formatting issue (#1606)
+  - Unordered index with duplicates doesn't yield scalar location for single
+    entry (#1586)
+  - Fix resampling of tz-aware time series with "anchored" freq (#1591)
+  - Fix DataFrame.rank error on integer data (#1589)
+  - Selection of multiple SparseDataFrame columns by list in __getitem__ (#1585)
+  - Override Index.tolist for compatibility with MultiIndex (#1576)
+  - Fix hierarchical summing bug with MultiIndex of length 1 (#1568)
+  - Work around numpy.concatenate use/bug in Series.set_value (#1561)
+  - Ensure Series/DataFrame are sorted before resampling (#1580)
+  - Fix unhandled IndexError when indexing very large time series (#1562)
+  - Fix DatetimeIndex intersection logic error with irregular indexes (#1551)
+  - Fix unit test errors on Python 3 (#1550)
+  - Fix .ix indexing bugs in duplicate DataFrame index (#1201)
+  - Better handle errors with non-existing objects in HDFStore (#1254)
+  - Don't copy int64 array data in DatetimeIndex when copy=False (#1624)
+  - Fix resampling of conforming periods quarterly to annual (#1622)
+  - Don't lose index name on resampling (#1631)
+  - Support python-dateutil version 2.1 (#1637)
+  - Fix broken scatter_matrix axis labeling, esp. with time series (#1625)
+  - Fix cases where extra keywords weren't being passed on to matplotlib from
+    Series.plot (#1636)
+  - Fix BusinessMonthBegin logic for dates before 1st bday of month (#1645)
+  - Ensure string alias converted (valid in DatetimeIndex.get_loc) in
+    DataFrame.xs / __getitem__ (#1644)
+  - Fix use of string alias timestamps with tz-aware time series (#1647)
+  - Fix Series.max/min and Series.describe on len-0 series (#1650)
+  - Handle None values in dict passed to concat (#1649)
+  - Fix Series.interpolate with method='values' and DatetimeIndex (#1646)
+  - Fix IndexError in left merges on a DataFrame with 0-length (#1628)
+  - Fix DataFrame column width display with UTF-8 encoded characters (#1620)
+  - Handle case in pandas.io.data.get_data_yahoo where Yahoo! returns duplicate
+    dates for most recent business day
+  - Avoid downsampling when plotting mixed frequencies on the same subplot (#1619)
+  - Fix read_csv bug when reading a single line (#1553)
+  - Fix bug in C code causing monthly periods prior to December 1969 to be off (#1570)
+
 pandas 0.8.0
 ============
 
@@ -140,6 +228,7 @@ pandas 0.8.0
 
 **API Changes**
 
+  - Rename `pandas._tseries` to `pandas.lib`
   - Rename Factor to Categorical and add improvements. Numerous Categorical bug
     fixes
   - Frequency name overhaul, WEEKDAY/EOM and rules with @
@@ -1661,92 +1750,58 @@ Thanks
 pandas 0.3.0
 ============
 
-This major release of pandas represents approximately 1 year of continuous
-development work and brings with it many new features, bug fixes, speed
-enhancements, and general quality-of-life improvements. The most significant
-change from the 0.2 release has been the completion of a rigorous unit test
-suite covering all of the core functionality.
-
 Release notes
 -------------
 
 **Release date:** February 20, 2011
 
 **New features / modules**
 
-* DataFrame / DataMatrix classes
-
- * `corrwith` function to compute column- or row-wise correlations between two
-   objects
- * Can boolean-index DataFrame objects, e.g. df[df > 2] = 2, px[px > last_px] = 0
- * Added comparison magic methods (__lt__, __gt__, etc.)
- * Flexible explicit arithmetic methods (add, mul, sub, div, etc.)
- * Added `reindex_like` method
-
-* WidePanel
-
- * Added `reindex_like` method
-
-* `pandas.io`: IO utilities
-
-  * `pandas.io.sql` module
-
-    * Convenience functions for accessing SQL-like databases
-
-  * `pandas.io.pytables` module
-
-   * Added (still experimental) HDFStore class for storing pandas data
-     structures using HDF5 / PyTables
-
-* `pandas.core.datetools`
-
-  * Added WeekOfMonth date offset
-
-* `pandas.rpy` (experimental) module created, provide some interfacing /
-  conversion between rpy2 and pandas
+  - `corrwith` function to compute column- or row-wise correlations between two
+	DataFrame objects
+  - Can boolean-index DataFrame objects, e.g. df[df > 2] = 2, px[px > last_px] = 0
+  - Added comparison magic methods (__lt__, __gt__, etc.)
+  - Flexible explicit arithmetic methods (add, mul, sub, div, etc.)
+  - Added `reindex_like` method
+  - Added `reindex_like` method to WidePanel
+  - Convenience functions for accessing SQL-like databases in `pandas.io.sql`
+	module
+  - Added (still experimental) HDFStore class for storing pandas data
+	structures using HDF5 / PyTables in `pandas.io.pytables` module
+  - Added WeekOfMonth date offset
+  - `pandas.rpy` (experimental) module created, provide some interfacing /
+   conversion between rpy2 and pandas
 
 **Improvements**
 
-* Unit test coverage: 100% line coverage of core data structures
-
-* Speed enhancement to rolling_{median, max, min}
-
-* Column ordering between DataFrame and DataMatrix is now consistent: before
-  DataFrame would not respect column order
-
-* Improved {Series, DataFrame}.plot methods to be more flexible (can pass
-  matplotlib Axis arguments, plot DataFrame columns in multiple subplots, etc.)
+  - Unit test coverage: 100% line coverage of core data structures
+  - Speed enhancement to rolling_{median, max, min}
+  - Column ordering between DataFrame and DataMatrix is now consistent: before
+	DataFrame would not respect column order
+  - Improved {Series, DataFrame}.plot methods to be more flexible (can pass
+	matplotlib Axis arguments, plot DataFrame columns in multiple subplots,
+	etc.)
 
 **API Changes**
 
-* Exponentially-weighted moment functions in `pandas.stats.moments`
-  have a more consistent API and accept a min_periods argument like
-  their regular moving counterparts.
-
-* **fillMethod** argument in Series, DataFrame changed to **method**,
-  `FutureWarning` added.
-
-* **fill** method in Series, DataFrame/DataMatrix, WidePanel renamed to
-  **fillna**, `FutureWarning` added to **fill**
-
-* Renamed **DataFrame.getXS** to **xs**, `FutureWarning` added
-
-* Removed **cap** and **floor** functions from DataFrame, renamed to
-  **clip_upper** and **clip_lower** for consistency with NumPy
+  - Exponentially-weighted moment functions in `pandas.stats.moments` have a
+	more consistent API and accept a min_periods argument like their regular
+	moving counterparts.
+  - **fillMethod** argument in Series, DataFrame changed to **method**,
+	`FutureWarning` added.
+  - **fill** method in Series, DataFrame/DataMatrix, WidePanel renamed to
+	**fillna**, `FutureWarning` added to **fill**
+  - Renamed **DataFrame.getXS** to **xs**, `FutureWarning` added
+  - Removed **cap** and **floor** functions from DataFrame, renamed to
+	**clip_upper** and **clip_lower** for consistency with NumPy
 
 **Bug fixes**
 
-* Fixed bug in IndexableSkiplist Cython code that was breaking
-  rolling_max function
-
-* Numerous numpy.int64-related indexing fixes
-
-* Several NumPy 1.4.0 NaN-handling fixes
-
-* Bug fixes to pandas.io.parsers.parseCSV
-
-* Fixed `DateRange` caching issue with unusual date offsets
-
-* Fixed bug in `DateRange.union`
-
-* Fixed corner case in `IndexableSkiplist` implementation
+  - Fixed bug in IndexableSkiplist Cython code that was breaking
+	rolling_max function
+  - Numerous numpy.int64-related indexing fixes
+  - Several NumPy 1.4.0 NaN-handling fixes
+  - Bug fixes to pandas.io.parsers.parseCSV
+  - Fixed `DateRange` caching issue with unusual date offsets
+  - Fixed bug in `DateRange.union`
+  - Fixed corner case in `IndexableSkiplist` implementation

TODO.rst

@@ -57,4 +57,7 @@ Performance blog
 - Take
 
 git log v0.6.1..master --pretty=format:%aN | sort | uniq -c | sort -rn
-git log a8c2f88..master --pretty=format:%aN | sort | uniq -c | sort -rn
+
+git log 7ddfbd4..master --pretty=format:%aN | sort | uniq -c | sort -rn
+git log a0257f5..master --pretty=format:%aN | sort | uniq -c | sort -rn
+

doc/data/fx_prices

@@ -148,5 +148,4 @@ SepalLength,SepalWidth,PetalLength,PetalWidth,Name
 6.3,2.5,5.0,1.9,Iris-virginica
 6.5,3.0,5.2,2.0,Iris-virginica
 6.2,3.4,5.4,2.3,Iris-virginica
-5.9,3.0,5.1,1.8,Iris-virginica
-
+5.9,3.0,5.1,1.8,Iris-virginica

doc/data/iris.data

@@ -141,7 +141,7 @@ an axis and broadcasting over the same axis:
    major_mean
    wp.sub(major_mean, axis='major')
 
-And similarly for axis="items" and axis="minor".
+And similarly for ``axis="items"`` and ``axis="minor"``.
 
 .. note::
 
@@ -369,6 +369,15 @@ index labels with the minimum and maximum corresponding values:
    df1.idxmin(axis=0)
    df1.idxmax(axis=1)
 
+When there are multiple rows (or columns) matching the minimum or maximum
+value, ``idxmin`` and ``idxmax`` return the first matching index:
+
+.. ipython:: python
+
+   df3 = DataFrame([2, 1, 1, 3, np.nan], columns=['A'], index=list('edcba'))
+   df3
+   df3['A'].idxmin()
+
 Value counts (histogramming)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -826,6 +835,74 @@ For instance,
 
    for r in df2.itertuples(): print r
 
+.. _basics.string_methods:
+
+Vectorized string methods
+-------------------------
+
+Series is equipped (as of pandas 0.8.1) with a set of string processing methods
+that make it easy to operate on each element of the array. Perhaps most
+importantly, these methods exclude missing/NA values automatically. These are
+accessed via the Series's ``str`` attribute and generally have names matching
+the equivalent (scalar) build-in string methods:
+
+.. ipython:: python
+
+   s = Series(['A', 'B', 'C', 'Aaba', 'Baca', np.nan, 'CABA', 'dog', 'cat'])
+   s.str.lower()
+   s.str.upper()
+   s.str.len()
+
+Methods like ``split`` return a Series of lists:
+
+.. ipython:: python
+
+   s2 = Series(['a_b_c', 'c_d_e', np.nan, 'f_g_h'])
+   s2.str.split('_')
+
+Elements in the split lists can be accessed using ``get`` or ``[]`` notation:
+
+.. ipython:: python
+
+   s2.str.split('_').str.get(1)
+   s2.str.split('_').str[1]
+
+Methods like ``replace`` and ``findall`` take regular expressions, too:
+
+.. ipython:: python
+
+   s3 = Series(['A', 'B', 'C', 'Aaba', 'Baca',
+               '', np.nan, 'CABA', 'dog', 'cat'])
+   s3
+   s3.str.replace('^.a|dog', 'XX-XX ', case=False)
+
+.. csv-table::
+    :header: "Method", "Description"
+    :widths: 20, 80
+
+    ``cat``,Concatenate strings
+    ``split``,Split strings on delimiter
+    ``get``,Index into each element (retrieve i-th element)
+    ``join``,Join strings in each element of the Series with passed separator
+    ``contains``,Return boolean array if each string contains pattern/regex
+    ``replace``,Replace occurrences of pattern/regex with some other string
+    ``repeat``,Duplicate values (``s.str.repeat(3)`` equivalent to ``x * 3``)
+    ``pad``,"Add whitespace to left, right, or both sides of strings"
+    ``center``,Equivalent to ``pad(side='both')``
+    ``slice``,Slice each string in the Series
+    ``slice_replace``,Replace slice in each string with passed value
+    ``count``,Count occurrences of pattern
+    ``startswith``,Equivalent to ``str.startswith(pat)`` for each element
+    ``endswidth``,Equivalent to ``str.endswith(pat)`` for each element
+    ``findall``,Compute list of all occurrences of pattern/regex for each string
+    ``match``,"Call ``re.match`` on each element, returning matched groups as list"
+    ``len``,Compute string lengths
+    ``strip``,Equivalent to ``str.strip``
+    ``rstrip``,Equivalent to ``str.rstrip``
+    ``lstrip``,Equivalent to ``str.lstrip``
+    ``lower``,Equivalent to ``str.lower``
+    ``upper``,Equivalent to ``str.upper``
+
 .. _basics.sorting:
 
 Sorting by index and value

doc/source/basics.rst

@@ -299,7 +299,7 @@ average as
 
 .. math::
 
-    y_t = (1-\alpha) y_{t-1} + \alpha x_t
+    y_t = \alpha y_{t-1} + (1 - \alpha) x_t
 
 One must have :math:`0 < \alpha \leq 1`, but rather than pass :math:`\alpha`
 directly, it's easier to think about either the **span** or **center of mass

doc/source/computation.rst

@@ -32,6 +32,13 @@ between labels and data will not be broken unless done so explicitly by you.
 We'll give a brief intro to the data structures, then consider all of the broad
 categories of functionality and methods in separate sections.
 
+When using pandas, we recommend the following import convention:
+
+.. code-block:: python
+
+   import pandas as pd
+
+
 .. _basics.series:
 
 Series