Merge commit 'v0.7.0rc1-205-g8610be5' into debian

* commit 'v0.7.0rc1-205-g8610be5': (22 commits)
  BUG: open file in Python 3 with given encoding
  BUG: actually pass encoding to UnicodeWriter
  BUG: use UnicodeWriter on Python 2.x
  TST: fix vbench per Series constructor api change
  TST: fix numerical stability issue present on certain platforms
  DOC: readme update
  TST: fix debian / 2.5 test
  BUG: always return True for onOffset for regular DateOffset, close pandas-dev#770
  Fixed regression with Panels caused by NDFrame not checking if it was  DataFrame before assuming it had a 'columns' attribute.
  Tests for 'mixed depth' hierarchical columns. Tests for getting, setting, dropping and popping columns from dataframes which have MultiIndexed columns containing some empty strings at the lower levels, by indexing with only the upper levels. Also modified NDFrame._set_item to use the same shorthand.
  Modifications to column-hierarchical behaviour as per "'Mixed depth' hierarchical column labels'" discussion on pystatsmodels. If column label tuples contain only empty strings at the lower levels, allows the DataFrame to be indexed with only the higher levels. So for example if a column has the hierarchical label ('a','',''), and there are no other columns with 'a' at their top level, then this change allows the column to be pulled out with df['a'], rather than df['a','','']. It also names the resulting series 'a'. The use case for this is when some columns have a deeper hierarchy than others, and you want to treat the shallower columns as if the lower levels of the hierarchy weren't there.
  STY: fit in 80 columns
  Add sheet_names property to ExcelFile, with tests
  Put ExcelWriter in pandas namespace
  TST: test coverage
  TST: test coverage, pep8
  TST: skip test on Python 3
  TST: use BytesIO with encoded text for python3
  DOC: update docs to reflect new PyData home
  STY: pep8
  ...

Author: yarikoptic

README.rst

@@ -47,7 +47,7 @@ Here are just a few of the things that pandas does well:
 Where to get it
 ===============
 
-The source code is currently hosted on GitHub at: http://github.com/wesm/pandas
+The source code is currently hosted on GitHub at: http://github.com/pydata/pandas
 
 Binary installers for the latest released version are available at the Python
 package index::
@@ -91,7 +91,7 @@ On Windows, you will need to install MinGW and execute::
     python setup.py build --compiler=mingw32
     python setup.py install
 
-See http://pandas.sourceforge.net/ for more information.
+See http://pandas.pydata.org/ for more information.
 
 License
 =======
@@ -101,7 +101,7 @@ BSD
 Documentation
 =============
 
-The official documentation is hosted on SourceForge: http://pandas.sourceforge.net/
+The official documentation is hosted on PyData.org: http://pandas.pydata.org/
 
 The Sphinx documentation should provide a good starting point for learning how
 to use the library. Expect the docs to continue to expand as time goes on.
@@ -124,4 +124,3 @@ the pystatsmodels mailing list / Google group, where
 http://groups.google.com/group/pystatsmodels
 
   .. _NumPy: http://numpy.scipy.org/
-  .. _downloaded there: https://github.com/wesm/pandas/archives/master

RELEASE.rst

@@ -3,7 +3,7 @@ Release Notes
 =============
 
 This is the list of changes to pandas between each release. For full details,
-see the commit logs at http://github.com/wesm/pandas
+see the commit logs at http://github.com/pydata/pandas
 
 What is it
 ----------
@@ -18,14 +18,14 @@ analysis / manipulation tool available in any language.
 Where to get it
 ---------------
 
-* Source code: http://github.com/wesm/pandas
+* Source code: http://github.com/pydata/pandas
 * Binary installers on PyPI: http://pypi.python.org/pypi/pandas
-* Documentation: http://pandas.sourceforge.net
+* Documentation: http://pandas.pydata.org
 
 pandas 0.7.0
 ============
 
-**Release date:** NOT YET RELEASED
+**Release date:** 2/9/2012
 
 **New features / modules**
 
@@ -91,11 +91,13 @@ pandas 0.7.0
   - Add ``sort`` option to ``DataFrame.join`` (GH #731)
   - Improved handling of NAs (propagation) in binary operations with
     dtype=object arrays (GH #737)
+  - Add ``abs`` method to Pandas objects
+  - Added ``algorithms`` module to start collecting central algos
 
 **API Changes**
 
   - Label-indexing with integer indexes now raises KeyError if a label is not
-    found instead of falling back on location-based indexing
+    found instead of falling back on location-based indexing (GH #700)
   - Label-based slicing via ``ix`` or ``[]`` on Series will now only work if
     exact matches for the labels are found or if the index is monotonic (for
     range selections)
@@ -119,6 +121,8 @@ pandas 0.7.0
     #395)
   - The default delimiter for ``read_csv`` is comma rather than letting
     ``csv.Sniffer`` infer it
+  - Rename ``col_or_columns`` argument in ``DataFrame.drop_duplicates`` (GH
+    #734)
 
 **Improvements to existing features**
 
@@ -178,6 +182,14 @@ pandas 0.7.0
   - Substantially improved performance of groupby on DataFrames with many
     columns by aggregating blocks of columns all at once (GH #745)
   - Can pass a file handle or StringIO to Series/DataFrame.to_csv (GH #765)
+  - Can pass sequence of integers to DataFrame.irow(icol) and Series.iget, (GH
+    #654)
+  - Prototypes for some vectorized string functions
+  - Add float64 hash table to solve the Series.unique problem with NAs (GH #714)
+  - Memoize objects when reading from file to reduce memory footprint
+  - Can get and set a column of a DataFrame with hierarchical columns
+    containing "empty" ('') lower levels without passing the empty levels (PR
+    #768)
 
 **Bug fixes**
 
@@ -261,12 +273,41 @@ pandas 0.7.0
   - Use right dropna function for SparseSeries. Return dense Series for NA fill
     value (GH #730)
   - Fix Index.format bug causing incorrectly string-formatted Series with
-    datetime indexes (# 758)
+    datetime indexes (# 726, 758)
   - Fix errors caused by object dtype arrays passed to ols (GH #759)
+  - Fix error where column names lost when passing list of labels to
+    DataFrame.__getitem__, (GH #662)
+  - Fix error whereby top-level week iterator overwrote week instance
+  - Fix circular reference causing memory leak in sparse array / series /
+    frame, (GH #663)
+  - Fix integer-slicing from integers-as-floats (GH #670)
+  - Fix zero division errors in nanops from object dtype arrays in all NA case
+    (GH #676)
+  - Fix csv encoding when using unicode (GH #705, #717, #738)
+  - Fix assumption that each object contains every unique block type in concat,
+    (GH #708)
+  - Fix sortedness check of multiindex in to_panel (GH #719, 720)
+  - Fix that None was not treated as NA in PyObjectHashtable
+  - Fix hashing dtype because of endianness confusion (GH #747, #748)
+  - Fix SparseSeries.dropna to return dense Series in case of NA fill value (GH
+    #730)
+  - Use map_infer instead of np.vectorize. handle NA sentinels if converter
+    yields numeric array, (GH #753)
+  - Fixes and improvements to DataFrame.rank (GH #742)
+  - Fix catching AttributeError instead of NameError for bottleneck
+  - Try to cast non-MultiIndex to better dtype when calling reset_index (GH #726
+    #440)
+  - Fix #1.QNAN0' float bug on 2.6/win64
+  - Allow subclasses of dicts in DataFrame constructor, with tests
+  - Fix problem whereby set_index destroys column multiindex (GH #764)
+  - Hack around bug in generating DateRange from naive DateOffset (GH #770)
+  - Fix bug in DateRange.intersection causing incorrect results with some
+    overlapping ranges (GH #771)
 
 Thanks
 ------
 - Craig Austin
+- Chris Billington
 - Marius Cobzarenco
 - Mario Gamboa-Cavazos
 - Hans-Martin Gaudecker

doc/source/basics.rst

@@ -277,6 +277,7 @@ optional ``level`` parameter which applies only if the object has a
     ``median``, Arithmetic median of values
     ``min``, Minimum
     ``max``, Maximum
+    ``abs``, Absolute Value
     ``prod``, Product of values
     ``std``, Unbiased standard deviation
     ``var``, Unbiased variance

doc/source/index.rst

@@ -12,7 +12,7 @@ pandas: powerful Python data analysis toolkit
 
 **Installers:** http://pypi.python.org/pypi/pandas
 
-**Code Repository:** http://github.com/wesm/pandas
+**Code Repository:** http://github.com/pydata/pandas
 
 **Mailing List:** http://groups.google.com/group/pystatsmodels
 

doc/source/indexing.rst

@@ -140,6 +140,7 @@ label, respectively.
    panel.major_xs(date)
    panel.minor_xs('A')
 
+
 Slicing ranges
 ~~~~~~~~~~~~~~
 
@@ -473,6 +474,12 @@ operators:
    a & b
    a - b
 
+``isin`` method of Index objects
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+One additional operation is the ``isin`` method that works analogously to the
+``Series.isin`` method found :ref:`here <indexing.boolean>`.
+
 .. _indexing.hierarchical:
 
 Hierarchical indexing (MultiIndex)
@@ -643,6 +650,18 @@ but as you use it you may uncover corner cases or unintuitive behavior. If you
 do find something like this, do not hesitate to report the issue or ask on the
 mailing list.
 
+.. _indexing.xs:
+
+Cross-section with hierarchical index
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``xs`` method of ``DataFrame`` additionally takes a level argument to make
+selecting data at a particular level of a MultiIndex easier.
+
+.. ipython:: python
+
+    df.xs('one', level='second')
+
 .. _indexing.advanced_reindex:
 
 Advanced reindexing and alignment with hierarchical index

doc/source/install.rst

@@ -8,7 +8,7 @@ Installation
 
 You have the option to install an `official release
 <http://pypi.python.org/pypi/pandas>`__ or to build the `development version
-<http://github.com/wesm/pandas>`__. If you choose to install from source and
+<http://github.com/pydata/pandas>`__. If you choose to install from source and
 are running Windows, you will have to ensure that you have a compatible C
 compiler (MinGW or Visual Studio) installed. `How-to install MinGW on Windows
 <http://docs.cython.org/src/tutorial/appendix.html>`__
@@ -63,12 +63,12 @@ Installing from source
    files. I recommend installing the latest Cython via ``easy_install -U
    Cython``
 
-The source code is hosted at http://github.com/wesm/pandas, it can be checked
+The source code is hosted at http://github.com/pydata/pandas, it can be checked
 out using git and compiled / installed like so:
 
 ::
 
-  git clone git://github.com/wesm/pandas.git
+  git clone git://github.com/pydata/pandas.git
   cd pandas
   python setup.py install
 

doc/source/io.rst

@@ -95,6 +95,7 @@ data into a DataFrame object. They can take a number of arguments:
     columns, where keys are either integers or column labels
   - ``encoding``: a string representing the encoding to use if the contents are
     non-ascii, for python versions prior to 3 
+  - ``verbose`` : show number of NA values inserted in non-numeric columns
 
 .. ipython:: python
    :suppress:

doc/source/reshaping.rst

@@ -284,6 +284,39 @@ calling ``to_string`` if you wish:
 
 Note that ``pivot_table`` is also available as an instance method on DataFrame.
 
+Cross tabulations
+~~~~~~~~~~~~~~~~~
+
+Use the ``crosstab`` function to compute a cross-tabulation of two (or more)
+factors. By default ``crosstab`` computes a frequency table of the factors
+unless an array of values and an aggregation function are passed.
+
+It takes a number of arguments
+
+- ``rows``: array-like, values to group by in the rows
+- ``cols``: array-like, values to group by in the columns
+- ``values``: array-like, optional, array of values to aggregate according to
+  the factors
+- ``aggfunc``: function, optional, If no values array is passed, computes a
+  frequency table
+- ``rownames``: sequence, default None, must match number of row arrays passed
+- ``colnames``: sequence, default None, if passed, must match number of column
+  arrays passed
+- ``margins``: boolean, default False, Add row/column margins (subtotals)
+
+Any Series passed will have their name attributes used unless row or column
+names for the cross-tabulation are specified
+
+For example:
+
+.. ipython:: python
+
+    foo, bar, dull, shiny, one, two = 'foo', 'bar', 'dull', 'shiny', 'one', 'two'
+    a = np.array([foo, foo, bar, bar, foo, foo], dtype=object)
+    b = np.array([one, one, two, one, two, one], dtype=object)
+    c = np.array([dull, dull, shiny, dull, dull, shiny], dtype=object)
+    crosstab(a, [b, c], rownames=['a'], colnames=['b', 'c'])
+
 .. _reshaping.pivot.margins:
 
 Adding margins (partial aggregates)

doc/source/whatsnew/v0.4.x.txt

@@ -57,21 +57,21 @@ Performance Enhancements
 - Wrote fast time series merging / joining methods in Cython. Will be
   integrated later into DataFrame.join and related functions
 
-.. _PR146: https://github.com/wesm/pandas/pull/146
-.. _ENH1b: https://github.com/wesm/pandas/commit/1ba56251f0013ff7cd8834e9486cef2b10098371
-.. _ENHdc: https://github.com/wesm/pandas/commit/dca3c5c5a6a3769ee01465baca04cfdfa66a4f76
-.. _GH135: https://github.com/wesm/pandas/issues/135
-.. _PR151: https://github.com/wesm/pandas/pull/151
-.. _ENHed: https://github.com/wesm/pandas/commit/edd9f1945fc010a57fa0ae3b3444d1fffe592591
-.. _PR142: https://github.com/wesm/pandas/pull/142
-.. _PR141: https://github.com/wesm/pandas/pull/141
-.. _IS188: https://github.com/wesm/pandas/issues/188
-.. _ENH56: https://github.com/wesm/pandas/commit/56e0c9ffafac79ce262b55a6a13e1b10a88fbe93
-.. _GH187: https://github.com/wesm/pandas/issues/187
-.. _GH176: https://github.com/wesm/pandas/issues/176
-.. _GH104: https://github.com/wesm/pandas/issues/104
-.. _GH205: https://github.com/wesm/pandas/issues/205
-.. _PR209: https://github.com/wesm/pandas/pull/209
-.. _GH203: https://github.com/wesm/pandas/issues/203
-.. _PR200: https://github.com/wesm/pandas/pull/200
+.. _PR146: https://github.com/pydata/pandas/pull/146
+.. _ENH1b: https://github.com/pydata/pandas/commit/1ba56251f0013ff7cd8834e9486cef2b10098371
+.. _ENHdc: https://github.com/pydata/pandas/commit/dca3c5c5a6a3769ee01465baca04cfdfa66a4f76
+.. _GH135: https://github.com/pydata/pandas/issues/135
+.. _PR151: https://github.com/pydata/pandas/pull/151
+.. _ENHed: https://github.com/pydata/pandas/commit/edd9f1945fc010a57fa0ae3b3444d1fffe592591
+.. _PR142: https://github.com/pydata/pandas/pull/142
+.. _PR141: https://github.com/pydata/pandas/pull/141
+.. _IS188: https://github.com/pydata/pandas/issues/188
+.. _ENH56: https://github.com/pydata/pandas/commit/56e0c9ffafac79ce262b55a6a13e1b10a88fbe93
+.. _GH187: https://github.com/pydata/pandas/issues/187
+.. _GH176: https://github.com/pydata/pandas/issues/176
+.. _GH104: https://github.com/pydata/pandas/issues/104
+.. _GH205: https://github.com/pydata/pandas/issues/205
+.. _PR209: https://github.com/pydata/pandas/pull/209
+.. _GH203: https://github.com/pydata/pandas/issues/203
+.. _PR200: https://github.com/pydata/pandas/pull/200
 

doc/source/whatsnew/v0.5.0.txt

@@ -39,22 +39,22 @@ Performance Enhancements
 - VBENCH Significantly sped up conversion of nested dict into DataFrame (GH212_)
 - VBENCH Significantly speed up DataFrame ``__repr__`` and ``count`` on large mixed-type DataFrame objects
 
-.. _GH214: https://github.com/wesm/pandas/issues/214
-.. _GH248: https://github.com/wesm/pandas/issues/248
-.. _GH253: https://github.com/wesm/pandas/issues/253
-.. _GH261: https://github.com/wesm/pandas/issues/261
-.. _GH234: https://github.com/wesm/pandas/issues/234
-.. _GH243: https://github.com/wesm/pandas/issues/243
-.. _GH223: https://github.com/wesm/pandas/issues/223
-.. _PR244: https://github.com/wesm/pandas/pull/244
-.. _PR233: https://github.com/wesm/pandas/pull/233
-.. _GH230: https://github.com/wesm/pandas/issues/230
-.. _PR241: https://github.com/wesm/pandas/pull/241
-.. _GH242: https://github.com/wesm/pandas/issues/242
-.. _GH212: https://github.com/wesm/pandas/issues/212
-.. _GH211: https://github.com/wesm/pandas/issues/211
-.. _GH215: https://github.com/wesm/pandas/issues/215
-.. _GH213: https://github.com/wesm/pandas/issues/213
-.. _ENH61: https://github.com/wesm/pandas/commit/6141961
-.. _PR334: https://github.com/wesm/pandas/pull/334
-.. _ENH5c: https://github.com/wesm/pandas/commit/5ca6ff5d822ee4ddef1ec0d87b6d83d8b4bbd3eb
+.. _GH214: https://github.com/pydata/pandas/issues/214
+.. _GH248: https://github.com/pydata/pandas/issues/248
+.. _GH253: https://github.com/pydata/pandas/issues/253
+.. _GH261: https://github.com/pydata/pandas/issues/261
+.. _GH234: https://github.com/pydata/pandas/issues/234
+.. _GH243: https://github.com/pydata/pandas/issues/243
+.. _GH223: https://github.com/pydata/pandas/issues/223
+.. _PR244: https://github.com/pydata/pandas/pull/244
+.. _PR233: https://github.com/pydata/pandas/pull/233
+.. _GH230: https://github.com/pydata/pandas/issues/230
+.. _PR241: https://github.com/pydata/pandas/pull/241
+.. _GH242: https://github.com/pydata/pandas/issues/242
+.. _GH212: https://github.com/pydata/pandas/issues/212
+.. _GH211: https://github.com/pydata/pandas/issues/211
+.. _GH215: https://github.com/pydata/pandas/issues/215
+.. _GH213: https://github.com/pydata/pandas/issues/213
+.. _ENH61: https://github.com/pydata/pandas/commit/6141961
+.. _PR334: https://github.com/pydata/pandas/pull/334
+.. _ENH5c: https://github.com/pydata/pandas/commit/5ca6ff5d822ee4ddef1ec0d87b6d83d8b4bbd3eb