Merge branch 'master' into merge_indicator

Author: Nick Eubank

doc/source/advanced.rst

@@ -675,10 +675,7 @@ values NOT in the categories, similarly to how you can reindex ANY pandas index.
                                }).set_index('B')
 
       In [11]: df3.index
-      Out[11]:
-      CategoricalIndex([u'a', u'a', u'b', u'b', u'c', u'a'],
-                       categories=[u'a', u'b', u'c'],
-                       ordered=False)
+      Out[11]: CategoricalIndex([u'a', u'a', u'b', u'b', u'c', u'a'], categories=[u'a', u'b', u'c'], ordered=False, name=u'B', dtype='category')
 
       In [12]: pd.concat([df2,df3]
       TypeError: categories must match existing categories when appending

doc/source/api.rst

@@ -158,6 +158,7 @@ Top-level dealing with datetimelike
    bdate_range
    period_range
    timedelta_range
+   infer_freq
 
 Top-level evaluation
 ~~~~~~~~~~~~~~~~~~~~
@@ -491,6 +492,7 @@ These can be accessed like ``Series.dt.<property>``.
    Series.dt.to_pydatetime
    Series.dt.tz_localize
    Series.dt.tz_convert
+   Series.dt.normalize
 
 **Timedelta Properties**
 
@@ -534,17 +536,22 @@ strings and apply several methods to it. These can be acccessed like
    Series.str.find
    Series.str.findall
    Series.str.get
+   Series.str.index
    Series.str.join
    Series.str.len
    Series.str.ljust
    Series.str.lower
    Series.str.lstrip
    Series.str.match
+   Series.str.normalize
    Series.str.pad
+   Series.str.partition
    Series.str.repeat
    Series.str.replace
    Series.str.rfind
+   Series.str.rindex
    Series.str.rjust
+   Series.str.rpartition
    Series.str.rstrip
    Series.str.slice
    Series.str.slice_replace
@@ -553,6 +560,7 @@ strings and apply several methods to it. These can be acccessed like
    Series.str.strip
    Series.str.swapcase
    Series.str.title
+   Series.str.translate
    Series.str.upper
    Series.str.wrap
    Series.str.zfill
@@ -1364,6 +1372,7 @@ Time/Date Components
    DatetimeIndex.is_quarter_end
    DatetimeIndex.is_year_start
    DatetimeIndex.is_year_end
+   DatetimeIndex.inferred_freq
 
 Selecting
 ~~~~~~~~~
@@ -1414,6 +1423,7 @@ Components
    TimedeltaIndex.microseconds
    TimedeltaIndex.nanoseconds
    TimedeltaIndex.components
+   TimedeltaIndex.inferred_freq
 
 Conversion
 ~~~~~~~~~~

doc/source/basics.rst

@@ -1004,7 +1004,7 @@ Note that the following also works, but is a bit less obvious / clean:
 
 .. ipython:: python
 
-   df.reindex(df.index - ['a', 'd'])
+   df.reindex(df.index.difference(['a', 'd']))
 
 .. _basics.rename:
 

doc/source/computation.rst

@@ -1,23 +1,22 @@
 .. currentmodule:: pandas
-.. _computation:
 
 .. ipython:: python
    :suppress:
 
    import numpy as np
    np.random.seed(123456)
-   from pandas import *
-   import pandas.util.testing as tm
-   randn = np.random.randn
    np.set_printoptions(precision=4, suppress=True)
+   import pandas as pd
    import matplotlib
    try:
       matplotlib.style.use('ggplot')
    except AttributeError:
-      options.display.mpl_style = 'default'
+      pd.options.display.mpl_style = 'default'
    import matplotlib.pyplot as plt
    plt.close('all')
-   options.display.max_rows=15
+   pd.options.display.max_rows=15
+
+.. _computation:
 
 Computational tools
 ===================
@@ -36,13 +35,13 @@ NA/null values *before* computing the percent change).
 
 .. ipython:: python
 
-   ser = Series(randn(8))
+   ser = pd.Series(np.random.randn(8))
 
    ser.pct_change()
 
 .. ipython:: python
 
-   df = DataFrame(randn(10, 4))
+   df = pd.DataFrame(np.random.randn(10, 4))
 
    df.pct_change(periods=3)
 
@@ -56,8 +55,8 @@ The ``Series`` object has a method ``cov`` to compute covariance between series
 
 .. ipython:: python
 
-   s1 = Series(randn(1000))
-   s2 = Series(randn(1000))
+   s1 = pd.Series(np.random.randn(1000))
+   s2 = pd.Series(np.random.randn(1000))
    s1.cov(s2)
 
 Analogously, ``DataFrame`` has a method ``cov`` to compute pairwise covariances
@@ -78,7 +77,7 @@ among the series in the DataFrame, also excluding NA/null values.
 
 .. ipython:: python
 
-   frame = DataFrame(randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])
+   frame = pd.DataFrame(np.random.randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])
    frame.cov()
 
 ``DataFrame.cov`` also supports an optional ``min_periods`` keyword that
@@ -87,7 +86,7 @@ in order to have a valid result.
 
 .. ipython:: python
 
-   frame = DataFrame(randn(20, 3), columns=['a', 'b', 'c'])
+   frame = pd.DataFrame(np.random.randn(20, 3), columns=['a', 'b', 'c'])
    frame.ix[:5, 'a'] = np.nan
    frame.ix[5:10, 'b'] = np.nan
 
@@ -123,7 +122,7 @@ All of these are currently computed using pairwise complete observations.
 
 .. ipython:: python
 
-   frame = DataFrame(randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])
+   frame = pd.DataFrame(np.random.randn(1000, 5), columns=['a', 'b', 'c', 'd', 'e'])
    frame.ix[::2] = np.nan
 
    # Series with Series
@@ -140,7 +139,7 @@ Like ``cov``, ``corr`` also supports the optional ``min_periods`` keyword:
 
 .. ipython:: python
 
-   frame = DataFrame(randn(20, 3), columns=['a', 'b', 'c'])
+   frame = pd.DataFrame(np.random.randn(20, 3), columns=['a', 'b', 'c'])
    frame.ix[:5, 'a'] = np.nan
    frame.ix[5:10, 'b'] = np.nan
 
@@ -157,8 +156,8 @@ objects.
 
    index = ['a', 'b', 'c', 'd', 'e']
    columns = ['one', 'two', 'three', 'four']
-   df1 = DataFrame(randn(5, 4), index=index, columns=columns)
-   df2 = DataFrame(randn(4, 4), index=index[:4], columns=columns)
+   df1 = pd.DataFrame(np.random.randn(5, 4), index=index, columns=columns)
+   df2 = pd.DataFrame(np.random.randn(4, 4), index=index[:4], columns=columns)
    df1.corrwith(df2)
    df2.corrwith(df1, axis=1)
 
@@ -172,7 +171,7 @@ of the ranks (by default) for the group:
 
 .. ipython:: python
 
-   s = Series(np.random.randn(5), index=list('abcde'))
+   s = pd.Series(np.random.np.random.randn(5), index=list('abcde'))
    s['d'] = s['b'] # so there's a tie
    s.rank()
 
@@ -181,7 +180,7 @@ or the columns (``axis=1``). ``NaN`` values are excluded from the ranking.
 
 .. ipython:: python
 
-   df = DataFrame(np.random.randn(10, 6))
+   df = pd.DataFrame(np.random.np.random.randn(10, 6))
    df[4] = df[2][:5] # some ties
    df
    df.rank(1)
@@ -253,7 +252,7 @@ These functions can be applied to ndarrays or Series objects:
 
 .. ipython:: python
 
-   ts = Series(randn(1000), index=date_range('1/1/2000', periods=1000))
+   ts = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))
    ts = ts.cumsum()
 
    ts.plot(style='k--')
@@ -271,7 +270,7 @@ sugar for applying the moving window operator to all of the DataFrame's columns:
 
 .. ipython:: python
 
-   df = DataFrame(randn(1000, 4), index=ts.index,
+   df = pd.DataFrame(np.random.randn(1000, 4), index=ts.index,
                   columns=['A', 'B', 'C', 'D'])
    df = df.cumsum()
 
@@ -310,7 +309,7 @@ keyword. The list of recognized types are:
 
 .. ipython:: python
 
-   ser = Series(randn(10), index=date_range('1/1/2000', periods=10))
+   ser = pd.Series(np.random.randn(10), index=pd.date_range('1/1/2000', periods=10))
 
    rolling_window(ser, 5, 'triang')
 

doc/source/contributing.rst

@@ -112,8 +112,10 @@ want to clone your fork to your machine: ::
 This creates the directory `pandas-yourname` and connects your repository to
 the upstream (main project) *pandas* repository.
 
-You will also need to hook up Travis-CI to your GitHub repository so the suite
-is automatically run when a Pull Request is submitted.  Instructions are `here
+The testing suite will run automatically on Travis-CI once your Pull Request is
+submitted.  However, if you wish to run the test suite on a branch prior to 
+submitting the Pull Request, then Travis-CI needs to be hooked up to your
+GitHub repository.  Instructions are for doing so are `here
 <http://about.travis-ci.org/docs/user/getting-started/>`_.
 
 Creating a Branch
@@ -134,6 +136,17 @@ changes in this branch specific to one bug or feature so it is clear
 what the branch brings to *pandas*. You can have many shiny-new-features
 and switch in between them using the git checkout command.
 
+To update this branch, you need to retrieve the changes from the master branch::
+
+    git fetch upstream
+    git rebase upstream/master
+
+This will replay your commits on top of the lastest pandas git master.  If this
+leads to merge conflicts, you must resolve these before submitting your Pull
+Request.  If you have uncommitted changes, you will need to `stash` them prior
+to updating.  This will effectively store your changes and they can be reapplied
+after updating.
+
 .. _contributing.dev_env:
 
 Creating a Development Environment
@@ -338,7 +351,7 @@ dependencies.
 Building the documentation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-So how do you build the docs? Navigate to your local  the folder
+So how do you build the docs? Navigate to your local the folder
 ``pandas/doc/`` directory in the console and run::
 
     python make.py html
@@ -358,8 +371,9 @@ If you want to do a full clean build, do::
 
 Starting with 0.13.1 you can tell ``make.py`` to compile only a single section
 of the docs, greatly reducing the turn-around time for checking your changes.
-You will be prompted to delete `.rst` files that aren't required, since the
-last committed version can always be restored from git.
+You will be prompted to delete `.rst` files that aren't required.  This is okay
+since the prior version can be checked out from git, but make sure to 
+not commit the file deletions.
 
 ::
 
@@ -417,7 +431,7 @@ deprecation warnings where needed.
 Test-driven Development/Writing Code
 ------------------------------------
 
-*Pandas* is serious about `Test-driven Development (TDD)
+*Pandas* is serious about testing and strongly encourages individuals to embrace `Test-driven Development (TDD)
 <http://en.wikipedia.org/wiki/Test-driven_development>`_.
 This development process "relies on the repetition of a very short development cycle:
 first the developer writes an (initially failing) automated test case that defines a desired
@@ -550,8 +564,8 @@ Doing 'git status' again should give something like ::
     #       modified:   /relative/path/to/file-you-added.py
     #
 
-Finally, commit your changes to your local repository with an explanatory message.  An informal
-commit message format is in effect for the project.  Please try to adhere to it.  Here are
+Finally, commit your changes to your local repository with an explanatory message.  *Pandas*
+uses a convention for commit message prefixes and layout.  Here are
 some common prefixes along with general guidelines for when to use them:
 
     * ENH: Enhancement, new functionality

doc/source/ecosystem.rst

@@ -57,7 +57,7 @@ large data to thin clients.
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Hadley Wickham's `ggplot2 <http://ggplot2.org/>`__ is a foundational exploratory visualization package for the R language.
-Based on `"The Grammer of Graphics" <http://www.cs.uic.edu/~wilkinson/TheGrammarOfGraphics/GOG.html>`__ it
+Based on `"The Grammar of Graphics" <http://www.cs.uic.edu/~wilkinson/TheGrammarOfGraphics/GOG.html>`__ it
 provides a powerful, declarative and extremely general way to generate bespoke plots of any kind of data.
 It's really quite incredible. Various implementations to other languages are available,
 but a faithful implementation for python users has long been missing. Although still young

doc/source/index.rst.template

@@ -115,6 +115,7 @@ See the package overview for more detail about what's in the library.
     {%if not single -%}
     whatsnew
     install
+    contributing
     faq
     overview
     10min
@@ -149,7 +150,6 @@ See the package overview for more detail about what's in the library.
     api
     {% endif -%}
     {%if not single -%}
-    contributing
     internals
     release
     {% endif -%}

doc/source/install.rst

@@ -243,7 +243,7 @@ Optional Dependencies
 * `Cython <http://www.cython.org>`__: Only necessary to build development
   version. Version 0.19.1 or higher.
 * `SciPy <http://www.scipy.org>`__: miscellaneous statistical functions
-* `PyTables <http://www.pytables.org>`__: necessary for HDF5-based storage. Version 3.0.0 or higher required.
+* `PyTables <http://www.pytables.org>`__: necessary for HDF5-based storage. Version 3.0.0 or higher required, Version 3.2.0 or higher highly recommended.
 * `SQLAlchemy <http://www.sqlalchemy.org>`__: for SQL database support. Version 0.8.1 or higher recommended.
 * `matplotlib <http://matplotlib.sourceforge.net/>`__: for plotting
 * `statsmodels <http://statsmodels.sourceforge.net/>`__
@@ -255,6 +255,7 @@ Optional Dependencies
    * Alternative Excel writer.
 * `boto <https://pypi.python.org/pypi/boto>`__: necessary for Amazon S3
   access.
+* `blosc <https://pypi.python.org/pypi/blosc`__: for msgpack compression using ``blosc``
 * One of `PyQt4
   <http://www.riverbankcomputing.com/software/pyqt/download>`__, `PySide
   <http://qt-project.org/wiki/Category:LanguageBindings::PySide>`__, `pygtk

doc/source/internals.rst

@@ -94,8 +94,7 @@ not check (or care) whether the levels themselves are sorted. Fortunately, the
 constructors ``from_tuples`` and ``from_arrays`` ensure that this is true, but
 if you compute the levels and labels yourself, please be careful.
 
-
-.. _:
+.. _ref-subclassing-pandas:
 
 Subclassing pandas Data Structures
 ----------------------------------

doc/source/io.rst

@@ -2364,6 +2364,10 @@ for some advanced strategies
 
    As of version 0.15.0, pandas requires ``PyTables`` >= 3.0.0. Stores written with prior versions of pandas / ``PyTables`` >= 2.3 are fully compatible (this was the previous minimum ``PyTables`` required version).
 
+.. warning::
+   
+   There is a ``PyTables`` indexing bug which may appear when querying stores using an index.  If you see a subset of results being returned, upgrade to ``PyTables`` >= 3.2.  Stores created previously will need to be rewritten using the updated version.
+
 .. ipython:: python
    :suppress:
    :okexcept:
@@ -3996,6 +4000,24 @@ whether imported ``Categorical`` variables are ordered.
     a ``Categorial`` with string categories for the values that are labeled and
     numeric categories for values with no label.
 
+.. _io.other:
+
+Other file formats
+------------------
+
+pandas itself only supports IO with a limited set of file formats that map
+cleanly to its tabular data model. For reading and writing other file formats
+into and from pandas, we recommend these packages from the broader community.
+
+netCDF
+~~~~~~
+
+xray_ provides data structures inspired by the pandas DataFrame for working
+with multi-dimensional datasets, with a focus on the netCDF file format and
+easy conversion to and from pandas.
+
+.. _xray: http://xray.readthedocs.org/
+
 .. _io.perf:
 
 Performance Considerations