Merge remote-tracking branch 'upstream/master'

Author: Artemy Kolchinsky

.travis.yml

@@ -54,7 +54,7 @@ matrix:
       - JOB_NAME: "34_nslow"
     - python: 3.2
       env:
-      - NOSE_ARGS="not slow and not disabled"
+      - NOSE_ARGS="not slow and not network and not disabled"
       - FULL_DEPS=true
       - CLIPBOARD_GUI=qt4
       - BUILD_TYPE=pydata
@@ -71,7 +71,7 @@ matrix:
     allow_failures:
       - python: 3.2
         env:
-        - NOSE_ARGS="not slow and not disabled"
+        - NOSE_ARGS="not slow and not network and not disabled"
         - FULL_DEPS=true
         - CLIPBOARD_GUI=qt4
         - BUILD_TYPE=pydata

ci/requirements-3.2.txt

@@ -9,6 +9,7 @@ tables==3.0.0
 matplotlib==1.2.1
 patsy==0.1.0
 lxml==3.2.1
+html5lib
 scipy==0.12.0
 beautifulsoup4==4.2.1
 statsmodels==0.5.0

doc/README.rst

@@ -88,13 +88,12 @@ Furthermore, it is recommended to have all `optional dependencies
 installed. This is not needed, but be aware that you will see some error
 messages. Because all the code in the documentation is executed during the doc
 build, the examples using this optional dependencies will generate errors.
-Run ``pd.show_version()`` to get an overview of the installed version of all
+Run ``pd.show_versions()`` to get an overview of the installed version of all
 dependencies.
 
 .. warning::
 
-   Building the docs with Sphinx version 1.2 is broken. Use the
-   latest stable version (1.2.1) or the older 1.1.3.
+   Sphinx version >= 1.2.2 or the older 1.1.3 is required.
 
 Building pandas
 ^^^^^^^^^^^^^^^

doc/source/api.rst

@@ -190,6 +190,8 @@ Standard moving window functions
    rolling_quantile
    rolling_window
 
+.. _api.functions_expanding:
+
 Standard expanding window functions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -1164,6 +1166,8 @@ Attributes
 
    Index.values
    Index.is_monotonic
+   Index.is_monotonic_increasing
+   Index.is_monotonic_decreasing
    Index.is_unique
    Index.dtype
    Index.inferred_type

doc/source/categorical.rst

@@ -236,7 +236,7 @@ which are removed are replaced by ``np.nan``.:
     s = s.cat.remove_categories([4])
     s
 
-Renaming unused categories
+Removing unused categories
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Removing unused categories can also be done:
@@ -573,6 +573,7 @@ relevant columns back to `category` and assign the right categories and categori
     df2.dtypes
     df2["cats"]
 
+The same holds for writing to a SQL database with ``to_sql``.
 
 Missing Data
 ------------

doc/source/conf.py

@@ -44,12 +44,9 @@
               'ipython_sphinxext.ipython_directive',
               'ipython_sphinxext.ipython_console_highlighting',
               'sphinx.ext.intersphinx',
-              'sphinx.ext.todo',
               'sphinx.ext.coverage',
               'sphinx.ext.pngmath',
               'sphinx.ext.ifconfig',
-              'matplotlib.sphinxext.only_directives',
-              'matplotlib.sphinxext.plot_directive',
               ]
 
 

doc/source/cookbook.rst

@@ -588,6 +588,19 @@ Unlike agg, apply's callable is passed a sub-DataFrame which gives you access to
    df['beyer_shifted'] = df.groupby(level=0)['beyer'].shift(1)
    df
 
+`Select row with maximum value from each group
+<http://stackoverflow.com/q/26701849/190597>`__
+
+.. ipython:: python
+
+   df = pd.DataFrame({'host':['other','other','that','this','this'],
+                      'service':['mail','web','mail','mail','web'],
+                      'no':[1, 2, 1, 2, 1]}).set_index(['host', 'service'])
+   mask = df.groupby(level=0).agg('idxmax')
+   df_count = df.loc[mask['no']].reset_index()
+   df_count
+
+
 Expanding Data
 **************
 
@@ -1202,4 +1215,4 @@ of the data values:
       {'height': [60, 70],
        'weight': [100, 140, 180],
        'sex': ['Male', 'Female']})
-   df
+   df

doc/source/io.rst

@@ -3337,6 +3337,14 @@ With some databases, writing large DataFrames can result in errors due to packet
     flavors, columns with type ``timedelta64`` will be written as integer
     values as nanoseconds to the database and a warning will be raised.
 
+.. note::
+
+    Columns of ``category`` dtype will be converted to the dense representation
+    as you would get with ``np.asarray(categorical)`` (e.g. for string categories
+    this gives an array of strings).
+    Because of this, reading the database table back in does **not** generate
+    a categorical.
+
 
 Reading Tables
 ~~~~~~~~~~~~~~
@@ -3618,12 +3626,18 @@ outside of this range, the data is cast to ``int16``.
    if ``int64`` values are larger than 2**53.
 
 .. warning::
+
   :class:`~pandas.io.stata.StataWriter`` and
   :func:`~pandas.core.frame.DataFrame.to_stata` only support fixed width
   strings containing up to 244 characters, a limitation imposed by the version
   115 dta file format. Attempting to write *Stata* dta files with strings
   longer than 244 characters raises a ``ValueError``.
 
+.. warning::
+
+  *Stata* data files only support text labels for categorical data.  Exporting
+  data frames containing categorical data will convert non-string categorical values
+  to strings.
 
 .. _io.stata_reader:
 

doc/source/options.rst

@@ -86,7 +86,7 @@ pandas namespace.  To change an option, call ``set_option('option regex', new_va
    pd.set_option('mode.sim_interactive', True)
    pd.get_option('mode.sim_interactive')
 
-**Note:** that the option 'mode.sim_interactive' is mostly used for debugging purposes.  
+**Note:** that the option 'mode.sim_interactive' is mostly used for debugging purposes.
 
 All options also have a default value, and you can use ``reset_option`` to do just that:
 
@@ -213,7 +213,8 @@ will be given.
 
 ``display.max_info_rows``: ``df.info()`` will usually show null-counts for each column.
 For large frames this can be quite slow. ``max_info_rows`` and ``max_info_cols``
-limit this null check only to frames with smaller dimensions then specified.
+limit this null check only to frames with smaller dimensions then specified. Note that you
+can specify the option ``df.info(null_counts=True)`` to override on showing a particular frame.
 
 .. ipython:: python
 

doc/source/release.rst

@@ -45,22 +45,57 @@ analysis / manipulation tool available in any language.
 * Binary installers on PyPI: http://pypi.python.org/pypi/pandas
 * Documentation: http://pandas.pydata.org
 
+pandas 0.15.2
+-------------
+
+**Release date:** (December ??, 2014)
+
+This is a minor release from 0.15.1 and includes a small number of API changes, several new features, enhancements, and
+performance improvements along with a large number of bug fixes.
+
+See the :ref:`v0.15.2 Whatsnew <whatsnew_0152>` overview for an extensive list
+of all API changes, enhancements and bugs that have been fixed in 0.15.2.
+
+Thanks
+~~~~~~
+
 pandas 0.15.1
 -------------
 
-**Release date:** (November ??, 2014)
+**Release date:** (November 9, 2014)
 
 This is a minor release from 0.15.0 and includes a small number of API changes, several new features, enhancements, and
 performance improvements along with a large number of bug fixes.
 
-Highlights include:
-
-See the :ref:`v0.15.1 Whatsnew <whatsnew_0151>` overview or the issue tracker on GitHub for an extensive list
+See the :ref:`v0.15.1 Whatsnew <whatsnew_0151>` overview for an extensive list
 of all API changes, enhancements and bugs that have been fixed in 0.15.1.
 
 Thanks
 ~~~~~~
 
+- Aaron Staple
+- Andrew Rosenfeld
+- Anton I. Sipos
+- Artemy Kolchinsky
+- Bill Letson
+- Dave Hughes
+- David Stephens
+- Guillaume Horel
+- Jeff Reback
+- Joris Van den Bossche
+- Kevin Sheppard
+- Nick Stahl
+- Sanghee Kim
+- Stephan Hoyer
+- TomAugspurger
+- WANG Aiyong
+- behzad nouri
+- immerrr
+- jnmclarty
+- jreback
+- pallav-fdsi
+- unutbu
+
 pandas 0.15.0
 -------------
 

doc/source/remote_data.rst

@@ -86,8 +86,29 @@ If you don't want to download all the data, more specific requests can be made.
       data = aapl.get_call_data(expiry=expiry)
       data.iloc[0:5:, 0:5]
 
-Note that if you call ``get_all_data`` first, this second call will happen much faster, as the data is cached.
+Note that if you call ``get_all_data`` first, this second call will happen much faster,
+as the data is cached.
 
+If a given expiry date is not available, data for the next available expiry will be
+returned (January 15, 2015 in the above example).
+
+Available expiry dates can be accessed from the ``expiry_dates`` property.
+
+.. ipython:: python
+
+      aapl.expiry_dates
+      data = aapl.get_call_data(expiry=aapl.expiry_dates[0])
+      data.iloc[0:5:, 0:5]
+
+A list-like object containing dates can also be passed to the expiry parameter,
+returning options data for all expiry dates in the list.
+
+.. ipython:: python
+
+      data = aapl.get_near_stock_price(expiry=aapl.expiry_dates[0:3])
+      data.iloc[0:5:, 0:5]
+
+The ``month`` and ``year`` parameters can be used to get all options data for a given month.
 
 .. _remote_data.google:
 
@@ -143,6 +164,12 @@ World Bank
 `World Bank's World Development Indicators <http://data.worldbank.org>`__
 by using the ``wb`` I/O functions.
 
+Indicators
+~~~~~~~~~~
+
+Either from exploring the World Bank site, or using the search function included,
+every world bank indicator is accessible.  
+
 For example, if you wanted to compare the Gross Domestic Products per capita in
 constant dollars in North America, you would use the ``search`` function:
 
@@ -254,3 +281,56 @@ populations in rich countries tend to use cellphones at a higher rate:
     Skew:                          -2.314   Prob(JB):                     1.35e-26
     Kurtosis:                      11.077   Cond. No.                         45.8
     ==============================================================================
+
+Country Codes
+~~~~~~~~~~~~~
+
+.. versionadded:: 0.15.1
+
+The ``country`` argument accepts a string or list of mixed 
+`two <http://en.wikipedia.org/wiki/ISO_3166-1_alpha-2>`__ or `three <http://en.wikipedia.org/wiki/ISO_3166-1_alpha-3>`__ character
+ISO country codes, as well as dynamic `World Bank exceptions <http://data.worldbank.org/node/18>`__ to the ISO standards.
+
+For a list of the the hard-coded country codes (used solely for error handling logic) see ``pandas.io.wb.country_codes``.
+
+Problematic Country Codes & Indicators
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+
+   The World Bank's country list and indicators are dynamic. As of 0.15.1, 
+   :func:`wb.download()` is more flexible.  To achieve this, the warning
+   and exception logic changed.
+   
+The world bank converts some country codes,
+in their response, which makes error checking by pandas difficult.
+Retired indicators still persist in the search.
+
+Given the new flexibility of 0.15.1, improved error handling by the user
+may be necessary for fringe cases.
+
+To help identify issues:
+
+There are at least 4 kinds of country codes:
+
+1. Standard (2/3 digit ISO) - returns data, will warn and error properly.
+2. Non-standard (WB Exceptions) - returns data, but will falsely warn.
+3. Blank - silently missing from the response.
+4. Bad - causes the entire response from WB to fail, always exception inducing.
+
+There are at least 3 kinds of indicators:
+
+1. Current - Returns data.
+2. Retired - Appears in search results, yet won't return data. 
+3. Bad - Will not return data.
+
+Use the ``errors`` argument to control warnings and exceptions.  Setting
+errors to ignore or warn, won't stop failed responses.  (ie, 100% bad
+indicators, or a single "bad" (#4 above) country code).  
+
+See docstrings for more info.
+
+
+
+
+

doc/source/tutorials.rst

@@ -109,6 +109,21 @@ For more resources, please visit the main `repository <https://bitbucket.org/hro
   - Combining data from various sources
 
 
+Practical data analysis with Python
+-----------------------------------
+
+This `guide <http://wavedatalab.github.io/datawithpython>`_ is a comprehensive introduction to the data analysis process using the Python data ecosystem and an interesting open dataset.
+There are four sections covering selected topics as follows:
+
+-  `Munging Data <http://wavedatalab.github.io/datawithpython/munge.html>`_
+
+-  `Aggregating Data <http://wavedatalab.github.io/datawithpython/aggregate.html>`_
+
+-  `Visualizing Data <http://wavedatalab.github.io/datawithpython/visualize.html>`_
+
+-  `Time Series <http://wavedatalab.github.io/datawithpython/timeseries.html>`_
+
+
 Excel charts with pandas, vincent and xlsxwriter
 ------------------------------------------------
 

doc/source/whatsnew.rst

@@ -18,6 +18,8 @@ What's New
 
 These are new features and improvements of note in each release.
 
+.. include:: whatsnew/v0.15.2.txt
+
 .. include:: whatsnew/v0.15.1.txt
 
 .. include:: whatsnew/v0.15.0.txt

doc/source/whatsnew/v0.10.0.txt

@@ -48,6 +48,7 @@ want to broadcast, we are phasing out this special case (Zen of Python:
 talking about:
 
 .. ipython:: python
+   :okwarning:
 
    import pandas as pd
    df = pd.DataFrame(np.random.randn(6, 4),