Merge commit 'v0.10.0b1-65-g6cadd6c' into debian

* commit 'v0.10.0b1-65-g6cadd6c':
  BUG: workaround numpy issue in 1.6.2 on certain distros
  BUG: use "for" as keyword to test against
  ENH: option names must valid, non-keyword python identifiers
  TST: option names must valid, non-keyword python identifiers
  BUG: make sure pd.options style access triggers warnings and callbacks as needed
  ENH: provide attribute style access to options via pd.options
  REF: rename option key "print" to "display"
  BUG: use Series name attributes for colnames in concat with axis=1. close pandas-dev#2489
  ENH: add max_cols as keyword in DataFrame.info pandas-dev#2524
  BUG: allow users to set the max number of columns before per column info is hidden away pandas-dev#2524
  BUG: DataFrame.from_dict does not work with dict of sequence and orient=index pandas-dev#2496
  ENH: df.select uses bool(crit(x)) rather then crit(x)
  DOC: add FAQ section on monkey-patching
  BUG: diff should cast n to int pandas-dev#2523

Author: yarikoptic

RELEASE.rst

@@ -73,7 +73,8 @@ pandas 0.10.0
     backfilling time series data (GH2284_)
   - New option configuration system and functions `set_option`, `get_option`,
     `describe_option`, and `reset_option`. Deprecate `set_printoptions` and
-    `reset_printoptions` (GH2393_)
+    `reset_printoptions` (GH2393_).
+    You can also access options as attributes via ``pandas.options.X``
   - Wide DataFrames can be viewed more easily in the console with new
     `expand_frame_repr` and `line_width` configuration options. This is on by
     default now (GH2436_)
@@ -170,6 +171,7 @@ pandas 0.10.0
   - Optimize ``unstack`` memory usage by compressing indices (GH2278_)
   - Fix HTML repr in IPython qtconsole if opening window is small (GH2275_)
   - Escape more special characters in console output (GH2492_)
+  - df.select now invokes bool on the result of crit(x) (GH2487_)
 
 **Bug fixes**
 
@@ -234,6 +236,7 @@ pandas 0.10.0
   - DataFrame.combine_first will always result in the union of the index and
     columns, even if one DataFrame is length-zero (GH2525_)
   - Fix several DataFrame.icol/irow with duplicate indices issues (GH2228_, GH2259_)
+  - Use Series names for column names when using concat with axis=1 (GH2489_)
 
 .. _GH407: https://github.com/pydata/pandas/issues/407
 .. _GH821: https://github.com/pydata/pandas/issues/821
@@ -297,6 +300,7 @@ pandas 0.10.0
 .. _GH2447: https://github.com/pydata/pandas/issues/2447
 .. _GH2275: https://github.com/pydata/pandas/issues/2275
 .. _GH2492: https://github.com/pydata/pandas/issues/2492
+.. _GH2487: https://github.com/pydata/pandas/issues/2487
 .. _GH2273: https://github.com/pydata/pandas/issues/2273
 .. _GH2266: https://github.com/pydata/pandas/issues/2266
 .. _GH2038: https://github.com/pydata/pandas/issues/2038
@@ -351,6 +355,7 @@ pandas 0.10.0
 .. _GH2525: https://github.com/pydata/pandas/issues/2525
 .. _GH2228: https://github.com/pydata/pandas/issues/2228
 .. _GH2259: https://github.com/pydata/pandas/issues/2259
+.. _GH2489: https://github.com/pydata/pandas/issues/2489
 
 
 pandas 0.9.1

doc/source/basics.rst

@@ -1049,34 +1049,49 @@ Working with package options
 .. _basics.working_with_options:
 
 Introduced in 0.10.0, pandas supports a new system for working with options.
-The 4 relavent functions are available directly from the ``pandas`` namespace,
-and they are:
+Options have a full "dotted-style", case-insensitive name (e.g. ``display.max_rows``),
+
+You can get/set options directly as attributes of the top-level ``options`` attribute:
+
+.. ipython:: python
+
+   import pandas as pd
+   pd.options.display.max_rows
+   pd.options.display.max_rows = 999
+   pd.options.display.max_rows
+
+
+There is also an API composed of 4 relavent functions, available directly from the ``pandas``
+namespace, and they are:
 
 - ``get_option`` / ``set_option`` - get/set the value of a single option.
 - ``reset_option`` - reset one or more options to their default value.
 - ``describe_option`` - print the descriptions of one or more options.
 
 **Note:** developers can check out pandas/core/config.py for more info.
 
-Options have a full "dotted-style", case-insensitive name (e.g. ``print.max_rows``),
+
 but all of the functions above accept a regexp pattern (``re.search`` style) as argument,
 so passing in a substring will work - as long as it is unambiguous :
 
 .. ipython:: python
 
-   get_option("print.max_rows")
-   set_option("print.max_rows",101)
-   get_option("print.max_rows")
+   get_option("display.max_rows")
+   set_option("display.max_rows",101)
+   get_option("display.max_rows")
    set_option("max_r",102)
-   get_option("print.max_rows")
+   get_option("display.max_rows")
 
 
-However, the following will **not work** because it matches multiple option names, e.g.``print.max_colwidth``, ``print.max_rows``, ``print.max_columns``:
+However, the following will **not work** because it matches multiple option names, e.g.``display.max_colwidth``, ``display.max_rows``, ``display.max_columns``:
 
 .. ipython:: python
    :okexcept:
 
-   get_option("print.max_")
+   try:
+       get_option("display.max_")
+   except KeyError as e:
+       print(e)
 
 
 **Note:** Using this form of convenient shorthand may make your code break if new options with similar names are added in future versions.
@@ -1103,23 +1118,23 @@ All options also have a default value, and you can use the ``reset_option`` to d
 .. ipython:: python
    :suppress:
 
-   reset_option("print.max_rows")
+   reset_option("display.max_rows")
 
 
 .. ipython:: python
 
-   get_option("print.max_rows")
-   set_option("print.max_rows",999)
-   get_option("print.max_rows")
-   reset_option("print.max_rows")
-   get_option("print.max_rows")
+   get_option("display.max_rows")
+   set_option("display.max_rows",999)
+   get_option("display.max_rows")
+   reset_option("display.max_rows")
+   get_option("display.max_rows")
 
 
 and you also set multiple options at once:
 
 .. ipython:: python
 
-   reset_option("^print\.")
+   reset_option("^display\.")
 
 
 

doc/source/faq.rst

@@ -21,6 +21,47 @@ Frequently Asked Questions (FAQ)
    import matplotlib.pyplot as plt
    plt.close('all')
 
+.. _ref-monkey-patching:
+
+
+----------------------------------------------------
+
+Pandas is a powerful tool and already has a plethora of data manipulation
+operations implemented, most of them are very fast as well.
+It's very possible however that certain functionality that would make your
+life easier is missing. In that case you have several options:
+
+1) Open an issue on `Github <https://github.com/pydata/pandas/issues/>`_ , explain your need and the sort of functionality you would like to see implemented.
+2) Fork the repo, Implement the functionality yourself and open a PR
+   on Github.
+3) Write a method that performs the operation you are interested in and
+   Monkey-patch the pandas class as part of your IPython profile startup
+   or PYTHONSTARTUP file.
+
+   For example, here is an example of adding an ``just_foo_cols()``
+   method to the dataframe class:
+
+.. ipython:: python
+
+   import pandas as pd
+   def just_foo_cols(self):
+       """Get a list of column names containing the string 'foo'
+
+       """
+       return [x for x in self.columns if 'foo' in x]
+
+   pd.DataFrame.just_foo_cols = just_foo_cols # monkey-patch the DataFrame class
+   df = pd.DataFrame([range(4)],columns= ["A","foo","foozball","bar"])
+   df.just_foo_cols()
+   del pd.DataFrame.just_foo_cols # you can also remove the new method
+
+
+Monkey-patching is usually frowned upon because it makes your code
+less portable and can cause subtle bugs in some circumstances.
+Monkey-patching existing methods is usually a bad idea in that respect.
+When used with proper care, however, it's a very useful tool to have.
+
+
 .. _ref-scikits-migration:
 
 Migrating from scikits.timeseries to pandas >= 0.8.0
@@ -171,5 +212,3 @@ interval (``'start'`` or ``'end'``) convention:
    data = Series(np.random.randn(50), index=rng)
    resampled = data.resample('A', kind='timestamp', convention='end')
    resampled.index
-
-

doc/source/v0.10.0.txt

@@ -160,11 +160,11 @@ Convenience methods ``ffill`` and  ``bfill`` have been added:
     arguments. print all registered options.
 
   Note: ``set_printoptions``/ ``reset_printoptions`` are now deprecated (but
-  functioning), the print options now live under "print.XYZ". For example:
+  functioning), the print options now live under "display.XYZ". For example:
 
   .. ipython:: python
 
-     get_option("print.max_rows")
+     get_option("display.max_rows")
 
 - to_string() methods now always return unicode strings  (GH2224_).
 
@@ -284,7 +284,7 @@ Updated PyTables Support
       df['string'] = 'string'
       df['int']    = 1
       store.append('df',df)
-      df1 = store.select('df') 
+      df1 = store.select('df')
       df1
       df1.get_dtype_counts()
 

pandas/core/api.py

@@ -31,4 +31,4 @@
 import pandas.core.datetools as datetools
 
 from pandas.core.config import get_option,set_option,reset_option,\
-                               describe_option
+                               describe_option, options

pandas/core/common.py

@@ -455,6 +455,7 @@ def mask_out_axis(arr, mask, axis, fill_value=np.nan):
 }
 
 def diff(arr, n, axis=0):
+    n = int(n)
     dtype = arr.dtype
     if issubclass(dtype.type, np.integer):
         dtype = np.float64
@@ -1212,7 +1213,7 @@ def in_qtconsole():
 # 2) If you need to send something to the console, use console_encode().
 #
 #    console_encode() should (hopefully) choose the right encoding for you
-#    based on the encoding set in option "print.encoding"
+#    based on the encoding set in option "display.encoding"
 #
 # 3) if you need to write something out to file, use
 #    pprint_thing_encoded(encoding).
@@ -1271,10 +1272,10 @@ def pprint_thing(thing, _nest_lvl=0, escape_chars=None):
          hasattr(thing,'next'):
         return unicode(thing)
     elif (isinstance(thing, dict) and
-          _nest_lvl < get_option("print.pprint_nest_depth")):
+          _nest_lvl < get_option("display.pprint_nest_depth")):
         result = _pprint_dict(thing, _nest_lvl)
     elif _is_sequence(thing) and _nest_lvl < \
-		get_option("print.pprint_nest_depth"):
+		get_option("display.pprint_nest_depth"):
         result = _pprint_seq(thing, _nest_lvl, escape_chars=escape_chars)
     else:
         # when used internally in the package, everything
@@ -1312,8 +1313,8 @@ def console_encode(object, **kwds):
     this is the sanctioned way to prepare something for
     sending *to the console*, it delegates to pprint_thing() to get
     a unicode representation of the object relies on the global encoding
-    set in print.encoding. Use this everywhere
+    set in display.encoding. Use this everywhere
     where you output to the console.
     """
     return pprint_thing_encoded(object,
-             get_option("print.encoding"))
+             get_option("display.encoding"))

pandas/core/config.py

@@ -138,6 +138,39 @@ def _reset_option(pat):
     for k in keys:
         _set_option(k, _registered_options[k].defval)
 
+class DictWrapper(object):
+    """ provide attribute-style access to a nested dict
+    """
+    def __init__(self,d,prefix=""):
+        object.__setattr__(self,"d",d)
+        object.__setattr__(self,"prefix",prefix)
+
+    def __setattr__(self,key,val):
+        prefix = object.__getattribute__(self,"prefix")
+        if prefix:
+            prefix += "."
+        prefix += key
+        # you can't set new keys
+        # can you can't overwrite subtrees
+        if key in self.d and not isinstance(self.d[key],dict):
+            _set_option(prefix,val)
+            self.d[key]=val
+        else:
+            raise KeyError("You can only set the value of existing options")
+
+    def __getattr__(self,key):
+        prefix = object.__getattribute__(self,"prefix")
+        if prefix:
+            prefix += "."
+        prefix += key
+        v=object.__getattribute__(self,"d")[key]
+        if isinstance(v,dict):
+            return DictWrapper(v,prefix)
+        else:
+            return _get_option(prefix)
+
+    def __dir__(self):
+        return self.d.keys()
 
 # For user convenience,  we'd like to have the available options described
 # in the docstring. For dev convenience we'd like to generate the docstrings
@@ -266,7 +299,7 @@ def __doc__(self):
 set_option = CallableDyanmicDoc(_set_option, _set_option_tmpl)
 reset_option = CallableDyanmicDoc(_reset_option, _reset_option_tmpl)
 describe_option = CallableDyanmicDoc(_describe_option, _describe_option_tmpl)
-
+options = DictWrapper(_global_config)
 
 ######################################################
 # Functions for use by pandas developers, in addition to User - api
@@ -316,7 +349,7 @@ def register_option(key, defval, doc='', validator=None, cb=None):
     ValueError if `validator` is specified and `defval` is not a valid value.
 
     """
-
+    import tokenize, keyword
     key = key.lower()
 
     if key in _registered_options:
@@ -330,6 +363,13 @@ def register_option(key, defval, doc='', validator=None, cb=None):
 
     # walk the nested dict, creating dicts as needed along the path
     path = key.split('.')
+
+    for k in path:
+        if not bool(re.match('^'+tokenize.Name+'$', k)):
+            raise ValueError("%s is not a valid identifier" % k)
+        if keyword.iskeyword(key):
+            raise ValueError("%s is a python keyword" % k)
+
     cursor = _global_config
     for i, p in enumerate(path[:-1]):
         if not isinstance(cursor, dict):
@@ -343,6 +383,7 @@ def register_option(key, defval, doc='', validator=None, cb=None):
         raise KeyError("Path prefix to option '%s' is already an option"
                        % '.'.join(path[:-1]))
 
+
     cursor[path[-1]] = defval  # initialize
 
     # save the option metadata

pandas/core/config_init.py

@@ -16,7 +16,7 @@
 
 
 ###########################################
-# options from the "print" namespace
+# options from the "display" namespace
 
 pc_precision_doc="""
 : int
@@ -45,6 +45,12 @@
     columns that can fit on it.
 """
 
+pc_max_info_cols_doc="""
+: int
+    max_info_columns is used in DataFrame.info method to decide if
+    per column information will be printed.
+"""
+
 pc_nb_repr_h_doc="""
 : boolean
     When True (default), IPython notebook will use html representation for
@@ -115,13 +121,15 @@
     When printing wide DataFrames, this is the width of each line.
 """
 
-with cf.config_prefix('print'):
+with cf.config_prefix('display'):
     cf.register_option('precision', 7, pc_precision_doc, validator=is_int)
     cf.register_option('float_format', None, float_format_doc)
     cf.register_option('column_space', 12, validator=is_int)
     cf.register_option('max_rows', 100, pc_max_rows_doc, validator=is_int)
     cf.register_option('max_colwidth', 50, max_colwidth_doc, validator=is_int)
     cf.register_option('max_columns', 20, pc_max_cols_doc, validator=is_int)
+    cf.register_option('max_info_columns', 100, pc_max_info_cols_doc,
+                       validator=is_int)
     cf.register_option('colheader_justify', 'right', colheader_justify_doc,
                        validator=is_text)
     cf.register_option('notebook_repr_html', True, pc_nb_repr_h_doc,