Merge commit 'v0.7.0rc1-29-g4d29e47' into debian

* commit 'v0.7.0rc1-29-g4d29e47': (26 commits)
  BUG: fix indexing corner case with MultiIndex re: pandas-dev#671
  BUG: fix integer-slicing from integers-as-floats, GH pandas-dev#670
  DOC: added release notes, docs for pandas-dev#288, pandas-dev#647
  ENH: can pass sequence of integers to DataFrame.{irow/icol} and Series.iget, GH pandas-dev#654"
  BUG: pass on sort kind from Series.sort to order and in argsort GH pandas-dev#668
  BUG: make groupby play nice with sparse objects, modify SparseSeries.take to return SparseSeries, dictification tests, GH pandas-dev#666
  TST: added tests for cummin, cummax. closes pandas-dev#647
  Update pandas/core/generic.py
  * Added cummax and cummin methods for Series And DataFrame.
  ENH: closes pandas-dev#288
  DOC: gotcha docs re: pandas-dev#656
  BUG: week as top-level iterator overwrites week instance
  TST: test behavior of passing None to Series constructor
  ENH: string format None and None and not NaN
  ENH: break circular reference causing memory leak in sparse array / series / frame, GH pandas-dev#663
  DOC: release notes
  ENH: handle list of values intelligently as grouping array when possible, GH pandas-dev#659
  BUG: don't lose columns name when passing list of labels to DataFrame.__getitem__, GH pandas-dev#662
  BUG: handling of tuples in MultiIndex level
  ENH: try to convert dtypes in Index.map
  ...

Author: yarikoptic

RELEASE.rst

@@ -78,8 +78,11 @@ pandas 0.7.0
     yielding an aggregated result with hierarchical columns (GH #166)
   - Add integer-indexing functions ``iget`` in Series and ``irow`` / ``iget``
     in DataFrame (GH #628)
-  - Add automatic realignment functionality (when possible) to comparisons and
-    logical operators for Series
+  - Add new ``Series.unique`` function, significantly faster than
+    ``numpy.unique`` (GH #658)
+  - Add new ``cummin`` and ``cummax`` instance methods to ``Series`` and
+    ``DataFrame`` (GH #647)
+  - Add new ``value_range`` function to return min/max of a dataframe (GH #288)
 
 **API Changes**
 
@@ -155,6 +158,10 @@ pandas 0.7.0
     row/column the function application failed on (GH #614)
   - Improved ability of read_table and read_clipboard to parse
     console-formatted DataFrames (can read the row of index names, etc.)
+  - Can pass list of group labels (without having to convert to an ndarray
+    yourself) to ``groupby`` in some cases (GH #659)
+  - Use ``kind`` argument to Series.order for selecting different sort kinds
+    (GH #668)
 
 **Bug fixes**
 
@@ -220,6 +227,10 @@ pandas 0.7.0
   - Add support for legacy WidePanel objects to be read from HDFStore
   - Fix out-of-bounds segfault in pad_object and backfill_object methods when
     either source or target array are empty
+  - Could not create a new column in a DataFrame from a list of tuples
+  - Fix bugs preventing SparseDataFrame and SparseSeries working with groupby
+    (GH #666)
+  - Use sort kind in Series.sort / argsort (GH #668)
 
 Thanks
 ------

TODO.rst

@@ -1,7 +1,8 @@
 DOCS 0.7.0
 ----------
-- no sort in groupby
-- concat with dict
+- ??? no sort in groupby
+- DONE concat with dict
+- Gotchas re: integer indexing
 
 DONE
 ----

doc/source/basics.rst

@@ -285,6 +285,8 @@ optional ``level`` parameter which applies only if the object has a
     ``quantile``, Sample quantile (value at %)
     ``cumsum``, Cumulative sum
     ``cumprod``, Cumulative product
+    ``cummax``, Cumulative maximum
+    ``cummin``, Cumulative minimum
 
 Note that by chance some NumPy methods, like ``mean``, ``std``, and ``sum``,
 will exclude NAs on Series input by default:
@@ -332,6 +334,9 @@ number of unique values and most frequently occurring values:
    s = Series(['a', 'a', 'b', 'b', 'a', 'a', np.nan, 'c', 'd', 'a'])
    s.describe()
 
+There also is a utility function, ``value_range`` which takes a DataFrame and
+returns a series with the minimum/maximum values in the DataFrame.
+
 .. _basics.idxmin:
 
 Index of Min/Max Values

doc/source/conf.py

@@ -61,7 +61,7 @@
 
 # General information about the project.
 project = u'pandas'
-copyright = u'2008-2011, AQR and Wes McKinney'
+copyright = u'2008-2011, the pandas development team'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the

doc/source/gotchas.rst

@@ -80,3 +80,58 @@ specific dates. To enable this, we made the design design to make label-based sl
 This is most definitely a "practicality beats purity" sort of thing, but it is
 something to watch out for is you expect label-based slicing to behave exactly
 in the way that standard Python integer slicing works.
+
+Miscellaneous indexing gotchas
+------------------------------
+
+Reindex versus ix gotchas
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Many users will find themselves using the ``ix`` indexing capabilities as a
+concise means of selecting data from a pandas object:
+
+.. ipython:: python
+
+   df = DataFrame(randn(6, 4), columns=['one', 'two', 'three', 'four'],
+                  index=list('abcdef'))
+   df
+   df.ix[['b', 'c', 'e']]
+
+This is, of course, completely equivalent *in this case* to using th
+``reindex`` method:
+
+.. ipython:: python
+
+   df.reindex(['b', 'c', 'e'])
+
+Some might conclude that ``ix`` and ``reindex`` are 100% equivalent based on
+this. This is indeed true **except in the case of integer indexing**. For
+example, the above operation could alternately have been expressed as:
+
+.. ipython:: python
+
+   df.ix[[1, 2, 4]]
+
+If you pass ``[1, 2, 4]`` to ``reindex`` you will get another thing entirely:
+
+.. ipython:: python
+
+   df.reindex([1, 2, 4])
+
+So it's important to remember that ``reindex`` is **strict label indexing
+only**. This can lead to some potentially surprising results in pathological
+cases where an index contains, say, both integers and strings:
+
+.. ipython:: python
+
+   s = Series([1, 2, 3], index=['a', 0, 1])
+   s
+   s.ix[[0, 1]]
+   s.reindex([0, 1])
+
+Because the index in this case does not contain solely integers, ``ix`` falls
+back on integer indexing. By contrast, ``reindex`` only looks for the values
+passed in the index, thus finding the integers ``0`` and ``1``. While it would
+be possible to insert some logic to check whether a passed sequence is all
+contained in the index, that logic would exact a very high cost in large data
+sets.

doc/source/themes/agogo/static/agogo.css_t

@@ -453,3 +453,7 @@ div.viewcode-block:target {
     border-top: 1px solid #ac9;
     border-bottom: 1px solid #ac9;
 }
+
+th.field-name {
+    white-space: nowrap;
+}

doc/source/whatsnew/v0.7.0.txt

@@ -88,6 +88,12 @@ New features
   aggregate with groupby on a DataFrame, yielding an aggregated result with
   hierarchical columns (GH166_)
 
+- Can call ``cummin`` and ``cummax`` on Series and DataFrame to get cumulative
+  minimum and maximum, respectively (GH647_)
+
+- ``value_range`` added as utility function to get min and max of a dataframe
+  (GH288_)
+
 API Changes to integer indexing
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -266,6 +272,7 @@ similar operation to the above but using a Python function:
 .. _GH115: https://github.com/wesm/pandas/issues/115
 .. _GH166: https://github.com/wesm/pandas/issues/166
 .. _GH220: https://github.com/wesm/pandas/issues/220
+.. _GH288: https://github.com/wesm/pandas/issues/288
 .. _GH249: https://github.com/wesm/pandas/issues/249
 .. _GH267: https://github.com/wesm/pandas/issues/267
 .. _GH273: https://github.com/wesm/pandas/issues/273
@@ -288,6 +295,7 @@ similar operation to the above but using a Python function:
 .. _GH545: https://github.com/wesm/pandas/issues/545
 .. _GH554: https://github.com/wesm/pandas/issues/554
 .. _GH595: https://github.com/wesm/pandas/issues/595
+.. _GH647: https://github.com/wesm/pandas/issues/647
 .. _GH93: https://github.com/wesm/pandas/issues/93
 .. _GH93: https://github.com/wesm/pandas/issues/93
 .. _PR521: https://github.com/wesm/pandas/pull/521

pandas/__init__.py

@@ -31,3 +31,4 @@
 
 from pandas.tools.merge import merge, concat
 from pandas.tools.pivot import pivot_table, crosstab
+from pandas.tools.describe import value_range

pandas/core/common.py

@@ -595,6 +595,8 @@ def _make_int_format(x):
         return _make_int_format(s)
     else:
         if na_rep is not None and lib.checknull(s):
+            if s is None:
+                return 'None'
             return na_rep
         else:
             # object dtype
@@ -798,7 +800,7 @@ def load(path):
 
     Parameters
     ----------
-p    path : string
+    path : string
         File path
 
     Returns

pandas/core/datetools.py

@@ -714,9 +714,9 @@ class Second(Tick):
 
 
 for i, weekday in enumerate(['MON', 'TUE', 'WED', 'THU', 'FRI']):
-    for week in xrange(4):
-        _offsetMap['WOM@%d%s' % (week + 1, weekday)] = \
-            WeekOfMonth(week=week, weekday=i)
+    for iweek in xrange(4):
+        _offsetMap['WOM@%d%s' % (iweek + 1, weekday)] = \
+            WeekOfMonth(week=iweek, weekday=i)
 
 _offsetNames = dict([(v, k) for k, v in _offsetMap.iteritems()])
 

pandas/core/frame.py

@@ -1220,38 +1220,48 @@ def set_value(self, index, col, value):
 
     def irow(self, i):
         """
-        Retrieve the i-th row of the DataFrame by location as a Series. Can
-        also pass a slice object
+        Retrieve the i-th row or rows of the DataFrame by location
 
         Parameters
         ----------
-        i : int or slice
+        i : int, slice, or sequence of integers
+
+        Notes
+        -----
+        If slice passed, the resulting data will be a view
 
         Returns
         -------
-        row : Series
+        row : Series (int) or DataFrame (slice, sequence)
         """
         if isinstance(i, slice):
             return self[i]
         else:
             label = self.index[i]
-            return self.xs(label)
+            if isinstance(label, Index):
+                return self.reindex(label)
+            else:
+                return self.xs(label)
 
     def icol(self, i):
         """
-        Retrieve the i-th column of the DataFrame by location as a Series. Can
-        also pass a slice object
+        Retrieve the i-th column or columns of the DataFrame by location
 
         Parameters
         ----------
-        i : int or slice
+        i : int, slice, or sequence of integers
+
+        Notes
+        -----
+        If slice passed, the resulting data will be a view
 
         Returns
         -------
-        column : Series
+        column : Series (int) or DataFrame (slice, sequence)
         """
         label = self.columns[i]
         if isinstance(i, slice):
+            # need to return view
             lab_slice = slice(label[0], label[-1])
             return self.ix[:, lab_slice]
         else:
@@ -1283,7 +1293,7 @@ def __getitem__(self, key):
         # either boolean or fancy integer index
         elif isinstance(key, (np.ndarray, list)):
             if isinstance(key, list):
-                key = np.array(key, dtype=object)
+                key = lib.list_to_object_array(key)
 
             # also raises Exception if object array with NA values
             if com._is_bool_indexer(key):
@@ -1307,7 +1317,10 @@ def _getitem_array(self, key):
             mask = indexer == -1
             if mask.any():
                 raise KeyError("No column(s) named: %s" % str(key[mask]))
-            return self.reindex(columns=key)
+            result = self.reindex(columns=key)
+            if result.columns.name is None:
+                result.columns.name = self.columns.name
+            return result
 
     def _slice(self, slobj, axis=0):
         if axis == 0:
@@ -1427,9 +1440,7 @@ def _sanitize_column(self, value):
                 assert(len(value) == len(self.index))
 
                 if not isinstance(value, np.ndarray):
-                    value = np.array(value)
-                    if value.dtype.type == np.str_:
-                        value = np.array(value, dtype=object)
+                    value = com._asarray_tuplesafe(value)
                 else:
                     value = value.copy()
         else:
@@ -1472,7 +1483,7 @@ def xs(self, key, axis=0, level=None, copy=True):
         """
         labels = self._get_axis(axis)
         if level is not None:
-            loc = labels.get_loc_level(key, level=level)
+            loc, new_ax = labels.get_loc_level(key, level=level)
 
             # level = 0
             if not isinstance(loc, slice):
@@ -1484,7 +1495,8 @@ def xs(self, key, axis=0, level=None, copy=True):
 
             result = self.ix[indexer]
 
-            new_ax = result._get_axis(axis).droplevel(level)
+            # new_ax = result._get_axis(axis).droplevel(level)
+
             setattr(result, result._get_axis_name(axis), new_ax)
             return result
 
@@ -1495,14 +1507,23 @@ def xs(self, key, axis=0, level=None, copy=True):
             return data
 
         self._consolidate_inplace()
-        loc = self.index.get_loc(key)
+
+        index = self.index
+        if isinstance(index, MultiIndex):
+            loc, new_index = self.index.get_loc_level(key)
+        else:
+            loc = self.index.get_loc(key)
+
         if np.isscalar(loc):
             new_values = self._data.fast_2d_xs(loc, copy=copy)
             return Series(new_values, index=self.columns, name=key)
         else:
-            new_data = self._data.xs(key, axis=1, copy=copy)
-            result = DataFrame(new_data)
-            result.index = _maybe_droplevels(result.index, key)
+            result = self[loc]
+            result.index = new_index
+
+            # new_data = self._data.xs(key, axis=1, copy=copy)
+            # result = DataFrame(new_data)
+            # result.index = _maybe_droplevels(result.index, key)
             return result
 
     def lookup(self, row_labels, col_labels):