WIP: categoricals as an internal CategoricalBlock GH5313

.gitignore

@@ -88,3 +88,5 @@ doc/source/vbench
 doc/source/vbench.rst
 doc/source/index.rst
 doc/build/html/index.html
+# Windows specific leftover:
+doc/tmp.sv

doc/source/api.rst

@@ -429,7 +429,7 @@ Time series-related
    Series.tz_localize
 
 String handling
-~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~
 ``Series.str`` can be used to access the values of the series as
 strings and apply several methods to it. Due to implementation
 details the methods show up here as methods of the
@@ -468,6 +468,60 @@ details the methods show up here as methods of the
    StringMethods.upper
    StringMethods.get_dummies
 
+.. _api.categorical:
+
+Categorical
+~~~~~~~~~~~
+
+.. currentmodule:: pandas.core.categorical
+
+If the Series is of dtype ``category``, ``Series.cat`` can be used to access the the underlying
+``Categorical``. This data type is similar to the otherwise underlying numpy array
+and has the following usable methods and properties (all available as
+``Series.cat.<method_or_property>``).
+
+
+.. autosummary::
+   :toctree: generated/
+
+   Categorical
+   Categorical.from_codes
+   Categorical.levels
+   Categorical.ordered
+   Categorical.reorder_levels
+   Categorical.remove_unused_levels
+   Categorical.min
+   Categorical.max
+   Categorical.mode
+   Categorical.describe
+
+``np.asarray(categorical)`` works by implementing the array interface. Be aware, that this converts
+the Categorical back to a numpy array, so levels and order information is not preserved!
+
+.. autosummary::
+   :toctree: generated/
+
+   Categorical.__array__
+
+To create compatibility with `pandas.Series` and `numpy` arrays, the following (non-API) methods
+are also introduced.
+
+.. autosummary::
+   :toctree: generated/
+
+   Categorical.from_array
+   Categorical.get_values
+   Categorical.copy
+   Categorical.dtype
+   Categorical.ndim
+   Categorical.sort
+   Categorical.equals
+   Categorical.unique
+   Categorical.order
+   Categorical.argsort
+   Categorical.fillna
+
+
 Plotting
 ~~~~~~~~
 .. currentmodule:: pandas

doc/source/basics.rst

@@ -1574,7 +1574,8 @@ dtypes:
                    'float64': np.arange(4.0, 7.0),
                    'bool1': [True, False, True],
                    'bool2': [False, True, False],
-                   'dates': pd.date_range('now', periods=3).values})
+                   'dates': pd.date_range('now', periods=3).values}),
+                   'category': pd.Categorical(list("ABC))
    df['tdeltas'] = df.dates.diff()
    df['uint64'] = np.arange(3, 6).astype('u8')
    df['other_dates'] = pd.date_range('20130101', periods=3).values
@@ -1630,6 +1631,11 @@ All numpy dtypes are subclasses of ``numpy.generic``:
 
     subdtypes(np.generic)
 
+.. note::
+
+    Pandas also defines an additional ``category`` dtype, which is not integrated into the normal
+    numpy hierarchy and wont show up with the above function.
+
 .. note::
 
    The ``include`` and ``exclude`` parameters must be non-string sequences.