reomove union_categoricals kw from concat

Author: sinhrks

doc/source/categorical.rst

@@ -702,30 +702,33 @@ Concatenation
 
 This section describes concatenations specific to ``category`` dtype. See :ref:`Concatenating objects<merging.concat>` for general description.
 
-By default, ``Series`` or ``DataFrame`` concatenation which contains different
-categories results in ``object`` dtype.
+By default, ``Series`` or ``DataFrame`` concatenation which contains the same categories
+results in ``category`` dtype, otherwise results in ``object`` dtype.
+Use ``.astype`` or ``union_categoricals`` to get ``category`` result.
 
 .. ipython:: python
 
+   # same categories
    s1 = pd.Series(['a', 'b'], dtype='category')
-   s2 = pd.Series(['b', 'c'], dtype='category')
+   s2 = pd.Series(['a', 'b', 'a'], dtype='category')
    pd.concat([s1, s2])
 
-Specifying ``union_categoricals=True`` allows to concat categories following
-``union_categoricals`` rule.
+   # different categories
+   s3 = pd.Series(['b', 'c'], dtype='category')
+   pd.concat([s1, s3])
 
-.. ipython:: python
+   pd.concat([s1, s3]).astype('category')
+   union_categoricals([s1.values, s3.values])
 
-   pd.concat([s1, s2], union_categoricals=True)
 
 Following table summarizes the results of ``Categoricals`` related concatenations.
 
-| arg1         | arg2                                                              | default     | ``union_categoricals=True`` |
-|---------|-------------------------------------------|---------|------------------------------|
-| category | category (identical categories) | category | category |
-| category | category (different categories, both not ordered) | object (dtype is inferred) | category |
-| category | category (different categories, either one is ordered) | object (dtype is inferred) | object (dtype is inferred) |
-| category | not category | object (dtype is inferred) | object (dtype is inferred)
+| arg1         | arg2                                 | result  |
+|---------|-------------------------------------------|---------|
+| category | category (identical categories) | category |
+| category | category (different categories, both not ordered) | object (dtype is inferred) |
+| category | category (different categories, either one is ordered) | object (dtype is inferred) |
+| category | not category | object (dtype is inferred) |
 
 Getting Data In/Out
 -------------------

doc/source/merging.rst

@@ -79,7 +79,7 @@ some configurable handling of "what to do with the other axes":
 
     pd.concat(objs, axis=0, join='outer', join_axes=None, ignore_index=False,
               keys=None, levels=None, names=None, verify_integrity=False,
-              union_categoricals=False, copy=True)
+              copy=True)
 
 - ``objs`` : a sequence or mapping of Series, DataFrame, or Panel objects. If a
   dict is passed, the sorted keys will be used as the `keys` argument, unless
@@ -107,11 +107,6 @@ some configurable handling of "what to do with the other axes":
 - ``verify_integrity`` : boolean, default False. Check whether the new
   concatenated axis contains duplicates. This can be very expensive relative
   to the actual data concatenation.
-- ``union_categoricals`` : boolean, default False.
-  If True, use union_categoricals rule to concat category dtype.
-  If False, category dtype is kept if both categories are identical,
-  otherwise results in object dtype.
-  See :ref:`Categoricals Concatenation<categorical.concat>` for detail.
 - ``copy`` : boolean, default True. If False, do not copy data unnecessarily.
 
 Without a little bit of context and example many of these arguments don't make

doc/source/whatsnew/v0.19.0.txt

@@ -220,7 +220,7 @@ they are in the file or passed in as the ``names`` parameter (:issue:`7160`, :is
    data = '0,1,2\n3,4,5'
    names = ['a', 'b', 'a']
 
-Previous behaviour:
+Previous Behavior:
 
 .. code-block:: ipython
 
@@ -233,7 +233,7 @@ Previous behaviour:
 The first ``a`` column contains the same data as the second ``a`` column, when it should have
 contained the values ``[0, 3]``.
 
-New behaviour:
+New Behavior:
 
 .. ipython :: python
 
@@ -293,22 +293,23 @@ Categorical Concatenation
     b = pd.Categorical(["a", "b"])
     union_categoricals([a, b])
 
-- ``concat`` and ``append`` now can concat unordered ``category`` dtypes using ``union_categorical`` internally. (:issue:`13524`)
+- ``concat`` and ``append`` now can concat ``category`` dtypes wifht different
+``categories`` as ``object`` dtype (:issue:`13524`)
 
-  By default, different categories results in ``object`` dtype.
+Previous Behavior:
 
-  .. ipython:: python
+  .. code-block:: ipython
 
-    s1 = pd.Series(['a', 'b'], dtype='category')
-    s2 = pd.Series(['b', 'c'], dtype='category')
-    pd.concat([s1, s2])
+    In [1]: s1 = pd.Series(['a', 'b'], dtype='category')
+    In [2]: s2 = pd.Series(['b', 'c'], dtype='category')
+    In [3]: pd.concat([s1, s2])
+    ValueError: incompatible categories in categorical concat
 
-  Specifying ``union_categoricals=True`` allows to concat categories following
-  ``union_categoricals`` rule.
+New Behavior:
 
   .. ipython:: python
 
-    pd.concat([s1, s2], union_categoricals=True)
+    pd.concat([s1, s2])
 
 .. _whatsnew_0190.enhancements.semi_month_offsets:
 
@@ -411,11 +412,11 @@ get_dummies dtypes
 
 The ``pd.get_dummies`` function now returns dummy-encoded columns as small integers, rather than floats (:issue:`8725`). This should provide an improved memory footprint.
 
-Previous behaviour:
+Previous Behavior:
 
 .. code-block:: ipython
 
-   In [1]:    pd.get_dummies(['a', 'b', 'a', 'c']).dtypes
+   In [1]: pd.get_dummies(['a', 'b', 'a', 'c']).dtypes
 
    Out[1]:
    a    float64
@@ -437,7 +438,7 @@ Other enhancements
 
 - The ``.get_credentials()`` method of ``GbqConnector`` can now first try to fetch `the application default credentials <https://developers.google.com/identity/protocols/application-default-credentials>`__. See the :ref:`docs <io.bigquery_authentication>` for more details (:issue:`13577`).
 
-- The ``.tz_localize()`` method of ``DatetimeIndex`` and ``Timestamp`` has gained the ``errors`` keyword, so you can potentially coerce nonexistent timestamps to ``NaT``. The default behaviour remains to raising a ``NonExistentTimeError`` (:issue:`13057`)
+- The ``.tz_localize()`` method of ``DatetimeIndex`` and ``Timestamp`` has gained the ``errors`` keyword, so you can potentially coerce nonexistent timestamps to ``NaT``. The default behavior remains to raising a ``NonExistentTimeError`` (:issue:`13057`)
 - ``pd.to_numeric()`` now accepts a ``downcast`` parameter, which will downcast the data if possible to smallest specified numerical dtype (:issue:`13352`)
 
   .. ipython:: python
@@ -544,7 +545,7 @@ API changes
 ``Series.tolist()`` will now return Python types
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-``Series.tolist()`` will now return Python types in the output, mimicking NumPy ``.tolist()`` behaviour (:issue:`10904`)
+``Series.tolist()`` will now return Python types in the output, mimicking NumPy ``.tolist()`` behavior (:issue:`10904`)
 
 
 .. ipython:: python
@@ -579,7 +580,7 @@ including ``DataFrame`` (:issue:`1134`, :issue:`4581`, :issue:`13538`)
 
 .. warning::
    Until 0.18.1, comparing ``Series`` with the same length, would succeed even if
-   the ``.index`` are different (the result ignores ``.index``). As of 0.19.0, this will raises ``ValueError`` to be more strict. This section also describes how to keep previous behaviour or align different indexes, using the flexible comparison methods like ``.eq``.
+   the ``.index`` are different (the result ignores ``.index``). As of 0.19.0, this will raises ``ValueError`` to be more strict. This section also describes how to keep previous behavior or align different indexes, using the flexible comparison methods like ``.eq``.
 
 
 As a result, ``Series`` and ``DataFrame`` operators behave as below:
@@ -647,7 +648,7 @@ Logical operators
 
 Logical operators align both ``.index``.
 
-Previous Behavior (``Series``), only left hand side ``index`` is kept:
+Previous behavior (``Series``), only left hand side ``index`` is kept:
 
 .. code-block:: ipython
 
@@ -966,7 +967,7 @@ Index ``+`` / ``-`` no longer used for set operations
 
 Addition and subtraction of the base Index type (not the numeric subclasses)
 previously performed set operations (set union and difference). This
-behaviour was already deprecated since 0.15.0 (in favor using the specific
+behavior was already deprecated since 0.15.0 (in favor using the specific
 ``.union()`` and ``.difference()`` methods), and is now disabled. When
 possible, ``+`` and ``-`` are now used for element-wise operations, for
 example for concatenating strings  (:issue:`8227`, :issue:`14127`).
@@ -986,13 +987,13 @@ The same operation will now perform element-wise addition:
     pd.Index(['a', 'b']) + pd.Index(['a', 'c'])
 
 Note that numeric Index objects already performed element-wise operations.
-For example, the behaviour of adding two integer Indexes:
+For example, the behavior of adding two integer Indexes:
 
 .. ipython:: python
 
     pd.Index([1, 2, 3]) + pd.Index([2, 3, 4])
 
-is unchanged. The base ``Index`` is now made consistent with this behaviour.
+is unchanged. The base ``Index`` is now made consistent with this behavior.
 
 
 .. _whatsnew_0190.api.difference:
@@ -1143,7 +1144,7 @@ the result of calling :func:`read_csv` without the ``chunksize=`` argument.
 
    data = 'A,B\n0,1\n2,3\n4,5\n6,7'
 
-Previous behaviour:
+Previous Behavior:
 
 .. code-block:: ipython
 
@@ -1155,7 +1156,7 @@ Previous behaviour:
    0  4  5
    1  6  7
 
-New behaviour:
+New Behavior:
 
 .. ipython :: python
 
@@ -1281,7 +1282,7 @@ These types are the same on many platform, but for 64 bit python on Windows,
 ``np.int_`` is 32 bits, and ``np.intp`` is 64 bits.  Changing this behavior improves performance for many
 operations on that platform.
 
-Previous behaviour:
+Previous Behavior:
 
 .. code-block:: ipython
 
@@ -1290,7 +1291,7 @@ Previous behaviour:
    In [2]: i.get_indexer(['b', 'b', 'c']).dtype
    Out[2]: dtype('int32')
 
-New behaviour:
+New Behavior:
 
 .. code-block:: ipython
 

pandas/core/frame.py

@@ -4322,8 +4322,7 @@ def infer(x):
     # ----------------------------------------------------------------------
     # Merging / joining methods
 
-    def append(self, other, ignore_index=False, verify_integrity=False,
-               union_categoricals=False):
+    def append(self, other, ignore_index=False, verify_integrity=False):
         """
         Append rows of `other` to the end of this frame, returning a new
         object. Columns not in this frame are added as new columns.
@@ -4336,10 +4335,6 @@ def append(self, other, ignore_index=False, verify_integrity=False,
             If True, do not use the index labels.
         verify_integrity : boolean, default False
             If True, raise ValueError on creating index with duplicates.
-        union_categoricals : bool, default False
-            If True, use union_categoricals rule to concat category dtype.
-            If False, category dtype is kept if both categories are identical,
-            otherwise results in object dtype.
 
         Returns
         -------
@@ -4416,8 +4411,7 @@ def append(self, other, ignore_index=False, verify_integrity=False,
         else:
             to_concat = [self, other]
         return concat(to_concat, ignore_index=ignore_index,
-                      verify_integrity=verify_integrity,
-                      union_categoricals=union_categoricals)
+                      verify_integrity=verify_integrity)
 
     def join(self, other, on=None, how='left', lsuffix='', rsuffix='',
              sort=False):

pandas/core/internals.py

@@ -1144,7 +1144,7 @@ def get_result(other):
             return self._try_coerce_result(result)
 
         # error handler if we have an issue operating with the function
-        def handle_error(detail):
+        def handle_error():
 
             if raise_on_error:
                 raise TypeError('Could not operate %s with block values %s' %
@@ -1165,7 +1165,7 @@ def handle_error(detail):
         except ValueError as detail:
             raise
         except Exception as detail:
-            result = handle_error(detail)
+            result = handle_error()
 
         # technically a broadcast error in numpy can 'work' by returning a
         # boolean False
@@ -4771,8 +4771,7 @@ def _putmask_smart(v, m, n):
     return nv
 
 
-def concatenate_block_managers(mgrs_indexers, axes, concat_axis,
-                               copy, union_categoricals=False):
+def concatenate_block_managers(mgrs_indexers, axes, concat_axis, copy):
     """
     Concatenate block managers into one.
 
@@ -4782,19 +4781,14 @@ def concatenate_block_managers(mgrs_indexers, axes, concat_axis,
     axes : list of Index
     concat_axis : int
     copy : bool
-    union_categoricals : bool, default False
-            If True, use union_categoricals rule to concat CategoricalBlock.
-            If False, CategoricalBlock is kept if both categories are
-            identical, otherwise results in ObjectBlock.
 
     """
     concat_plan = combine_concat_plans(
         [get_mgr_concatenation_plan(mgr, indexers)
          for mgr, indexers in mgrs_indexers], concat_axis)
 
     blocks = [make_block(
-        concatenate_join_units(join_units, concat_axis, copy=copy,
-                               union_categoricals=union_categoricals),
+        concatenate_join_units(join_units, concat_axis, copy=copy),
         placement=placement) for placement, join_units in concat_plan]
 
     return BlockManager(blocks, axes)
@@ -4880,8 +4874,7 @@ def get_empty_dtype_and_na(join_units):
         raise AssertionError("invalid dtype determination in get_concat_dtype")
 
 
-def concatenate_join_units(join_units, concat_axis, copy,
-                           union_categoricals=False):
+def concatenate_join_units(join_units, concat_axis, copy):
     """
     Concatenate values from several join units along selected axis.
     """
@@ -4901,8 +4894,7 @@ def concatenate_join_units(join_units, concat_axis, copy,
         if copy and concat_values.base is not None:
             concat_values = concat_values.copy()
     else:
-        concat_values = _concat._concat_compat(
-            to_concat, axis=concat_axis, union_categoricals=union_categoricals)
+        concat_values = _concat._concat_compat(to_concat, axis=concat_axis)
 
     return concat_values
 

pandas/core/series.py

@@ -1525,8 +1525,7 @@ def searchsorted(self, v, side='left', sorter=None):
     # -------------------------------------------------------------------
     # Combination
 
-    def append(self, to_append, ignore_index=False, verify_integrity=False,
-               union_categoricals=False):
+    def append(self, to_append, ignore_index=False, verify_integrity=False):
         """
         Concatenate two or more Series.
 
@@ -1540,10 +1539,6 @@ def append(self, to_append, ignore_index=False, verify_integrity=False,
 
         verify_integrity : boolean, default False
             If True, raise Exception on creating index with duplicates
-        union_categoricals : bool, default False
-            If True, use union_categoricals rule to concat category dtype.
-            If False, category dtype is kept if both categories are identical,
-            otherwise results in object dtype.
 
         Returns
         -------
@@ -1597,8 +1592,7 @@ def append(self, to_append, ignore_index=False, verify_integrity=False,
         else:
             to_concat = [self, to_append]
         return concat(to_concat, ignore_index=ignore_index,
-                      verify_integrity=verify_integrity,
-                      union_categoricals=union_categoricals)
+                      verify_integrity=verify_integrity)
 
     def _binop(self, other, func, level=None, fill_value=None):
         """