BUG: Fix MultiIndex names handling in pd.concat

closes #15787

Author: Tong Shen <[email protected]>

Closes #15955 from funnycrab/fix_bug_in_concat and squashes the following commits:

8c0e721 [Tong Shen] explicitly specify dtype when constructing DataFrame to avoid test failure
db7866f [Tong Shen] construct expected results as DataFrame instead of FrozenList
7f82be9 [Tong Shen] BUG: Fix MultiIndex names handling in pd.concat

Author: funnycrab

doc/source/whatsnew/v0.20.0.txt

@@ -1241,6 +1241,7 @@ Indexing
 - Bug in creating a ``MultiIndex`` with tuples and not passing a list of names; this will now raise ``ValueError`` (:issue:`15110`)
 - Bug in the HTML display with with a ``MultiIndex`` and truncation (:issue:`14882`)
 - Bug in the display of ``.info()`` where a qualifier (+) would always be displayed with a ``MultiIndex`` that contains only non-strings (:issue:`15245`)
+- Bug in ``pd.concat()`` where the names of ``MultiIndex`` of resulting ``DataFrame`` are not handled correctly when ``None`` is presented in the names of ``MultiIndex`` of input ``DataFrame`` (:issue:`15787`)
 
 I/O
 ^^^

pandas/indexes/api.py

@@ -107,7 +107,7 @@ def _get_consensus_names(indexes):
     # find the non-none names, need to tupleify to make
     # the set hashable, then reverse on return
     consensus_names = set([tuple(i.names) for i in indexes
-                           if all(n is not None for n in i.names)])
+                           if any(n is not None for n in i.names)])
     if len(consensus_names) == 1:
         return list(list(consensus_names)[0])
     return [None] * indexes[0].nlevels

pandas/tests/tools/test_concat.py

@@ -1048,6 +1048,30 @@ def test_concat_multiindex_with_tz(self):
         result = concat([df, df])
         tm.assert_frame_equal(result, expected)
 
+    def test_concat_multiindex_with_none_in_index_names(self):
+        # GH 15787
+        index = pd.MultiIndex.from_product([[1], range(5)],
+                                           names=['level1', None])
+        df = pd.DataFrame({'col': range(5)}, index=index, dtype=np.int32)
+
+        result = concat([df, df], keys=[1, 2], names=['level2'])
+        index = pd.MultiIndex.from_product([[1, 2], [1], range(5)],
+                                           names=['level2', 'level1', None])
+        expected = pd.DataFrame({'col': list(range(5)) * 2},
+                                index=index, dtype=np.int32)
+        assert_frame_equal(result, expected)
+
+        result = concat([df, df[:2]], keys=[1, 2], names=['level2'])
+        level2 = [1] * 5 + [2] * 2
+        level1 = [1] * 7
+        no_name = list(range(5)) + list(range(2))
+        tuples = list(zip(level2, level1, no_name))
+        index = pd.MultiIndex.from_tuples(tuples,
+                                          names=['level2', 'level1', None])
+        expected = pd.DataFrame({'col': no_name}, index=index,
+                                dtype=np.int32)
+        assert_frame_equal(result, expected)
+
     def test_concat_keys_and_levels(self):
         df = DataFrame(np.random.randn(1, 3))
         df2 = DataFrame(np.random.randn(1, 4))