BUG: Fix concat key name #14292

bkandel · 2016-09-23T19:44:46Z

closes BUG: keys argument in pandas.concat does not preserve Index name #14252
tests added / passed
passes git diff upstream/master | flake8 --diff
whatsnew entry

Fixes a bug where pd.concat didn't propagate the names of keys used to create
a hierarchical index.

jreback · 2016-09-23T19:46:51Z

pandas/tests/frame/test_combine_concat.py

+                        'bar': [0.1, 0.2, 0.3, 0.4]})
+        index = Index(['a', 'b'], name='baz')
+        concatted = pd.concat([df, df], keys=index)
+        assert_equal(concatted.index.names, FrozenList(['baz', None]))


always use assert_frame_equal and construct and expected frame

I'm not sure that assert_frame_equal checks for names of indices here.

sure it does
see doc string

I see -- fixed now. Also added a couple extra test cases.

jreback · 2016-09-23T19:47:56Z

pandas/tools/merge.py

+        if names is None and hasattr(keys, 'names'):
+            self.names = keys.names
+        else:
+            self.names = names


names or getattr(keys, 'names', None)

codecov-io · 2016-09-23T21:54:47Z

Current coverage is 85.25% (diff: 100%)

Merging #14292 into master will decrease coverage by <.01%

@@             master     #14292   diff @@
==========================================
  Files           140        140          
  Lines         50632      50634     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43171      43170     -1   
- Misses         7461       7464     +3   
  Partials          0          0

Powered by Codecov. Last update daba8e5...cdc76f6

jreback · 2016-09-27T22:15:13Z

pandas/tools/merge.py

                clean_objs.append(v)
            objs = clean_objs
-            keys = clean_keys
+            clean_keys_index = Index(clean_keys_list)


use _ensure_index

jreback · 2016-09-27T22:16:47Z

pandas/tools/merge.py

                clean_objs.append(v)
            objs = clean_objs
-            keys = clean_keys
+            clean_keys_index = Index(clean_keys_list)
+            if isinstance(keys, Index):


this logic seems convoluted, can you simplify?

Seems like you should be able to change the name of clean_keys_list back to clean_keys and

if isinstance(keys, Index): keys = Index(clean_keys, name=keys.name)

(untested)

bkandel · 2016-10-02T00:26:31Z

@jreback I think all comments are addressed now.

Fixes a bug where `pd.concat` didn't propagate the names of keys used to create a hierarchical index.

bkandel · 2016-10-07T20:04:28Z

@jreback Changed whatsnew entry to 0.19.1. Is there anything left to do before merging?

jreback · 2016-10-08T14:46:15Z

pandas/tools/merge.py

@@ -1369,7 +1369,9 @@ def __init__(self, objs, axis=0, join='outer', join_axes=None,
                clean_keys.append(k)
                clean_objs.append(v)
            objs = clean_objs
-            keys = clean_keys
+            name = getattr(keys, 'name', None)
+            keys = _ensure_index(Index(clean_keys))


hmm I don't think you need the _ensure_index if you are forcing the index

yes appears that is correct: cdc76f6 looks fine.

jreback · 2016-10-09T17:38:30Z

thanks @bkandel

* commit 'v0.19.0-14-ga40e185': (37 commits) BUG: Bug in localizing an ambiguous timezone when a boolean is passed Convert readthedocs links for their .org -> .io migration for hosted projects (pandas-dev#14406) DOC: formatting in basics.rst BLD/CI: cython cache pxd files (pandas-dev#14363) BUG: set_levels set illegal levels. (pandas-dev#14236) DOC: add whitespace to v0.19.1 bug fix section change impl details slightly for pandas-dev#14292 BUG: Fix concat key name DOC: add 0.19.1 whatsnew file (pandas-dev#14366) DOC: to_csv warns regarding quoting behaviour for floats pandas-dev#14195 (pandas-dev#14228) DOC: fix formatting issue with msgpack table TST: pandas-dev#14345 fixes TestDatetimeIndexOps test_nat AssertionErrors on 32-bit docs: Remove old warning from dsintro.rst (pandas-dev#14365) DOC: minor v0.19.0 whatsnew corrections RLS: v0.19.0 DOC: update release notes DOC: Latest fixes for whatsnew file to_latex encoding follows the documentation (py2 ascii, py3 utf8) (pandas-dev#14329) DOC: fix some sphinx build issues (pandas-dev#14332) TST: fix period tests for numpy 1.9.3 (GH14183) (pandas-dev#14331) ...

closes pandas-dev#14252 Fixes a bug where `pd.concat` didn't propagate the names of keys used to create a hierarchical index. Author: Ben Kandel <[email protected]> Closes pandas-dev#14292 from bkandel/fix_concat_key_name and squashes the following commits: cdc76f6 [Ben Kandel] take out _ensure_index 4a301f8 [Ben Kandel] put back in Index coercion d8e2c17 [Ben Kandel] remove coercion to Index before _ensure_index 44932cc [Ben Kandel] changed whatsnew entry to 0.19.1 c51df19 [Ben Kandel] _ensure_index b54b081 [Ben Kandel] simplified logic 3256119 [Ben Kandel] typo 789ecd4 [Ben Kandel] use _ensure_index 350e724 [Ben Kandel] simplified logic 9615a69 [Ben Kandel] extra tests 5c0108b [Ben Kandel] comments dd3c4cc [Ben Kandel] comments 5cd8392 [Ben Kandel] added test for names bc5f1fb [Ben Kandel] cleanup ef6db68 [Ben Kandel] BUG: Propagate key names in concat.

jreback requested changes Sep 23, 2016

View reviewed changes

jreback added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Sep 24, 2016

jreback reviewed Sep 27, 2016

View reviewed changes

bkandel force-pushed the fix_concat_key_name branch from 05217c8 to 40325a9 Compare September 30, 2016 17:39

Ben Kandel added 12 commits October 7, 2016 16:02

BUG: Propagate key names in concat.

ef6db68

Fixes a bug where `pd.concat` didn't propagate the names of keys used to create a hierarchical index.

cleanup

bc5f1fb

added test for names

5cd8392

comments

dd3c4cc

comments

5c0108b

extra tests

9615a69

simplified logic

350e724

use _ensure_index

789ecd4

typo

3256119

simplified logic

b54b081

_ensure_index

c51df19

changed whatsnew entry to 0.19.1

44932cc

bkandel force-pushed the fix_concat_key_name branch from da560b0 to 44932cc Compare October 7, 2016 20:03

jreback added this to the 0.19.1 milestone Oct 8, 2016

jreback reviewed Oct 8, 2016

View reviewed changes

Ben Kandel added 3 commits October 8, 2016 23:21

remove coercion to Index before _ensure_index

d8e2c17

put back in Index coercion

4a301f8

take out _ensure_index

cdc76f6

jreback closed this in 16b64f1 Oct 9, 2016

jreback added a commit that referenced this pull request Oct 9, 2016

change impl details slightly for #14292

f9ebeae

tworec pushed a commit to RTBHOUSE/pandas that referenced this pull request Oct 21, 2016

change impl details slightly for pandas-dev#14292

ba06d60

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: Fix concat key name #14292

BUG: Fix concat key name #14292

bkandel commented Sep 23, 2016

jreback Sep 23, 2016

bkandel Sep 23, 2016

jreback Sep 23, 2016

bkandel Sep 25, 2016

jreback Sep 23, 2016

bkandel Sep 23, 2016

codecov-io commented Sep 23, 2016 •

edited

Loading

jreback Sep 27, 2016

bkandel Sep 30, 2016

jreback Sep 27, 2016

TomAugspurger Sep 28, 2016 •

edited

Loading

bkandel Sep 30, 2016

bkandel commented Oct 2, 2016

bkandel commented Oct 7, 2016

jreback Oct 8, 2016

bkandel Oct 9, 2016

jreback commented Oct 9, 2016

BUG: Fix concat key name #14292

BUG: Fix concat key name #14292

Conversation

bkandel commented Sep 23, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Sep 23, 2016 • edited Loading

Current coverage is 85.25% (diff: 100%)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger Sep 28, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bkandel commented Oct 2, 2016

bkandel commented Oct 7, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Oct 9, 2016

codecov-io commented Sep 23, 2016 •

edited

Loading

TomAugspurger Sep 28, 2016 •

edited

Loading