Skip to content

Fix concat not respecting order of OrderedDict #25224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Mar 13, 2019
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c54e3fe
Fix concat not respecting order of OrderedDict
alexander-ponomaroff Feb 8, 2019
073ade8
Merge conflict fix
alexander-ponomaroff Feb 8, 2019
d6cec5c
Fix pep8 issues
alexander-ponomaroff Feb 8, 2019
d4e70ef
Fix import order
alexander-ponomaroff Feb 8, 2019
db82b07
Checks reversed to PY36
alexander-ponomaroff Feb 12, 2019
f1e90ea
Merge conflict
alexander-ponomaroff Feb 12, 2019
0772212
Forgot to remove PY37 from imports
alexander-ponomaroff Feb 12, 2019
06d3472
Change order of expected result in test_groupby_agg_ohlc_non_first() …
alexander-ponomaroff Feb 12, 2019
2aee611
Fixed accidental empty line removal
alexander-ponomaroff Feb 12, 2019
feb7a36
Fix test_groupby_agg_ohlc_non_first()
alexander-ponomaroff Feb 12, 2019
a11cfc8
Testing if this will pass tests in <3.6
alexander-ponomaroff Feb 14, 2019
bbe7019
Merge
alexander-ponomaroff Feb 14, 2019
4f6fc06
Revert back to compat check for now
alexander-ponomaroff Feb 14, 2019
48cce24
Merge master
alexander-ponomaroff Feb 14, 2019
451c43b
Requested changes
alexander-ponomaroff Feb 17, 2019
5e47944
Merge master
alexander-ponomaroff Feb 17, 2019
262208b
Merge
alexander-ponomaroff Feb 19, 2019
2fb7214
Merge
alexander-ponomaroff Feb 19, 2019
154c639
Remaining small requested fixes
alexander-ponomaroff Mar 4, 2019
8faccf8
Merge
alexander-ponomaroff Mar 4, 2019
b74dc54
Fix unwanted import error
alexander-ponomaroff Mar 4, 2019
3a20a1d
Use OrderedDict in groupby/generic.py
alexander-ponomaroff Mar 12, 2019
1557c21
Merge
alexander-ponomaroff Mar 12, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.25.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ Reshaping
- Bug in :func:`pandas.merge` adds a string of ``None`` if ``None`` is assigned in suffixes instead of remain the column name as-is (:issue:`24782`).
- Bug in :func:`merge` when merging by index name would sometimes result in an incorrectly numbered index (:issue:`24212`)
- :func:`to_records` now accepts dtypes to its `column_dtypes` parameter (:issue:`24895`)
- Bug in :func:`concat` order of OrderedDict is not respected (:issue:`21510`)


Sparse
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/reshape/concat.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ def __init__(self, objs, axis=0, join='outer', join_axes=None,

if isinstance(objs, dict):
if keys is None:
keys = sorted(objs)
keys = com.dict_keys_to_ordered_list(objs)
objs = [objs[k] for k in keys]
else:
objs = list(objs)
Expand Down
4 changes: 4 additions & 0 deletions pandas/tests/groupby/test_groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -1695,6 +1695,10 @@ def test_groupby_agg_ohlc_non_first():
('foo', 'sum', 'foo'))), index=pd.date_range(
'2018-01-01', periods=2, freq='D'))

if compat.PY36:
sorted_keys = sorted(expected, key=lambda x: x[1], reverse=True)
expected = expected.reindex(columns=sorted_keys)

result = df.groupby(pd.Grouper(freq='D')).agg(['sum', 'ohlc'])

tm.assert_frame_equal(result, expected)
12 changes: 10 additions & 2 deletions pandas/tests/reshape/test_concat.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from collections import deque
from collections import OrderedDict, deque
import datetime as dt
from datetime import datetime
from decimal import Decimal
Expand Down Expand Up @@ -1163,7 +1163,7 @@ def test_concat_dict(self):
'baz': DataFrame(np.random.randn(4, 3)),
'qux': DataFrame(np.random.randn(4, 3))}

sorted_keys = sorted(frames)
sorted_keys = pd.core.common.dict_keys_to_ordered_list(frames)

result = concat(frames)
expected = concat([frames[k] for k in sorted_keys], keys=sorted_keys)
Expand Down Expand Up @@ -2399,6 +2399,14 @@ def test_concat_different_extension_dtypes_upcasts(self):
], dtype=object)
tm.assert_series_equal(result, expected)

def test_concat_odered_dict(self):
# GH 21510
expected = pd.concat([pd.Series(range(3)), pd.Series(range(4))],
keys=['First', 'Another'])
result = pd.concat(OrderedDict([('First', pd.Series(range(3))),
('Another', pd.Series(range(4)))]))
tm.assert_series_equal(result, expected)


@pytest.mark.parametrize('pdt', [pd.Series, pd.DataFrame, pd.Panel])
@pytest.mark.parametrize('dt', np.sctypes['float'])
Expand Down