Skip to content

BUG: from_dict ignored order of OrderedDict (#8425) #26875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 8, 2019
Merged

BUG: from_dict ignored order of OrderedDict (#8425) #26875

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3929945
BUG: from_dict ignored order of OrderedDict (#8425)
mazayo Jun 15, 2019
28f3e72
Merge remote-tracking branch 'upstream/master' into from_ordered_dict
mazayo Jun 21, 2019
ae19e6c
Merged upstream/master
mazayo Jun 21, 2019
de4406b
Changed for compatibility. Python=3.5 needs sort.
mazayo Jul 3, 2019
8720697
Merge remote-tracking branch 'upstream/master' into from_ordered_dict
mazayo Jul 4, 2019
50c5467
Fixed linting error
mazayo Jul 4, 2019
7bc716b
Resolved the error found in black
mazayo Jul 5, 2019
74966cb
Fixed according to the review comments
mazayo Jul 7, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.25.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -671,7 +671,6 @@ Indexing
- Fixed bug where assigning a :class:`arrays.PandasArray` to a :class:`pandas.core.frame.DataFrame` would raise error (:issue:`26390`)
- Allow keyword arguments for callable local reference used in the :meth:`DataFrame.query` string (:issue:`26426`)


Missing
^^^^^^^

Expand Down Expand Up @@ -767,6 +766,7 @@ Reshaping
- Bug in :func:`DataFrame.sort_index` where an error is thrown when a multi-indexed ``DataFrame`` is sorted on all levels with the initial level sorted last (:issue:`26053`)
- Bug in :meth:`Series.nlargest` treats ``True`` as smaller than ``False`` (:issue:`26154`)
- Bug in :func:`DataFrame.pivot_table` with a :class:`IntervalIndex` as pivot index would raise ``TypeError`` (:issue:`25814`)
- Bug in which :meth:`DataFrame.from_dict` ignored order of ``OrderedDict`` when ``orient='index'`` (:issue:`8425`).

Sparse
^^^^^^
Expand Down
4 changes: 3 additions & 1 deletion pandas/core/internals/construction.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,8 +300,10 @@ def extract_index(data):
raise ValueError('If using all scalar values, you must pass'
' an index')

if have_series or have_dicts:
if have_series:
index = _union_indexes(indexes)
elif have_dicts:
index = _union_indexes(indexes, sort=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I suppose this is the key one. IIUC, we want to sort for Python=3.5. So this should be

index = _union_indexes(indexes, sort=not compat.PY36)

That will be True for python 3.5 and False for 3.6 and newer.

You'll need to import pandas.compat at the top.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't checked yet if you need to inspect the contents of the dicts to see if they're ordered or not. Hopefully you won't have to.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I'll take care of this later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of

index = _union_indexes(indexes, sort=not compat.PY36)

changed to

index = _union_indexes(indexes, sort=not (compat.PY36 or have_ordered))

to prevent sorting a OrderedDict in Python3.5


if have_raw_arrays:
lengths = list(set(raw_lengths))
Expand Down
16 changes: 13 additions & 3 deletions pandas/tests/frame/test_constructors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -484,7 +484,7 @@ def test_constructor_subclass_dict(self):
dct.update(v.to_dict())
data[k] = dct
frame = DataFrame(data)
tm.assert_frame_equal(self.frame.sort_index(), frame)
tm.assert_frame_equal(self.frame, frame)

def test_constructor_dict_block(self):
expected = np.array([[4., 3., 2., 1.]])
Expand Down Expand Up @@ -1154,7 +1154,7 @@ def test_constructor_list_of_series(self):

sdict = OrderedDict(zip(['x', 'Unnamed 0'], data))
expected = DataFrame.from_dict(sdict, orient='index')
tm.assert_frame_equal(result.sort_index(), expected)
tm.assert_frame_equal(result, expected)

# none named
data = [OrderedDict([['a', 1.5], ['b', 3], ['c', 4], ['d', 6]]),
Expand Down Expand Up @@ -1289,7 +1289,7 @@ def test_constructor_list_of_namedtuples(self):
def test_constructor_orient(self):
data_dict = self.mixed_frame.T._series
recons = DataFrame.from_dict(data_dict, orient='index')
expected = self.mixed_frame.sort_index()
expected = self.mixed_frame.reindex(recons.index)
tm.assert_frame_equal(recons, expected)

# dict of sequence
Expand All @@ -1299,6 +1299,16 @@ def test_constructor_orient(self):
xp = DataFrame.from_dict(a).T.reindex(list(a.keys()))
tm.assert_frame_equal(rs, xp)

def test_constructor_from_ordered_dict(self):
# GH8425
a = OrderedDict([
('one', OrderedDict([('col_a', 'foo1'), ('col_b', 'bar1')])),
('two', OrderedDict([('col_a', 'foo2'), ('col_b', 'bar2')])),
('three', OrderedDict([('col_a', 'foo3'), ('col_b', 'bar3')]))])
expected = DataFrame.from_dict(a, orient='columns').T
result = DataFrame.from_dict(a, orient='index')
tm.assert_frame_equal(result, expected)

def test_from_dict_columns_parameter(self):
# GH 18529
# Test new columns parameter for from_dict that was added to make
Expand Down