Skip to content

BUG:#29928 Fix to_json output 'table' orient for single level MultiIndex. #34375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,7 @@ I/O
- Bug in :class:`HDFStore` was dropping timezone information when exporting :class:`Series` with ``datetime64[ns, tz]`` dtypes with a fixed HDF5 store (:issue:`20594`)
- :func:`read_csv` was closing user-provided binary file handles when ``engine="c"`` and an ``encoding`` was requested (:issue:`36980`)
- Bug in :meth:`DataFrame.to_hdf` was not dropping missing rows with ``dropna=True`` (:issue:`35719`)
- Bug in :meth:`~DataFrame.to_json` with 'table' orient was writting wrong index field name for MultiIndex Dataframe with a single level (:issue:`29928`)

Plotting
^^^^^^^^
Expand Down
11 changes: 8 additions & 3 deletions pandas/io/json/_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,8 +259,6 @@ def __init__(
)
raise ValueError(msg)

self.schema = build_table_schema(obj, index=self.index)

# NotImplemented on a column MultiIndex
if obj.ndim == 2 and isinstance(obj.columns, MultiIndex):
raise NotImplementedError(
Expand All @@ -277,10 +275,17 @@ def __init__(
raise ValueError(msg)

obj = obj.copy()

# Convert DataFrame to handled types before serializing
if obj.index.nlevels == 1 and isinstance(obj.index, MultiIndex):
obj.index = obj.index.get_level_values(0)

self.schema = build_table_schema(obj, index=self.index)

timedeltas = obj.select_dtypes(include=["timedelta"]).columns
if len(timedeltas):
obj[timedeltas] = obj[timedeltas].applymap(lambda x: x.isoformat())
# Convert PeriodIndex to datetimes before serializing

if is_period_dtype(obj.index.dtype):
obj.index = obj.index.to_timestamp()

Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/io/json/test_json_table_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,23 @@ def test_to_json_categorical_index(self):

assert result == expected

@pytest.mark.parametrize("name", [None, "foo"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you replicate all of the tests in the OP (there are 4 cases)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I provided tests for cases 1 and 2 in the OP.
Case 3 is the standard behavior of orient='table' on MultiIndex with several levels, which is tested in test_read_json_table_orient.
Case 4 is a kind of workaround for single level MultiIndex.

def test_multiindex_single_level(self, name):
# GH29928
index = pd.Index([1, 2, 3, 4], name=name)
expected = DataFrame(
data=[[1, 1], [2, 2], [3, 3], [4, 4]], columns=["A", "B"], index=index
)

index = pd.MultiIndex.from_tuples([(1,), (2,), (3,), (4,)], names=[name])
df = DataFrame(
data=[[1, 1], [2, 2], [3, 3], [4, 4]], columns=["A", "B"], index=index
)
js = df.to_json(orient="table")
result = pd.read_json(js, orient="table")

tm.assert_frame_equal(result, expected)

@pytest.mark.filterwarnings(
"ignore:an integer is required (got type float)*:DeprecationWarning"
)
Expand Down