Skip to content

Commit 5cd6b84

Browse files
committed
This fix enables you to preserve the datetime precision when using the melt method. It fixes the issues raised in #55254.
1 parent 6f0cd8d commit 5cd6b84

File tree

3 files changed

+45
-4
lines changed

3 files changed

+45
-4
lines changed

doc/source/whatsnew/v2.2.0.rst

+1-3
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,6 @@ Other enhancements
7777
- :func:`read_csv` now supports ``on_bad_lines`` parameter with ``engine="pyarrow"``. (:issue:`54480`)
7878
- :meth:`ExtensionArray._explode` interface method added to allow extension type implementations of the ``explode`` method (:issue:`54833`)
7979
- DataFrame.apply now allows the usage of numba (via ``engine="numba"``) to JIT compile the passed function, allowing for potential speedups (:issue:`54666`)
80-
- Implement masked algorithms for :meth:`Series.value_counts` (:issue:`54984`)
8180
-
8281

8382
.. ---------------------------------------------------------------------------
@@ -219,7 +218,6 @@ Other Deprecations
219218
- Deprecated allowing non-keyword arguments in :meth:`DataFrame.to_parquet` except ``path``. (:issue:`54229`)
220219
- Deprecated allowing non-keyword arguments in :meth:`DataFrame.to_pickle` except ``path``. (:issue:`54229`)
221220
- Deprecated allowing non-keyword arguments in :meth:`DataFrame.to_string` except ``buf``. (:issue:`54229`)
222-
- Deprecated allowing non-keyword arguments in :meth:`DataFrame.to_xml` except ``path_or_buffer``. (:issue:`54229`)
223221
- Deprecated automatic downcasting of object-dtype results in :meth:`Series.replace` and :meth:`DataFrame.replace`, explicitly call ``result = result.infer_objects(copy=False)`` instead. To opt in to the future version, use ``pd.set_option("future.no_silent_downcasting", True)`` (:issue:`54710`)
224222
- Deprecated downcasting behavior in :meth:`Series.where`, :meth:`DataFrame.where`, :meth:`Series.mask`, :meth:`DataFrame.mask`, :meth:`Series.clip`, :meth:`DataFrame.clip`; in a future version these will not infer object-dtype columns to non-object dtype, or all-round floats to integer dtype. Call ``result.infer_objects(copy=False)`` on the result for object inference, or explicitly cast floats to ints. To opt in to the future version, use ``pd.set_option("future.no_silent_downcasting", True)`` (:issue:`53656`)
225223
- Deprecated including the groups in computations when using :meth:`DataFrameGroupBy.apply` and :meth:`DataFrameGroupBy.resample`; pass ``include_groups=False`` to exclude the groups (:issue:`7155`)
@@ -252,6 +250,7 @@ Bug fixes
252250
- Bug in :class:`AbstractHolidayCalendar` where timezone data was not propagated when computing holiday observances (:issue:`54580`)
253251
- Bug in :class:`pandas.core.window.Rolling` where duplicate datetimelike indexes are treated as consecutive rather than equal with ``closed='left'`` and ``closed='neither'`` (:issue:`20712`)
254252
- Bug in :meth:`DataFrame.apply` where passing ``raw=True`` ignored ``args`` passed to the applied function (:issue:`55009`)
253+
- Bug in :meth:`pandas.DataFrame.melt` where it would not preserve the datetime (:issue:`55254`)
255254
- Bug in :meth:`pandas.read_excel` with a ODS file without cached formatted cell for float values (:issue:`55219`)
256255

257256
Categorical
@@ -316,7 +315,6 @@ MultiIndex
316315
I/O
317316
^^^
318317
- Bug in :func:`read_csv` where ``on_bad_lines="warn"`` would write to ``stderr`` instead of raise a Python warning. This now yields a :class:`.errors.ParserWarning` (:issue:`54296`)
319-
- Bug in :func:`read_csv` with ``engine="pyarrow"`` where ``usecols`` wasn't working with a csv with no headers (:issue:`54459`)
320318
- Bug in :func:`read_excel`, with ``engine="xlrd"`` (``xls`` files) erroring when file contains NaNs/Infs (:issue:`54564`)
321319
- Bug in :func:`to_excel`, with ``OdsWriter`` (``ods`` files) writing boolean/string value (:issue:`54994`)
322320

pandas/core/reshape/melt.py

+3-1
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,9 @@ def melt(
134134

135135
mcolumns = id_vars + var_name + [value_name]
136136

137-
if frame.shape[1] > 0:
137+
if frame.shape[1] > 0 and not any(
138+
not isinstance(dt, np.dtype) and dt._supports_2d for dt in frame.dtypes.values
139+
):
138140
mdata[value_name] = concat(
139141
[frame.iloc[:, i] for i in range(frame.shape[1])]
140142
).values

pandas/tests/reshape/test_melt.py

+41
Original file line numberDiff line numberDiff line change
@@ -459,6 +459,47 @@ def test_melt_ea_columns(self):
459459
)
460460
tm.assert_frame_equal(result, expected)
461461

462+
def test_melt_preserves_datetime(self):
463+
df = DataFrame(
464+
data=[
465+
{
466+
"type": "A0",
467+
"start_date": pd.Timestamp("2023/03/01", tz="Asia/Tokyo"),
468+
"end_date": pd.Timestamp("2023/03/10", tz="Asia/Tokyo"),
469+
},
470+
{
471+
"type": "A1",
472+
"start_date": pd.Timestamp("2023/03/01", tz="Asia/Tokyo"),
473+
"end_date": pd.Timestamp("2023/03/11", tz="Asia/Tokyo"),
474+
},
475+
],
476+
index=["aaaa", "bbbb"],
477+
)
478+
result = df.melt(
479+
id_vars=["type"],
480+
value_vars=["start_date", "end_date"],
481+
var_name="start/end",
482+
value_name="date",
483+
)
484+
expected = DataFrame(
485+
{
486+
"type": {0: "A0", 1: "A1", 2: "A0", 3: "A1"},
487+
"start/end": {
488+
0: "start_date",
489+
1: "start_date",
490+
2: "end_date",
491+
3: "end_date",
492+
},
493+
"date": {
494+
0: pd.Timestamp("2023-03-01 00:00:00+0900", tz="Asia/Tokyo"),
495+
1: pd.Timestamp("2023-03-01 00:00:00+0900", tz="Asia/Tokyo"),
496+
2: pd.Timestamp("2023-03-10 00:00:00+0900", tz="Asia/Tokyo"),
497+
3: pd.Timestamp("2023-03-11 00:00:00+0900", tz="Asia/Tokyo"),
498+
},
499+
}
500+
)
501+
tm.assert_frame_equal(result, expected)
502+
462503

463504
class TestLreshape:
464505
def test_pairs(self):

0 commit comments

Comments
 (0)