Skip to content

Commit 6d1541e

Browse files
authored
REF: move get_filepath_buffer into get_handle (#37639)
1 parent 34a4c51 commit 6d1541e

35 files changed

+598
-673
lines changed

doc/source/user_guide/io.rst

+5-4
Original file line numberDiff line numberDiff line change
@@ -1024,17 +1024,18 @@ Writing CSVs to binary file objects
10241024

10251025
.. versionadded:: 1.2.0
10261026

1027-
``df.to_csv(..., mode="w+b")`` allows writing a CSV to a file object
1028-
opened binary mode. For this to work, it is necessary that ``mode``
1029-
contains a "b":
1027+
``df.to_csv(..., mode="wb")`` allows writing a CSV to a file object
1028+
opened binary mode. In most cases, it is not necessary to specify
1029+
``mode`` as Pandas will auto-detect whether the file object is
1030+
opened in text or binary mode.
10301031

10311032
.. ipython:: python
10321033
10331034
import io
10341035
10351036
data = pd.DataFrame([0, 1, 2])
10361037
buffer = io.BytesIO()
1037-
data.to_csv(buffer, mode="w+b", encoding="utf-8", compression="gzip")
1038+
data.to_csv(buffer, encoding="utf-8", compression="gzip")
10381039
10391040
.. _io.float_precision:
10401041

doc/source/whatsnew/v1.2.0.rst

+4-2
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,8 @@ Support for binary file handles in ``to_csv``
8484

8585
:meth:`to_csv` supports file handles in binary mode (:issue:`19827` and :issue:`35058`)
8686
with ``encoding`` (:issue:`13068` and :issue:`23854`) and ``compression`` (:issue:`22555`).
87-
``mode`` has to contain a ``b`` for binary handles to be supported.
87+
If Pandas does not automatically detect whether the file handle is opened in binary or text mode,
88+
it is necessary to provide ``mode="wb"``.
8889

8990
For example:
9091

@@ -94,7 +95,7 @@ For example:
9495
9596
data = pd.DataFrame([0, 1, 2])
9697
buffer = io.BytesIO()
97-
data.to_csv(buffer, mode="w+b", encoding="utf-8", compression="gzip")
98+
data.to_csv(buffer, encoding="utf-8", compression="gzip")
9899
99100
Support for short caption and table position in ``to_latex``
100101
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -515,6 +516,7 @@ I/O
515516
- :func:`read_csv` was closing user-provided binary file handles when ``engine="c"`` and an ``encoding`` was requested (:issue:`36980`)
516517
- Bug in :meth:`DataFrame.to_hdf` was not dropping missing rows with ``dropna=True`` (:issue:`35719`)
517518
- Bug in :func:`read_html` was raising a ``TypeError`` when supplying a ``pathlib.Path`` argument to the ``io`` parameter (:issue:`37705`)
519+
- :meth:`to_excel` and :meth:`to_markdown` support writing to fsspec URLs such as S3 and Google Cloud Storage (:issue:`33987`)
518520

519521
Plotting
520522
^^^^^^^^

pandas/_typing.py

-5
Original file line numberDiff line numberDiff line change
@@ -146,10 +146,5 @@
146146
CompressionOptions = Optional[Union[str, CompressionDict]]
147147

148148

149-
# let's bind types
150-
ModeVar = TypeVar("ModeVar", str, None, Optional[str])
151-
EncodingVar = TypeVar("EncodingVar", str, None, Optional[str])
152-
153-
154149
# type of float formatter in DataFrameFormatter
155150
FloatFormatType = Union[str, Callable, "EngFormatter"]

pandas/core/frame.py

+5-5
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,7 @@
157157
from pandas.core.series import Series
158158
from pandas.core.sorting import get_group_index, lexsort_indexer, nargsort
159159

160-
from pandas.io.common import get_filepath_or_buffer
160+
from pandas.io.common import get_handle
161161
from pandas.io.formats import console, format as fmt
162162
from pandas.io.formats.info import DataFrameInfo
163163
import pandas.plotting
@@ -2301,10 +2301,10 @@ def to_markdown(
23012301
result = tabulate.tabulate(self, **kwargs)
23022302
if buf is None:
23032303
return result
2304-
ioargs = get_filepath_or_buffer(buf, mode=mode, storage_options=storage_options)
2305-
assert not isinstance(ioargs.filepath_or_buffer, (str, mmap.mmap))
2306-
ioargs.filepath_or_buffer.writelines(result)
2307-
ioargs.close()
2304+
2305+
with get_handle(buf, mode, storage_options=storage_options) as handles:
2306+
assert not isinstance(handles.handle, (str, mmap.mmap))
2307+
handles.handle.writelines(result)
23082308
return None
23092309

23102310
@deprecate_kwarg(old_arg_name="fname", new_arg_name="path")

pandas/core/generic.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -3221,7 +3221,7 @@ def to_csv(
32213221
File path or object, if None is provided the result is returned as
32223222
a string. If a non-binary file object is passed, it should be opened
32233223
with `newline=''`, disabling universal newlines. If a binary
3224-
file object is passed, `mode` needs to contain a `'b'`.
3224+
file object is passed, `mode` might need to contain a `'b'`.
32253225
32263226
.. versionchanged:: 0.24.0
32273227

0 commit comments

Comments
 (0)