Skip to content

ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Nov 17, 2021
32 changes: 29 additions & 3 deletions pandas/io/excel/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -681,14 +681,18 @@ class ExcelWriter(metaclass=abc.ABCMeta):
be parsed by ``fsspec``, e.g., starting "s3://", "gcs://".

.. versionadded:: 1.2.0
if_sheet_exists : {'error', 'new', 'replace'}, default 'error'
if_sheet_exists : {'error', 'new', 'replace', 'overlay'}, default 'error'
How to behave when trying to write to a sheet that already
exists (append mode only).

* error: raise a ValueError.
* new: Create a new sheet, with a name determined by the engine.
* replace: Delete the contents of the sheet before writing to it.

.. versionadded:: 1.4.0
* overlay: Write contents to the existing sheet without removing the old
contents.

.. versionadded:: 1.3.0
engine_kwargs : dict, optional
Keyword arguments to be passed into the engine.
Expand Down Expand Up @@ -755,6 +759,28 @@ class ExcelWriter(metaclass=abc.ABCMeta):
>>> with ExcelWriter("path_to_file.xlsx", mode="a", engine="openpyxl") as writer:
... df.to_excel(writer, sheet_name="Sheet3")

Here, the `if_sheet_exists` parameter can be set to replace a sheet if it
already exists:

>>> with ExcelWriter(
... "path_to_file.xlsx",
... mode="a",
... engine="openpyxl",
... if_sheet_exists="replace",
... ) as writer:
... df.to_excel(writer, sheet_name="Sheet1")

You can also write multiple DataFrames to a single sheet. Note that the
``if_sheet_exists`` parameter needs to be set to ``overlay``:

>>> with ExcelWriter("path_to_file.xlsx",
... mode="a",
... engine="openpyxl",
... if_sheet_exists="overlay",
... ) as writer:
... df1.to_excel(writer, sheet_name="Sheet1")
... df2.to_excel(writer, sheet_name="Sheet1", startcol=3)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does overlay require startrow to be not 1? (e.g should we error if this is not specified)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without startrow (e.g. startrow=0) it will overwrite the headers. This could be desired behaviour so I think we should not error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I would expect - unspecified starts at the very top left corner of the sheet.


You can store Excel file in RAM:

>>> import io
Expand Down Expand Up @@ -940,10 +966,10 @@ def __init__(

self.mode = mode

if if_sheet_exists not in [None, "error", "new", "replace"]:
if if_sheet_exists not in [None, "error", "new", "replace", "overlay"]:
raise ValueError(
f"'{if_sheet_exists}' is not valid for if_sheet_exists. "
"Valid options are 'error', 'new' and 'replace'."
"Valid options are 'error', 'new', 'replace' and 'overlay'."
)
if if_sheet_exists and "r+" not in mode:
raise ValueError("if_sheet_exists is only valid in append mode (mode='a')")
Expand Down
4 changes: 3 additions & 1 deletion pandas/io/excel/_openpyxl.py
Original file line number Diff line number Diff line change
Expand Up @@ -437,10 +437,12 @@ def write_cells(
f"Sheet '{sheet_name}' already exists and "
f"if_sheet_exists is set to 'error'."
)
elif self.if_sheet_exists == "overlay":
wks = self.sheets[sheet_name]
else:
raise ValueError(
f"'{self.if_sheet_exists}' is not valid for if_sheet_exists. "
"Valid options are 'error', 'new' and 'replace'."
"Valid options are 'error', 'new', 'replace' and 'overlay'."
)
else:
wks = self.sheets[sheet_name]
Expand Down
3 changes: 2 additions & 1 deletion pandas/tests/io/excel/test_openpyxl.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@ def test_write_append_mode(ext, mode, expected):
[
("new", 2, ["apple", "banana"]),
("replace", 1, ["pear"]),
("overlay", 1, ["pear", "banana"]),
],
)
def test_if_sheet_exists_append_modes(ext, if_sheet_exists, num_sheets, expected):
Expand Down Expand Up @@ -170,7 +171,7 @@ def test_if_sheet_exists_append_modes(ext, if_sheet_exists, num_sheets, expected
(
"invalid",
"'invalid' is not valid for if_sheet_exists. Valid options "
"are 'error', 'new' and 'replace'.",
"are 'error', 'new', 'replace' and 'overlay'.",
),
(
"error",
Expand Down