Skip to content

[CLN] Excel Module Cleanups #25275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Feb 20, 2019
Merged

[CLN] Excel Module Cleanups #25275

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
7bbe910
Address comments of @gfyoung in #25153
tdamsma Feb 11, 2019
c98f72e
satisfy isort
tdamsma Feb 11, 2019
bc074ed
removed seemingly dead code as per suggestion @WillAyd
tdamsma Feb 12, 2019
62ce7db
remove test for registering custom excel extensions
tdamsma Feb 12, 2019
5e49439
Revert "remove test for registering custom excel extensions"
tdamsma Feb 13, 2019
cf3c5bc
Remove test for registering excel file with custom extension as this …
tdamsma Feb 13, 2019
e42b10c
trigger test rerun
tdamsma Feb 13, 2019
a82119a
improve docstring
tdamsma Feb 14, 2019
7f81675
imprve docstring
tdamsma Feb 14, 2019
03bc237
Merge commit '4a20d5b9e526435a53b7cb1dc0a819299d31f040' into excel-mo…
tdamsma Feb 17, 2019
f30ab5c
remove trailing whitespace
tdamsma Feb 17, 2019
9ef747f
Merge commit '659e0cae6be2d7ab3370cc7d8ab936bc3ee1b159' into excel-mo…
tdamsma Feb 18, 2019
19160fa
Apply suggestions from code review
WillAyd Feb 19, 2019
bcf053e
Improve docstrings as per WillAyd's suggestions
tdamsma Feb 19, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions pandas/io/excel/_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -590,9 +590,8 @@ def __new__(cls, path, engine=None, **kwargs):
if engine == 'auto':
engine = _get_default_writer(ext)
except KeyError:
error = ValueError("No engine for filetype: '{ext}'"
.format(ext=ext))
raise error
raise ValueError("No engine for filetype: '{ext}'"
.format(ext=ext))
cls = get_writer(engine)

return object.__new__(cls)
Expand Down
44 changes: 28 additions & 16 deletions pandas/io/excel/_util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,46 @@

from pandas.core.dtypes.common import is_integer, is_list_like

from pandas.core import config
_writers = {}

_writer_extensions = ["xlsx", "xls", "xlsm"]

def register_writer(klass):
"""
Add engine to the excel writer registry. You must use this method to
integrate with ``to_excel``.

_writers = {}
Parameters
----------
klass : class
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just list the type here as ExcelWriter then instead of class

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Instance of ExcelWriter

Returns
-------
None

"""

def register_writer(klass):
"""Adds engine to the excel writer registry. You must use this method to
integrate with ``to_excel``. Also adds config options for any new
``supported_extensions`` defined on the writer."""
if not callable(klass):
raise ValueError("Can only register callables as engines")
engine_name = klass.engine
_writers[engine_name] = klass
for ext in klass.supported_extensions:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just to be clear this was never part of the public API hence why I think OK to remove. If a user in spite of that still registered a custom subclass with an extension outside of .xls, .xlsx or .xlsm then potentially it would have an impact, but that seems like such an edge case to me hence why I think OK to remove.

cc @jreback in any case

Copy link
Member

@gfyoung gfyoung Feb 13, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm...I'm -0.5 on this. Discarding because it's an "edge case" does not seem like a strong argument for removing this code if we're in the business of robustness. Potentially there are better ways to implement this, but I wouldn't necessarily go as far as removing for now.

Copy link
Member

@WillAyd WillAyd Feb 13, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main point is that it's not part of the public API and is a no-op with current internal code

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main point is that it's not part of the public API and is a no-op with current internal code

True. I'm still a little on the fence on this though for the reason I gave earlier, so getting @jreback input would be helpful for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is ok i think

if ext.startswith('.'):
ext = ext[1:]
if ext not in _writer_extensions:
config.register_option("io.excel.{ext}.writer".format(ext=ext),
engine_name, validator=str)
_writer_extensions.append(ext)


def _get_default_writer(ext):
"""
Return the default writer for the given extension.

Parameters
----------
ext : str
The excel file extension for which to get the default engine.

Returns
-------
engine : str
The default engine for the extension.

"""
_default_writers = {'xlsx': 'openpyxl', 'xlsm': 'openpyxl', 'xls': 'xlwt'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this a module level variable, but maybe needs a followup for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could see this going either way, so agreed that a followup makes sense.

try:
import xlsxwriter # noqa
Expand Down Expand Up @@ -230,8 +244,6 @@ def _fill_mi_header(row, control_row):

return _maybe_convert_to_string(row), control_row

# fill blank if index_col not None


def _pop_header_name(row, index_col):
"""
Expand Down
7 changes: 2 additions & 5 deletions pandas/tests/io/test_excel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2359,7 +2359,7 @@ def test_register_writer(self):
class DummyClass(ExcelWriter):
called_save = False
called_write_cells = False
supported_extensions = ['test', 'xlsx', 'xls']
supported_extensions = ['xlsx', 'xls']
engine = 'dummy'

def save(self):
Expand All @@ -2377,12 +2377,9 @@ def check_called(func):

with pd.option_context('io.excel.xlsx.writer', 'dummy'):
register_writer(DummyClass)
writer = ExcelWriter('something.test')
writer = ExcelWriter('something.xlsx')
assert isinstance(writer, DummyClass)
df = tm.makeCustomDataframe(1, 1)

func = lambda: df.to_excel('something.test')
check_called(func)
check_called(lambda: df.to_excel('something.xlsx'))
check_called(
lambda: df.to_excel(
Expand Down