Skip to content

BUG: Allow IOErrors when attempting to retrieve default client encoding. #21531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Sep 19, 2018
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6c2987d
BUG: Except IOErrors when attempting to retrieve default client encod…
Jun 18, 2018
c31b914
BUG: Allow IOErrors when attempting to retrieve client encoding. (#21…
JayOfferdahl Jun 20, 2018
510f4b1
Merge branch 'pending-branch' into get-client-encoding-on-mod-wsgi
JayOfferdahl Jun 20, 2018
1afda5f
Merge branch 'master' into get-client-encoding-on-mod-wsgi
JayOfferdahl Jun 20, 2018
b37cb75
Fixup tests to add more scenarios. Code review changes.
JayOfferdahl Jun 20, 2018
0b1ca1d
Merge branch 'get-client-encoding-on-mod-wsgi' of https://github.com/…
JayOfferdahl Jun 20, 2018
bf26adb
PEP8 changes.
JayOfferdahl Jun 20, 2018
77a95d2
Mock out stdout/stdin instead of trying to access their properties.
JayOfferdahl Jun 26, 2018
2598595
Merge branch 'master' into get-client-encoding-on-mod-wsgi
JayOfferdahl Jun 26, 2018
59e0993
Use context manager when monkeypatching standard library items.
JayOfferdahl Jul 13, 2018
8b51b34
Merge branch 'get-client-encoding-on-mod-wsgi' of https://github.com/…
JayOfferdahl Jul 13, 2018
984da5c
Merge branch 'master' into get-client-encoding-on-mod-wsgi
JayOfferdahl Jul 13, 2018
4790e10
Fix linter erros/code review changes.
JayOfferdahl Jul 14, 2018
6156825
Move fix to 0.24.0 release; simplify tests.
JayOfferdahl Jul 16, 2018
fb1dc2e
Attempt at trying to get side_effects right without using mock.
JayOfferdahl Jul 24, 2018
7cedfdc
Add TODO(py27) to test helper.
JayOfferdahl Jul 25, 2018
35c0b3b
Fix pylint error.
JayOfferdahl Jul 25, 2018
49ee7b5
Fix silly linting errors. Use pep8.
JayOfferdahl Jul 26, 2018
40b3588
Move raise_or_return to MockEncoding; clean up parameters.
JayOfferdahl Aug 4, 2018
5e3c8af
Merge branch 'master' into get-client-encoding-on-mod-wsgi
JayOfferdahl Aug 16, 2018
f485223
Merge branch 'master' into get-client-encoding-on-mod-wsgi
JayOfferdahl Aug 23, 2018
ab67024
Merge remote-tracking branch 'upstream/master' into get-client-encodi…
JayOfferdahl Sep 18, 2018
81547da
Merge branch 'get-client-encoding-on-mod-wsgi' of https://github.com/…
JayOfferdahl Sep 18, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.23.2.txt
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ Bug Fixes

- Bug in :func:`read_csv` that caused it to incorrectly raise an error when ``nrows=0``, ``low_memory=True``, and ``index_col`` was not ``None`` (:issue:`21141`)
- Bug in :func:`json_normalize` when formatting the ``record_prefix`` with integer columns (:issue:`21536`)
- Bug in :meth:`detect_client_encoding` where potential ``IOError`` goes unhandled when importing in a mod_wsgi process due to restricted access to stdout. (:issue:`21552`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove from here

-

**Plotting**
Expand Down
2 changes: 1 addition & 1 deletion pandas/io/formats/console.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ def detect_console_encoding():
encoding = None
try:
encoding = sys.stdout.encoding or sys.stdin.encoding
except AttributeError:
except (AttributeError, IOError):
pass

# try again for something better
Expand Down
51 changes: 51 additions & 0 deletions pandas/tests/io/formats/test_console.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import pytest
from functools import partial

from pandas.io.formats.console import detect_console_encoding, locale


def mock_raises_exception(error=Exception):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this for?

raise error


@pytest.mark.parametrize('empty,filled', [
['stdin', 'stdout'],
['stdout', 'stdin']
])
def test_detect_console_encoding_from_stdout_stdin(monkeypatch, empty, filled):
# Ensures that when sys.stdout.encoding or sys.stdin.encoding is used when
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can add a reference to the issue for this and the other new tests, i.e. # GH 21552

# they have values filled.
class MockEncoding(object):
def __init__(self, encoding):
super(MockEncoding, self).__init__()
self.encoding = encoding

monkeypatch.setattr('sys.{}'.format(empty), MockEncoding(''))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance you saw the context manager for monkeypatch? Seems to be preferred for patching standard library items (may require a version bump for pytest dependency).

monkeypatch.setattr('sys.{}'.format(filled), MockEncoding(filled))
assert detect_console_encoding() == filled


@pytest.mark.parametrize('stdEncoding', [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PEP8 so std_encoding or simply encoding

partial(mock_raises_exception, AttributeError),
partial(mock_raises_exception, IOError),
lambda: 'ascii'
])
def test_detect_console_encoding_fallback_to_locale(monkeypatch, stdEncoding):
monkeypatch.setattr('locale.getpreferredencoding', lambda: 'foo')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can put this in the context manager, no?

monkeypatch.setattr('sys.stdout', stdEncoding)
assert detect_console_encoding() == 'foo'


@pytest.mark.parametrize('std,locale', [
['ascii', 'ascii'],
['ascii', mock_raises_exception],
[mock_raises_exception, 'ascii'],
[mock_raises_exception, mock_raises_exception]
])
def test_detect_console_encoding_fallback_to_default(monkeypatch, std, locale):
# When both the stdout/stdin encoding and locale preferred encoding checks
# fail (or return 'ascii', we should default to the sys default encoding.
monkeypatch.setattr('sys.stdout', std)
monkeypatch.setattr('locale.getpreferredencoding', locale)
monkeypatch.setattr('sys.getdefaultencoding', lambda: 'sysDefaultEncoding')
assert detect_console_encoding() == 'sysDefaultEncoding'