Skip to content

BUG/CI: unreliable tests in io\parser\test_common.py #35660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
fangchenli opened this issue Aug 11, 2020 · 0 comments · Fixed by #35772
Closed

BUG/CI: unreliable tests in io\parser\test_common.py #35660

fangchenli opened this issue Aug 11, 2020 · 0 comments · Fixed by #35772
Labels
CI Continuous Integration Unreliable Test Unit tests that occasionally fail Windows Windows OS
Milestone

Comments

@fangchenli
Copy link
Member

fangchenli commented Aug 11, 2020

Those two tests

pandas\tests\io\parser\test_common.py:test_chunks_have_consistent_numerical_type
pandas\tests\io\parser\test_common.py:test_warn_if_chunks_have_mismatched_type

have been producing ResourceWarning occasionally on Windows-py37-np16 and travis-37. I think they might happen on other builds as well. I just haven't seen them yet.

all_parsers = <pandas.tests.io.parser.conftest.PythonParser object at 0x0000016ADEAFB808>

    def test_chunks_have_consistent_numerical_type(all_parsers):
        parser = all_parsers
        integers = [str(i) for i in range(499999)]
        data = "a\n" + "\n".join(integers + ["1.0", "2.0"] + integers)
    
        # Coercions should work without warnings.
        with tm.assert_produces_warning(None):
>           result = parser.read_csv(StringIO(data))

pandas\tests\io\parser\test_common.py:1148: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <contextlib._GeneratorContextManager object at 0x0000016AA37A05C8>
type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed <socket.socket fd=2972, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.1.0.4', 1650), raddr=('169.254.169.254', 80)>"), 'D:\\a\\1\\s\\pandas\\io\\parsers.py', 3007)]

C:\Miniconda\envs\pandas-dev\lib\contextlib.py:119: AssertionError
all_parsers = <pandas.tests.io.parser.conftest.PythonParser object at 0x000001F32E4A2748>

    def test_warn_if_chunks_have_mismatched_type(all_parsers):
        warning_type = None
        parser = all_parsers
        integers = [str(i) for i in range(499999)]
        data = "a\n" + "\n".join(integers + ["a", "b"] + integers)
    
        # see gh-3866: if chunks are different types and can't
        # be coerced using numerical types, then issue warning.
        if parser.engine == "c" and parser.low_memory:
            warning_type = DtypeWarning
    
        with tm.assert_produces_warning(warning_type):
>           df = parser.read_csv(StringIO(data))

pandas\tests\io\parser\test_common.py:1167: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <contextlib._GeneratorContextManager object at 0x000001F3076E7948>
type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed <socket.socket fd=6620, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.1.0.4', 1656), raddr=('169.254.169.254', 80)>"), 'D:\\a\\1\\s\\pandas\\io\\parsers.py', 3007)]

C:\Miniconda\envs\pandas-dev\lib\contextlib.py:119: AssertionError

Those two tests don't fail everytime. And they are skipped for now.

@fangchenli fangchenli added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 11, 2020
@jreback jreback added Testing pandas testing functions or related to the test suite and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 11, 2020
@jreback jreback added this to the 1.2 milestone Aug 11, 2020
@mroeschke mroeschke added CI Continuous Integration Unreliable Test Unit tests that occasionally fail Windows Windows OS and removed Testing pandas testing functions or related to the test suite labels Aug 11, 2020
fangchenli added a commit to fangchenli/pandas that referenced this issue Aug 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Unreliable Test Unit tests that occasionally fail Windows Windows OS
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants