Skip to content

TST: Use pytest-localserver instead of making network connections #53828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 27, 2023

Conversation

mroeschke
Copy link
Member

Since we have a lot of CI jobs with unit test that make network connections, it is better if the network connections could be made locally if possible. This PR refactors all tests that make network connections to use pytest-localserver instead

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite IO Network Local or Cloud (AWS, GCS, etc.) IO Issues labels Jun 24, 2023
@mroeschke mroeschke added this to the 2.1 milestone Jun 24, 2023
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome! Just a few questions.

Comment on lines 615 to 616
It is highly discouraged to add a test that connects to the internet due to flakiness of network connections and
lack of ownership of the server that is being connected to. If network connectivity is absolutely required, use the
``tm.network`` decorator.
lack of ownership of the server that is being connected to. If network connectivity is absolutely required, mock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this, are these still highly discouraged? Seems like all the reasons to discourage them no longer apply.

Copy link
Member Author

@mroeschke mroeschke Jun 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point I'll clarify the wording here, "highly discouraged" is probably too strong if they can be mocked properly

Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice idea

@mroeschke mroeschke merged commit b86d085 into pandas-dev:main Jun 27, 2023
@mroeschke mroeschke deleted the tst/rm/network branch June 27, 2023 16:58
@phofl
Copy link
Member

phofl commented Jul 3, 2023

This broke the env on ARM Mac. pytest local server requires aiosmtpd which isn't available on Mac

Edit: Does anyone have bandwidth to look into it? Otherwise I'd revert this for now

@WillAyd
Copy link
Member

WillAyd commented Jul 3, 2023

I am OK with revert unless @mroeschke has an objection

@mroeschke
Copy link
Member Author

Could we just remove pytest localserver from environment.yaml? This plugin helps save a few minutes from CI runtime and having the network rests run locally probably isn't a huge priority

Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
…ndas-dev#53828)

* Use pytest-localserver instead of making network connections

* Fix test, remove network function

* remove network from init

* Ignore disutils from datareader, s3so

* specify encoding

* Specify encoding

* Clarify contributing doc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO Network Local or Cloud (AWS, GCS, etc.) IO Issues Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants