Skip to content

PYTHON-4292 Improve TLS read performance #2020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 2, 2024

Conversation

ShaneHarvey
Copy link
Member

@ShaneHarvey ShaneHarvey commented Nov 27, 2024

PYTHON-4292 Improve TLS read performance

Use a socket timeout based approach for cancelling pending operations. This results in >=2x better throughput when reading large TLS responses from multiple threads.

Before:

time python bench-100-threads-find-large.py
Python: 3.13.0, PyMongo: 4.11.0.dev0, MongoDB: 8.0.1, auth: True, tls: True
python bench-100-threads-find-large.py  15.05s user 53.82s system 256% cpu 26.805 total

After:

time python bench-100-threads-find-large.py
Python: 3.13.0, PyMongo: 4.11.0.dev0, MongoDB: 8.0.1, auth: True, tls: True
python bench-100-threads-find-large.py  9.31s user 23.28s system 244% cpu 13.327 total

Use a socket timeout based approach for cancelling pending operations.
This results in >=2x better throughput when reading large TLS responses from
multiple threads.
@ShaneHarvey ShaneHarvey marked this pull request as ready for review November 27, 2024 20:54
Copy link
Member

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ShaneHarvey
Copy link
Member Author

ShaneHarvey commented Dec 2, 2024

Update I also confirmed the perf improvement on a rhel8.7-small host (updated the benchmark to use 20 threads to time's sake). Before:

$ time python bench-20-threads-find-large.py
Python: 3.13.0, PyMongo: 4.11.0.dev0, MongoDB: 8.0.4-rc0, auth: True, tls: True

real	0m13.271s
user	0m12.706s
sys	0m10.766s

After:

$ time python bench-20-threads-find-large.py
Python: 3.13.0, PyMongo: 4.11.0.dev0, MongoDB: 8.0.4-rc0, auth: True, tls: True

real	0m7.784s
user	0m7.708s
sys	0m4.563s

@ShaneHarvey ShaneHarvey merged commit a9e61f6 into mongodb:master Dec 2, 2024
46 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants