-
Notifications
You must be signed in to change notification settings - Fork 51
Fetch all messages in loop, non-blocking mode #122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Do you have any code that can reproduce the issues you listed? I'm going to try to review this and didn't know if you had something like that already. |
There is code in the 2 issues I linked. The MemoryError may be harder to reproduce (e.g. if it depends on connection latency etc). I can try to help further if it still isn't clear |
I'm not super familiar with the specifics of MQTT so I can't comment much on the details of the change. I did try it out on a device though. Using code posted in the issue on a Can confirm I observed the same issue with currently released libraries, mqtt time "falling behind" due to only getting one callback per loop call and request getting out of retries problem. Eventually it "catches up" with a bunch of callbacks at once (in my case it was dozens, I didn't count exactly) The new version of this library from the PR does seem to resolve it. With this version I am getting fairly consistently 5 mqtt callbacks per loop with the final one matching the time given by the http request. I inclined to approve, but we should probably try out a few of the learn guide projects that use minimqtt to ensure the differing functionality with callbacks does not have adverse effects in them. Assuming no issues with them, I think this is good. |
Regarding the "catching up" I figured out that happens because of the keep_alive feature. where it runs the |
merging EAGAIN changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me.
I tested successfully with PyPortal using this learn guide: https://learn.adafruit.com/adafruit-io-a-c-power-relay/circuitpython-setup
And Feather ESP32-S2 TFT with the time subscription reproducer script from the issue. And Fun House using the native networking example inside the examples folder of this repo.
Everything appears to function as intended to me even with the new behavior that gets all messages in the queue every loop instead of only 1 most loops.
Thanks for working on this fix @calcut
Updating https://github.com/adafruit/Adafruit_CircuitPython_BNO055 to 5.4.7 from 5.4.6: > Merge pull request adafruit/Adafruit_CircuitPython_BNO055#108 from tcfranks/main Updating https://github.com/adafruit/Adafruit_CircuitPython_RFM69 to 2.1.13 from 2.1.12: > Merge pull request adafruit/Adafruit_CircuitPython_RFM69#42 from tcfranks/main Updating https://github.com/adafruit/Adafruit_CircuitPython_DisplayIO_Layout to 1.19.9 from 1.19.8: > Merge pull request adafruit/Adafruit_CircuitPython_DisplayIO_Layout#76 from The-Debarghya/bug-75-zerodivisionerror Updating https://github.com/adafruit/Adafruit_CircuitPython_Logging to 4.2.0 from 4.1.6: > Merge pull request adafruit/Adafruit_CircuitPython_Logging#40 from tekktrik/dev/allow-root-logger Updating https://github.com/adafruit/Adafruit_CircuitPython_MiniMQTT to 5.5.2 from 5.5.1: > Merge pull request adafruit/Adafruit_CircuitPython_MiniMQTT#122 from calcut/fetch_all_messages_in_loop Updating https://github.com/adafruit/Adafruit_CircuitPython_PYOA to 2.5.12 from 2.5.11: > Merge pull request adafruit/Adafruit_CircuitPython_PYOA#37 from tekktrik/dev/fix-install
First PR, so bear with me!
Basically, I've found issues with callling
loop()
too frequently or too infrequently, that I'm trying to address here.Calling too frequently (with timeout >0) can lead to a nasty MemoryError. #101
Calling too infrequently can cause a build up of messages on the broker. #108
I've changed the default timeout to 0 (and added error handling of EAGAIN)
This might be a controversial change - but it means the function is really non-blocking as the comments say it should be. (Perhaps there is a history of why it was set to 1 by default?, I'd be happy keeping it at 1 if it prevents issues for others)
I've also made it call
_wait_for_msg()
in a while loop, similar to how the ping() function works.I've used the 10s
_recv_timeout
to make sure it can't get stuck in the loop.This means that all waiting messages are received, not just the next in the queue.