Skip to content

integration with Adafruit_CircuitPython_MiniMQTT is too fragile #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
flavio-fernandes opened this issue Jan 4, 2021 · 4 comments · Fixed by adafruit/Adafruit_CircuitPython_MiniMQTT#55
Closed

integration with Adafruit_CircuitPython_MiniMQTT is too fragile #98

flavio-fernandes opened this issue Jan 4, 2021 · 4 comments · Fixed by adafruit/Adafruit_CircuitPython_MiniMQTT#55

Comments

@flavio-fernandes
Copy link
Contributor

flavio-fernandes commented Jan 4, 2021 •
edited
Loading

When using pyportal's get_local_time() (and likely fetch()), there
seems to be incompatibilities with MiniMQTT. In the example code.py:

https://github.com/flavio-fernandes/pyportal_station/blob/41af3587d39d01f4ccfd93b41ed8c7ab9ec8986c/code.py#L240-L264

I see that calling pyportal.get_local_time() after client.loop() causes
the error below. My current suspicion is that when pyportal code
does "self._connect_esp()", it rubs MiniMQTT in a bad way. Any attempts
MQTT make to write (ping or publish) after pyportal.get_local_time() no
longer work.

48129.0: DEBUG - Sending PINGREQ
48129.1: DEBUG - Checking PINGRESP
.Sun, 3/Jan/2021 8:50 pm
...Sun, 3/Jan/2021 8:50 pm
..48189.3: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
48189.3: DEBUG - Sending PINGREQ
48189.3: DEBUG - Checking PINGRESP
.Sun, 3/Jan/2021 8:51 pm
...Sun, 3/Jan/2021 8:51 pm
.20:51:46 Interval localtime triggered
Getting time for timezone America/New_York
struct_time(tm_year=2021, tm_mon=1, tm_mday=3, tm_hour=20, tm_min=51, tm_sec=55, tm_wday=7, tm_yday=3, tm_isdst=None)
.48249.6: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
48249.6: DEBUG - Sending PINGREQ
48249.6: DEBUG - Checking PINGRESP
.Sun, 3/Jan/2021 8:52 pm
.Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
.Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
.Sun, 3/Jan/2021 8:52 pm
Failed mqtt client loop: Error response to command
.Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
Failed mqtt client loop: Error response to command
.Failed mqtt client loop: Error response to command
48312.3: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
48312.3: DEBUG - Sending PINGREQ
Failed mqtt client loop: Error response to command
20:53:6 Interval send_status triggered
48321.3: DEBUG - Sending PUBLISH
Topic: /pyportalkitchen/temperature
Msg: b'74.8062'                                
QoS: 0
Retain? False
Error in send_status, retrying in 10s - Failed to send 2 bytes (sent 0)
48321.3: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
48321.3: DEBUG - Sending PINGREQ
Failed mqtt client loop: Failed to send 2 bytes (sent 0)
.Sun, 3/Jan/2021 8:53 pm
.48324.3: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
48324.3: DEBUG - Sending PINGREQ
Failed mqtt client loop: Failed to send 2 bytes (sent 0)
@flavio-fernandes flavio-fernandes mentioned this issue Jan 5, 2021
Closed
@flavio-fernandes
Copy link
Contributor Author

Update: I have just noticed that I am not running the latest and greatest libraries. I will update these and
see if this issue is not longer "an issue". ;)

Using circup

$ circup list
Found device at /Volumes/CIRCUITPY, running CircuitPython 6.0.1.
The following modules are out of date or probably need an update.
Major Updates may include breaking changes. Review before updating.

Module                Version  Latest  Major Update
--------------------- -------- ------- -------------
adafruit_pyportal     3.6.2    4.0.0   True
adafruit_requests     1.9.1    1.9.2   False
adafruit_bitmap_font  1.3.1    1.3.2   False
adafruit_bus_device   5.0.1    5.0.2   False
adafruit_esp32spi     3.5.3    3.5.4   False

Versions after update:

$ circup update --all
...

$ circup freeze
Found device at /Volumes/CIRCUITPY, running CircuitPython 6.0.1.
adafruit_pyportal==4.0.0
adafruit_sdcard==3.3.1
neopixel==6.0.0
adafruit_button==1.4.0
adafruit_slideshow==1.5.3
adafruit_requests==1.9.2
adafruit_adt7410==1.2.2
adafruit_logging==1.2.4
adafruit_touchscreen==1.1.2
adafruit_minimqtt==4.0.1
adafruit_imageload==0.13.1
adafruit_bitmap_font==1.3.2
adafruit_esp32spi==3.5.4
adafruit_bus_device==5.0.2
adafruit_register==1.9.1
adafruit_io==5.0.2
adafruit_display_text==2.10.1
adafruit_display_shapes==2.0.3


$ circup list
Found device at /Volumes/CIRCUITPY, running CircuitPython 6.0.1.
All modules found on the device are up to date.

To be continued. ;)

@flavio-fernandes
Copy link
Contributor Author

To be continued. ;)

2021-Jan-09-Sat
Unfortunately, getting the newer library versions mentioned above did not make the issue go away.
I will see if I can collect more data on this and have something useful to add.

279180.0: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
279180.0: DEBUG - Sending PINGREQ
RuntimeError mqtt: Failed to send 2 bytes (sent 0)
279183.0: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
279183.0: DEBUG - Sending PINGREQ
RuntimeError mqtt: Failed to send 2 bytes (sent 0)
.279185.0: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
279185.0: DEBUG - Sending PINGREQ
RuntimeError mqtt: Failed to send 2 bytes (sent 0)
279189.0: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
279189.0: DEBUG - Sending PINGREQ
RuntimeError mqtt: Failed to send 2 bytes (sent 0)
15:13:15 Interval send_status triggered
279191.0: DEBUG - Sending PUBLISH
Topic: /pyportalkitchen/temperature
Msg: b'81.8375'                                
QoS: 0
Retain? False
Error in send_status, retrying in 10s: Failed to send 2 bytes (sent 0)
279192.0: DEBUG - KeepAlive period elapsed -                                    requesting a PINGRESP from the server...
279192.0: DEBUG - Sending PINGREQ
RuntimeError mqtt: Failed to send 2 bytes (sent 0)

flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 13, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
1a23d7b 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets as part of _wait_for_msg(),
  together with all other MQTT messages;

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
@flavio-fernandes
Copy link
Contributor Author

After further debugging on this issue, I came to the conclusion that this is not related to the Pyportal library.

It is a bug on how socket receive in Adafruit_CircuitPython_MiniMQTT was implemented. Being so, I created adafruit/Adafruit_CircuitPython_MiniMQTT#54 and will close this issue. Sorry for the
confusion!

@flavio-fernandes flavio-fernandes mentioned this issue Jan 13, 2021
Merged
flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 13, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
5947477 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets as part of _wait_for_msg(),
  together with all other MQTT messages;

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 13, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
c9fdc40 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets as part of _wait_for_msg(),
  together with all other MQTT messages;

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
@makermelissa
Copy link
Collaborator

Thanks for letting us know. :)

flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 13, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
e353adb 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets as part of _wait_for_msg(),
  together with all other MQTT messages;

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 13, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
b90bf88 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets as part of _wait_for_msg(),
  together with all other MQTT messages;

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 13, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
a1a04d6 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wrapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets as part of _wait_for_msg(),
  together with all other MQTT messages;

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
flavio-fernandes added a commit to flavio-fernandes/Adafruit_CircuitPython_MiniMQTT that referenced this issue Jan 14, 2021
@flavio-fernandes
Fix socket receive issues related to message buffer size
3e5a14d 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wrapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets via _wait_for_msg();

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
rtwfroody pushed a commit to rtwfroody/Adafruit_CircuitPython_MiniMQTT that referenced this issue Sep 18, 2022
@flavio-fernandes
Fix socket receive issues related to message buffer size
b20ec2c 
This change addresses a few issues in the handling of the MQTT
messages that caused the library to become unstable:

- Add wrapper for socket.recv() so that an exact number of bytes
  are read into the buffer before attempting to parse the MQTT
  message;

- Fix handling of ping response packets via _wait_for_msg();

- Fix disconnect so it can gracefully handle cases when socket writes
  are not possible. Also re-init _subscribed_topics as an empty list
  instead of None.

Related-to adafruit/Adafruit_CircuitPython_ESP32SPI#102
Fixes adafruit/Adafruit_CircuitPython_PyPortal#98
Fixes adafruit#54
Signed-off-by: Flavio Fernandes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@flavio-fernandes @makermelissa