After pystack limit reached, MQTT service continues retry until timeout

[jersu11]

Adafruit CircuitPython 9.0.4 on 2024-04-16; Adafruit PyPortal with samd51j20

Discovered while debugging a connection issue with adafruit_aws_iot library. If the pystack is exhausted, the MQTT client will continue retrying until eventually failing with MMQTTException('Repeated connect failures',)

1564.623: WARNING - Socket error when connecting: pystack exhausted
1564.625: DEBUG - Resetting reconnect backoff
1564.821: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1564.823: DEBUG - Attempting to establish MQTT connection...
1564.827: WARNING - Socket error when connecting: Socket already connected to mqtt://asxxxxxxxxxoy-ats.iot.us-west-2.amazonaws.com:8883
1564.829: DEBUG - Resetting reconnect backoff
1564.832: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1564.833: DEBUG - Attempting to establish MQTT connection...
1564.837: WARNING - Socket error when connecting: Socket already connected to mqtt://asxxxxxxxxxoy-ats.iot.us-west-2.amazonaws.com:8883
1565.035: DEBUG - Resetting reconnect backoff
1565.037: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1565.039: DEBUG - Attempting to establish MQTT connection...
1565.042: WARNING - Socket error when connecting: Socket already connected to mqtt://asxxxxxxxxxoy-ats.iot.us-west-2.amazonaws.com:8883
1565.044: DEBUG - Resetting reconnect backoff
1565.047: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1565.048: DEBUG - Attempting to establish MQTT connection...
1565.247: WARNING - Socket error when connecting: Socket already connected to mqtt://asxxxxxxxxxoy-ats.iot.us-west-2.amazonaws.com:8883
Traceback (most recent call last):
  File "code.py", line 159, in <module>
  File "adafruit_aws_iot.py", line 145, in connect
AWS_IOT_ERROR: ('Error connecting to AWS IoT: ', MMQTTException('Repeated connect failures',))

[justmobilize]

investigating

[chrisgilldc]

I have a similar issue on CircuitPython 8.2.10 with MiniMQTT 7.7.0.
I'm testing a project against for use with Home Assistant. I use a VM running HA and the Mosquitto Add-on broker.
If the broker is running when the code starts, it connects fine, publishes and receives, as expected.
If I stop the broker to simulate an HA restart, the project won't reconnect even after the broker restarts and seems to think the socket is still open.

1152.346: WARNING - Network: MQTT client not connected! Will retry in 10s.
1152.348: INFO - Network: Attempting MQTT reconnect...
1152.350: INFO - Network: Connecting to MQTT broker 172.16.10.46:1883
1152.352: DEBUG - Attempting to reconnect with MQTT broker
1152.353: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1152.355: DEBUG - Attempting to establish MQTT connection...
1152.358: WARNING - Socket error when connecting: Socket already connected to mqtt://172.16.10.46:1883
1152.360: DEBUG - Resetting reconnect backoff
1152.362: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1152.363: DEBUG - Attempting to establish MQTT connection...
1152.366: WARNING - Socket error when connecting: Socket already connected to mqtt://172.16.10.46:1883
1152.368: DEBUG - Resetting reconnect backoff
1152.370: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1152.371: DEBUG - Attempting to establish MQTT connection...
1152.375: WARNING - Socket error when connecting: Socket already connected to mqtt://172.16.10.46:1883
1152.375: DEBUG - Resetting reconnect backoff
1152.377: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1152.378: DEBUG - Attempting to establish MQTT connection...
1152.382: WARNING - Socket error when connecting: Socket already connected to mqtt://172.16.10.46:1883
1152.383: DEBUG - Resetting reconnect backoff
1152.385: DEBUG - Attempting to connect to MQTT broker (attempt #0)
1152.386: DEBUG - Attempting to establish MQTT connection...
1152.390: WARNING - Socket error when connecting: Socket already connected to mqtt://172.16.10.46:1883
1152.391: WARNING - Network: Could not connect to MQTT broker. Waiting 10s
1152.394: WARNING - Network: Received exception 'Repeated connect failures'
1162.395: WARNING - Network: MQTT not reconnected!
1162.397: WARNING - Network: MQTT client not connected! Will retry in 10s.```

This may or may not have a similar origin, but the error at least has similar errors at the end.

[chrisgilldc]

Updated to CP 9.0.4 and am having similar behavior. Client gets stuck in a PINGREQ loop and never gets out, even once the broker is back up.

[dhalbert]

@chrisgilldc Please say which board you are using and have you updated to the latest version of all the libraries?

[chrisgilldc]

@dhalbert This is on a Metro M4 Airlift Lite. Libraries were updated to the latest from the bundle. I'm out at the moment but can try updating the libraries individually later today.

[justmobilize]

Also, have you updated your airlift firmware?

[justmobilize]

@chrisgilldc I might also recommend this branch: #213 as it has a little better error logging