Use CircuitPython socket support

[dhalbert]

Change asyncio.open_connection() and asynico.start_server() to use CircuitPython's socket support.

[thorangutang]

Yes please. I believe this would allow a non blocking tcp/udp socket to be opened.

[furbrain]

I've tried to fix this with limited success in these functions

• CircuitPython uses SocketPool, which needs to be initialised with a radio. We can import wifi and do sp = SocketPool(wifi.radio), but this limits our ability to use other networking options as and when they appear
• socketpool.Socket doesn't accept setsockopt function calls, and there are few other cosmetic changes needed to get these functions working well to start with.
• however the socket ultimately gets sent via core.IOQueue._enqueue to select.poll.register() (c code) - which then crashes hard. This is probably because the poll code expects underlying objects of mp_stream types and the new sockets are very different types. At one point I had to nuke the flash on my pico W to get it functioning again.
• At this point I've given up because it's looking rather hard, and would possibly need code changes elsewhere in the code base
• On the other hand it should be fairly easy to do a solution in pure python - this would involve doing the polling for new data in python, but obviously this would come with a time penalty.

[furbrain]

However, the asyncio.StreamReader function does seem to at least start to play well with other "file-like" objects, such as UARTs, the usb_cdc.Serial objects and (not tested yet) the _bleio.CharacteristicBuffer streams - but see my other issue for more info

[dhalbert]

• CircuitPython uses SocketPool, which needs to be initialised with a radio. We can import wifi and do sp = SocketPool(wifi.radio), but this limits our ability to use other networking options as and when they appear

You could pass in the SocketPool. adafruit_requests and other libraries are agnostic about the source of sockets, and take it as a parameter.

[Commod0re]

I've taken a few cracks at this and it's really not so simple as that unfortunately. There are a few issues but the biggest thing is that SocketPool sockets do not expose a file descriptor, therefore they can't be used with select (and therefore also IOQueue and therefore also asyncio.stream.Stream etc)

esp-idf's examples around connecting and using non-blocking sockets also depend on having a file descriptor to use with select (see here: https://github.com/espressif/esp-idf/blob/master/examples/protocols/sockets/non_blocking/main/non_blocking_socket_example.c)

other issues, like the strange connection behavior I see with non-blocking sockets on my Qualia S3 with CircuitPython 9.2.1, can be worked around, but this piece is pretty fundamental to this particular problem. SocketPool Sockets' underlying file descriptor need to be exposed via a fileno() method so that select can work with them before Stream will work with it at all

EDITED: asyncio not aiohttp 😮‍💨

Addendum: after further testing I've determined that my issues with non-blocking TCP sockets only occur with CircuitPython >= 9.1.0 so for now I'm sticking with 9.0.5. I'll try to chase that issue down and slowly work my way back to this one

[pterricciano]

I am running on an Metro M4 Aitlift Lite. I was having a problem where trying to connect to a host which was offline would cause everything else to hang and crash the esp firmware (version 1.7.7). The application is running a control loop where keeping the Delta T jitter as low as possible was a goal. I have implemented a refactor of adafruit_miniqtt and everything underneath (connection managers, esp32_spi ...) to work in a asyncio environment. Other tasks will continue to run even while esp is trying connect. More testing is required but it is currently working for my application. Not sure how to make it available because it affects more than MQTT. The crash of the esp firmware is also fixed.