Airlift Support?

[mikeysklar]

The Metro M4 Lite running CircuitPython 9.0.0-beta2 does not contain the necessary socketpool library. Are the Airlift configurations going to be supported with HTTPServer?

forum thread for reference.

[dhalbert]

It is only the examples that use socketpool directly. The main code in the library does not use socketpool except for type annotation. ESP32SPI provides get_socket() as a source of sockets. If you change an example over to that does it work?

[mikeysklar]

Let's see.

[FoamyGuy]

I believe I looked into this when I did some refactoring work on ESP32SPI, Requests, and related libraries. IIRC it was not possible because the socket provided by ESP32SPI doesn't have all of the exact same support. In my memory it was perhaps something about bind() not existing or being different that led to HTTPServer not working, but it has been a little while I could be mistaken in my recollection. I believe it was something that would require changes within the nina firmware in order to make it work, or at least that was my understanding at the time.

While I don't think it's possible to use this library it is still possible to make a webserver more generally with ESP32SPI using the WSGI library. Here are the examples of it's usage: https://github.com/adafruit/Adafruit_CircuitPython_WSGI/tree/main/examples

[anecdata]

The ESP32SPI library (Airlift) exposes the APIs of the NINA firmware in the ESP32 wifi co-processor. The NINA firmware is written as a hybrid Arduino application (with some direct esp-idf calls), but the socket APIs are in the Arduino style rather than standard UNIX / CPython style. There is no bind, listen, or accept per se. It would require a rewrite of the NINA firmware, or maybe a CircuitPython abstraction layer to expose the interface in a standard way that adafruit_httpserver could use directly.

The most straightforward approach is as FoamyGuy suggests... to start with a server library written for ESP32SPI.

[michalpokusa]

How about creating a wrapper class in Python, that would take all necessary arguments and expose the bind(), listen(), setsocketopt() etc. methods, that internally would do the same as methods from socketpool? I imagine, of course if the capabilities are the same, that diffrence in APi can be solved by such wrapper class.

[justmobilize]

Looking at this, the commands and flow needed between native/wiznet and esp32spi are pretty different. The question would be what's the best approach to get this library working with the esp32spi. We could either:

1. create a base class, and then two subclasses
2. Have the main class detect the radio and call different methods internally

[anecdata]

I think we need to take some care not to make this library much heavier, perhaps that favors the first option.

[michalpokusa]

The only thing that would need changing is receiving/sending the data, most of the code is responsible for parsing the data, no matter from where it came from, and providing functionality like easy interface for different response types.

If one would be able to separate the connection related part of httpserver, I can imagine making a dependency injection like structure, where instead of providing a socketpool to Server constructor, one passes an interface that under the hood handles all the ins and outs of sending and receiving data.

I am happy to help integrating that into httpserver, but I do not have any device which would use Airlift or ESP32SPI and I am not very familiar with how it handles sockets, so if anyone would like to co-program, I am open to it.

[justmobilize]

So the issue is how the esp32spi handles the data. It comes back across multiple objects it controls (we'll call them sockets). It's not simply a read/write thing.

I was able to re-create a high level server today. First steps will be some updates to the esp32spi library to re-add support for sockets it creates directly.

[justmobilize]

Following up:

I have distilled the old library to the core needed for request/response.

I think I can translate this to helper methods that will work.

[dhalbert]

I wonder if this issue belongs in https://github.com/adafruit/Adafruit_CircuitPython_ESP32SPI/ rather than here. The API used here is "standard". I'd like to see any wrapping of the ESP32SPI functionality being done there, or in a separate wrapper library, rather than adding non-standard stuff to this library.

https://github.com/adafruit/Adafruit_CircuitPython_WSGI is an alternative but I'd rather see effort being spent on a single server library.

[eshawveratdirigible]

Seeing the same issue and would like to throw my support behind some resolution.

Using the Adafruit RP2040 Adalogger with CircuitPython version: 9.1.4, adafruit_httpserver version 4.5.9, and adafruit_esp32spi version 8.4.2.
Adafruit RP2040 Adalogger and AirLift Wifi FeatherWing Co-Processor are connected via a FeatherWing Tripler.

I got the httpserver started, but cannot access the endpoint from my laptop, but can ping the ip address of the microcontroller from my laptop.
Httpserver started with the addition of wrapper classes:

import board
import busio
from os import getenv
from digitalio import DigitalInOut
import adafruit_connection_manager
import adafruit_requests as requests
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_esp32spi import adafruit_esp32spi_socketpool as SocketPool
from adafruit_httpserver import Server, Request, Response

secrets = {
    "ssid": getenv("WIFI_SSID"),
    "password": getenv("WIFI_PASSWORD")
}

esp32_cs = DigitalInOut(board.D13)
esp32_ready = DigitalInOut(board.D11)
esp32_reset = DigitalInOut(board.D12)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

pool = adafruit_connection_manager.get_radio_socketpool(esp)
ssl_context = adafruit_connection_manager.get_radio_ssl_context(esp)
requests = requests.Session(pool, ssl_context)

esp.connect_AP(secrets["ssid"], secrets["password"])

pool = SocketPool.SocketPool(esp)

old_socket = SocketPool.Socket(pool)

class socket:
    def __init__(self, AF_INET, SOCK_STREAM):
        self.AF_INET = AF_INET
        self.SOCK_STREAM = SOCK_STREAM

    def __call__(self, AF_INET, SOCK_STREAM):
        return self

    def setsockopt(self, level: int, optname: int, value: int):
        return (level, optname, value)

    def bind(self, values: tuple):
        return (values)

    def listen(self, value: int):
        return value

    def setblocking(self, value: bool):
        return value

    def settimeout(self, value: float):
        return value

    def accept(self):
        conn = my_socket
        sock_num = 1
        sock = old_socket._interface.get_remote_data(sock_num)
        addr = self.bind((sock['ip_addr'], 80))
        return (conn, addr)

    def recv_into(self, buffer: bytearray, nbytes: int = 0):
        return old_socket.recv_into(buffer, nbytes)

    def close(self):
        return True

class My_Socket_Pool:
    def __init__(self, pool, socket):
        self.pool = pool
        self.socket = socket

    def getaddrinfo(self, host, port):
        return (host, port)

    def AF_INET(self):
        return self.pool.AF_INET

    def SOCK_STREAM(self):
        return self.pool.SOCK_STREAM

    def SOL_SOCKET(self):
        return 1

    def SO_REUSEADDR(self):
        return 4

my_socket = socket(pool.AF_INET, pool.SOCK_STREAM)
my_socket_pool = My_Socket_Pool(pool, my_socket)

server = Server(my_socket_pool, "/static", debug=True)

@server.route("/test")
def base(request: Request):
    return Response(request, "test worked")

server.serve_forever(str(esp.pretty_ip(esp.ip_address)), 80)

[Neradoc]

I had a compatibility layer a year or two ago to make ESP32SPI work with httpserver, but when ESP32SPI switched to a socketpool it broke everything and instead or redoing that I worked on making ESP32SPI more compatible with native sockets: adafruit/Adafruit_CircuitPython_ESP32SPI#218

The only blocker I encounter now is the unconditional import of ssl in server.py. (And memory use).
I have it working on a Matrix Portal M4, switching between small images from a web UI.