Refactor the network stack to allow chips without WiFi and better Ethernet and ppp compatibility

[me-no-dev]

Related area

Networking

Hardware specification

Proper H2 and P4 support

Is your feature request related to a problem?

Currently almost nothing about networking can operate without WiFi, so for chips like the ESP32-H2 and ESP32-P4 that do not have WiFi radios, almost nothing will compile, even if Ethernet is added.

Describe the solution you'd like

• Create a central Network class, that will keep track of interfaces, handle events and resolve domains (replacement for WiFiGeneric)
• WiFiClient, WiFiServer and WiFiUdp to be moved to the above networking library, renamed and added defines to the old names for compatibility
• Update all other libraries to not reference WiFi anymore
• Rewrite AP and STA interfaces by the Ethernet model (Create abstract class, based on esp_netif for common functionality)
• Provide interface to the old WiFi API for compatibility
• Discuss possible deprecation of the old APIs/names

Describe alternatives you've considered

We could probably get around without rewriting, with just a bunch #ifdef, but that will be ugly and still require WiFi code on a chip without WiFi.

Additional context

No response

I have checked existing list of Feature requests and the Contribution Guide

• I confirm I have checked existing list of Feature requests and Contribution Guide.

[lucasssvaz]

IMO we should to this as cleanly as possible even if it means rewriting some code. This will be much better in the long run for the code maintainability.

[TD-er]

I think it would also make sense to have each "network" device being split up into some kind of "driver" structure like how it is done in the ESP-IDF 5.1 and also have some "state" structure/class which has some generic interface to query simple states like "connected" and "gotIP" etc.

Right now this is a complete mess as you need to query WiFi related code to use Ethernet.
Also disabling WiFi after you processed an Ethernet DHCP will clear out DNS server records etc.

Don't get me started on all possible state changes when connecting to WiFi. (or while being connected)
These are way too many and should be dealt with in a specific piece of code only dealing with those for WiFi.
For example the WiFi disconnect reason can be very important on what to do next.
The most intuitive order of events is:

• Connected
• GotIP

But that doesn't always happen in this order and it should be dealt with in such a state machine struct/class specific to WiFi.

So these state classes should also keep track of their own "addresses". (not sure how it will be called for 802.15.4, have not yet looked into it)
This way RMII Ethernet chips and SPI Ethernet can have their own specifics in separate classes, which act all on their own.
From the outside it should not matter which one you're using.
From a user point of view, you're essentially only interested in a very limited set of states and also a limited set of configurations. (e.g. DHCP/static IP, speed)
And some configurations are very device specific, like scan, speed, half/full-duplex, etc.
Thus per type of interface you probably need some generic get/set config functions with some struct.

Not sure if "Network" is generic enough to cover all connectivity options.
Maybe "net" is slightly more generic? Or "interface" ("iface") ?

Create a central Network class, that will keep track of interfaces, handle events and resolve domains

Maybe it is exactly what you have in mind, but please let's not make a generic class which will manage all.
Please let there be a specific class for each type of network which does handle only its own events and state machine.
And maybe have some kind of callback function to register with each of these classes so you can get notified on a state change of another of those.
This way, such a 'central Network class' would be nothing more then a state machine to switch between interfaces, based on scenarios.

For ESPEasy I have tried to make it do it all just basing on events and I truly hate it how incredibly complex and unmanagable it has become. (read: a complete rewrite is pending)

My idea is that you should be able to have some scenarios of preferred connectivity and if those are not available you should have some kind of fallback state/scenario.
And such a fallback/preferred state could be "turn off". (or even start with all "off" for battery operated devices for example)

Just to give some idea of scenarios which might be useful for some (most?) users:

Scenario 1: Reconnect/warm boot
- Turn on WiFi
- Retry last known SSID/BSSID/channel (store last stable connected AP in RTC memory)
Failed: -> Cold boot scenario


Scenario 2: Cold boot with known credentials
- Collect known credentials
- Perform scan
- Sort known credentials based on scan
- Loop:
  - Select credentials[N}
  - Try connect
  - Failed:
    - ++N
    - If no more credentials:
      - start AP
      - If no-one connected to AP:  restart scenario 2


Scenario 3: No known credentials
- Perform scan (to speed up setup page)
- start AP (when scan completes)

Scenario 4: LAN available
- If connected, try getting IP
- If not connected or IP timeout: Scenarios 1...3

As you can see, those scenarios need to switch from one interface (or scenario) to another, so I think this central "Network" class should just do that and nothing more.

There are of course "rules" describing which state change is allowed and which isn't.
For example performing a WiFi scan while at the same time trying to make a connection to an AP is not really productive.
And when enabling a LAN interface, you should be able to detect whether an Ethernet cable is plugged in, not only receiving an event when such a connected state changes.

But this should be knowledge in the specific state machine class dealing with these interfaces.

Also just some questions:

What about the ESP-NOW layer?
Where does this belong in this redesign?
It is for sure related to WiFi, but is it a separate interface, or is it a specific branch of states for the WiFi interface?
And yes, I know you can run both ESP-NOW and STA/AP mode with IP simultaneous, as I am doing that :)

WiFi AP vs WiFi STA? Are those 2 separate interfaces?
They for sure are related, but from a networking point of view they are completely different.

The same question for WiFi in promiscuous mode.

Where does the ToF fit in here?

[mrengineer7777]

Intriguing idea. I actually considered refactoring the WiFi class into separate modules like this, but realized it's way too much work for one person. In one of my projects I created wrapper classes to partially abstract network functionality. I may have useful input during this refactoring.