Websockets compatibility/Implementation HTML5 vs Socket.io #42

hallard · 2016-01-20T00:11:48Z

Hi Markus,
I developed my test environment with my html/javascript and nodejs server to be easier to test my html/js code before putting it into ESP8266. I used websocket socket.io library. All on my environment is working fine, but now I'm trying to put all thing into ESP and here is my problem :

although your sample code html/js are working fine with ESP8266, as soon as I'm trying to connect to ESP8266 with socket.io it does not work or even connect.
I was thinking websockets was "standard" but it seems libraries may have different implementation.
At startup I wanted to use the HTML5 one on client side (the native one in browser, same as your examples) but for the same reason I didn't not succeed to connect with this one from my client broswer to my nodejs server running socket.io

I love socket.io because it implements all disconnect/reconnect mechanism (and other) but Im' afraid may be I need to use the basic HTML5 one to get it working with you lib ?

Any hints or explanation would be useful for my understanding because I'm lost on websocket jungle.
Thanks for your help

Links2004 · 2016-01-20T16:33:58Z

socket.io is a protocol on top of websockts.
yes its very nice, have some socket.io client code for the AVR, and use it for communication in all my nodejs projects.
Have a mesh of nodejs processes running on it :)

the protocol basics are here:
https://github.com/socketio/socket.io-parser/blob/master/index.js#L147

hallard · 2016-01-20T16:38:38Z

Thanks, I will take a look onto this one.
Are you able to do websocket to ESP8266 with a js socket.io client on browser ? If so which lib are you putting into ESP8266 ?

Links2004 · 2016-01-20T16:41:01Z

i have done the other part connecting the ESP/AVR to a socket.io server (nodejs).
to get it working the other way around some Implementation for the socket.io protocol is needed.
for the Basics it will not be that hard (if you skip the different channel and subscribe stuff).

hallard · 2016-01-20T16:41:14Z

Oh by the way I reverted to HTML5 browser integrated websocket and I'm able to talk with your ArduinoWebSocket compiled into ESP8266 ;-)
But for testing I needed also to change my nodejs to npm install websocket instead of socket.io
Like this my browser client code can work with ESP8266 and Nodejs test server with no modification ;-)
No more socket.io to put into spiffs system also, so things are keeping simple, like this :-)

hallard · 2016-01-20T16:47:50Z

anyway, amazing what we can do with this little chip, really cool
thanks for the websocket library

Links2004 · 2016-01-20T16:58:22Z

looks nice, is it bootstrap?

hallard · 2016-01-20T17:00:50Z

Yes bootstrap + some goodies such

Chartjs to graph
bootstrap-slider for advanced sliders
bootstrap-switch for nice on/off switches

all framework minified+gz to be positionned in spiffs FS ;-)

Vitorbnc · 2016-01-27T17:50:54Z

Hey, could you guys share some of your socket.io for Arduino code?

Links2004 · 2016-01-27T18:01:10Z

i plan to create 2 classes for server + client out of it and add it to this repo,
butt will need some time.
first I will allow to use this lib async on ESP.

hallard · 2016-01-27T18:21:02Z

@Vitorbnc
I wasn't able to talk to ESP with socket.io client (on browser) to ArduinoWebSocket due to different implementation as Markus said. So I used Markus ArduinoWebSocket Lib on ESP (classic Websockets) with HTML5 classic socket on browser, one advantage, I do not need to put socket.io client js code on SPIFFS ;-)
Here is my websocket event on ESP (based on Markus example)

/* ======================================================================
Function: webSocketEvent
Purpose : Manage routing of websocket events
Input   : -
Output  : - 
Comments: -
====================================================================== */
void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t lenght) {

  switch(type) {

    case WStype_DISCONNECTED:
      Debugf("--- [%u] Disconnected!\n", num);
      client_state = CLIENT_NONE;
    break;

    case WStype_CONNECTED:
    {
      IPAddress ip = webSocket.remoteIP(num);
      Debugf("+++ [%u] Connected from %d.%d.%d.%d url: %s\n", num, ip[0], ip[1], ip[2], ip[3], payload);

      // send message to client
      //webSocket.sendTXT(num, "Connected");
    }
    break;

    case WStype_TEXT:
    {
      String msg = String((char*)payload);

      if (msg=="system") {
       client_state = CLIENT_SYSTEM;
      } else if (msg=="config") {
       DebuglnF("config");
       client_state = CLIENT_CONFIG;
      } else if (msg=="log") {
       DebuglnF("log");
       client_state = CLIENT_LOG;
      } else if (msg=="spiffs") {
       DebuglnF("spiffs");
       client_state = CLIENT_SPIFFS;
      } else if (msg.startsWith("sensors:") && lenght >= 9 ) {
        int val = msg.substring(8).toInt();
        DebugF("sensor=");
        Debugln(val);
        client_state = CLIENT_SENSORS;
        if (val>=2 && val <=300)
          client_refresh = val;
        else
          client_refresh = 2;
     }

      // send message to client
      // webSocket.sendTXT(num, "message here");

      // send data to all connected clients
      // webSocket.broadcastTXT("message here");
      Debugf("ws [%u] msg[%d][%d]: %s\n", num, lenght, client_state, payload);
    }
    break;

    case WStype_BIN:
      Debugf("[%u] get binary lenght: %u\n", num, lenght);
      hexdump(payload, lenght);

      // send message to client
      // webSocket.sendBIN(num, payload, lenght);
    break;
  }

}

and the client side I'm using ReconnectingWebSocket lib (to allow automatic re connection) but it's not mandatory
On browser side, I just let some WS code

window.onload = function() {  

    // open a web socket
    socksrv = 'ws://'+location.hostname+':8081';
    ws = new ReconnectingWebSocket(socksrv);

  // When Web Socket is connected
    ws.onopen = function(evt)   {
        console.log( 'ws reconnect');
        $('#mdl_wait').modal('hide');   
        $('#connect').text('connected').removeClass('label-danger').addClass('label-success');  

        elapsed=0;
        clearInterval(el_timer);
        refreshPage();
    };

    // When websocket message
    ws.onmessage = function (evt) { 
        var obj =  eval( "(" + evt.data + ")" );
      var msg = obj.message;
      var data = obj.data;
      appendLog('ws msg:'+msg+' => '+ JSON.stringify(data) );
      if (msg=='sensors')   refreshSensors(data);
      if (msg=='system')    refreshSystem(data);
      if (msg=='log')   appendLog(data);
    };

    // When websocket is closed.
    ws.onclose = function(code, reason) { 
        $('#connect').text('disconnected').removeClass('label-success').addClass('label-danger');   
    };

  // When Web Socket error
    ws.onerror = function(evt)  {
        appendLog( 'ws error');
    };
}

hallard · 2016-01-27T18:21:57Z

@Links2004
Like that, I can't wait to test ;-)

Links2004 · 2016-01-29T12:48:46Z

@hallard async one for testing:
https://github.com/Links2004/arduinoWebSockets/tree/async

hallard · 2016-01-29T14:29:15Z

@Links2004
thanks, just in few words, I'm not sure to understand all deep work done inside, what's the "async" version does better than the sync (I'm using it as server, client is browser), just to be sure that I'm understanding well?
Thanks

Vitorbnc · 2016-01-29T15:10:30Z

@hallard thanks for sharing code! I'd like to know what exactly the async lib does too.

Links2004 · 2016-01-29T15:13:58Z

the sync version do handle all stuff in the arduino loop (the loop call of the lib) and one client after a other.

the async version is event bases,
this means it can handle clients in parallel and independent form what ever the arduino loop doing right now.
by this its possible to get more performance and the websocket not stop working when long stuff is done in the other arduino code some where.

but keep in mind the webSocketEvent callback will work like an interrupt then so no delay in there.

hallard · 2016-01-29T15:24:07Z

@Links2004
Thanks, good to know and this bring me another question.
Imagine I let all the stuff done async (and I I like the "multi user" like ;-))

But in my main loop, every seconds (example) I need to check connected clients (browser), how can I handle this? IE looping thru all client connected, check if they need data and send them ?

Vitorbnc · 2016-01-29T15:32:16Z

I was trying to test WebSocketClient example with async, but I'm getting:

AsyncPrinter.cpp:116:34: error: 'class cbuf' has no member named 'available'
size_t available = _tx_buffer->available();

Links2004 · 2016-01-29T15:33:08Z

you need latest git of the ESP core

Vitorbnc · 2016-01-29T16:19:02Z

Got the last version of ESP. Now everything is fine with the async version. Do I need a special sketch to test it properly?

Links2004 · 2016-01-29T17:05:55Z

the normal examples will work, only the loop call is not needed any more.
and async need to be activated ;)
https://github.com/Links2004/arduinoWebSockets/tree/async#esp-async-tcp

Links2004 · 2016-01-29T17:59:10Z

@hallard
you can send data to the clients any time you like (in the callback too)
use the broadcast functions to send to all clients then you not need to loop.

Vitorbnc · 2016-01-29T18:00:59Z

Hey, just a doubt. What CPU and Flash frequency do you use? Is it safe to use 160MHz for CPU and 80MHz for Flash?

hallard · 2016-01-29T18:02:09Z

@Vitorbnc
No it's not safe 80MHz flash, trust me, I spent 2 days to fix erratic crash due to this setting
160MHz CPU seems fine ;-)

Links2004 · 2016-01-29T18:03:31Z

its based on the flash you have some can handle 80Mhz fine some not,
most of my ESP run at 80MHz qio fine. CPU 160MHz is not a problem will only need more power.

hallard · 2016-01-29T18:07:39Z

@Links2004
Yeah I'm using now broadcast from loop, but just curious, from loop is there any method to know how many client are connected and which they are (avoiding creating table in callback to reference in loop of course)
@Vitorbnc, @Links2004 is right "most" the word that change all ;-) But I was in DIO not in QIO may be the difference. The trick is that it crash sometime may be 15/30 min after start, sometimes 2 sec, really strange. My advice, turn your app at 40MHz flash, verify all is working for long time (no crash, you need to monitor an uptime like), and then try 80MHz

hallard · 2016-01-29T18:09:52Z

Really depends on Board quality, good post here

Links2004 · 2016-01-29T18:14:33Z

you can get the Ip for every client that is connected,
remoteIP(uint8_t num)
but more info is not stored inside, the client identification and user management is not part of the Server, and need to be done in the APP,
since there are huge difference based on the needs of the APP.

hallard · 2016-01-29T18:22:46Z

Excellent, so something like that will work ?

Serial.println("WS Clients:") ;
for (uint8_t i=0; i<WEBSOCKETS_SERVER_CLIENT_MAX  ; i++) {
  if (webSocket.remoteIP(i) )
    Serial.println(webSocket.remoteIP(i)); 
}

Links2004 · 2016-01-29T18:25:02Z

yes

hallard · 2016-01-29T18:27:18Z

;-) thanks

Vitorbnc · 2016-01-29T22:57:39Z

Can I have both client and server? I'd like client to talk to Nodejs and server to talk to browsers directly.
Another thing, when I try the WS with SSL sketch, it says "enable sntp first". How do I do it?

Links2004 · 2016-01-29T23:08:53Z

yes you can have server + client a the same time.
for sntp add to your setup:
configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");
https://github.com/esp8266/Arduino/blob/4b8fb2093fa9dcabe19e4d836c9fb5fc7becf250/cores/esp8266/time.c#L63

mkeyno · 2016-04-15T22:03:48Z

@hallard , have you upgrade your sensor live data examples by markus library? also have you any plan to check video stream feature of Markus websocket library ?
BR

kiralikbeyin · 2016-05-21T14:10:00Z

I searched halliard's github and blog but couldn't find his screenshots codes.

Anyone know where it is?

https://cloud.githubusercontent.com/assets/2471931/12455916/dbb6efa2-bf9d-11e5-8e56-6434a2698d99.png

hallard · 2016-05-22T18:24:19Z

@kiralikbeyin
I'm currently under migration of all my software to AsyncTCP which also include WebSocket of @me-no-dev, I've done a little example with WebSocket2Serial

About the code you're talking about, it's not currently released but will be soon

Charles

kiralikbeyin · 2016-05-22T18:39:28Z

Hi Charles

Thank you for reply.

Your repos are really awesome. I also wrote to your blog;
https://community.hallard.me/topic/163/where-is-this-code

I will play with this code https://github.com/koffienl/ESPimatic until you
finish, maybe you can use eeprom or gives different ideas.

I am looking forward.

Thanks.

Kiralik

22 Mayıs 2016 Pazar tarihinde, Charles [email protected] yazdı:

@kiralikbeyin https://github.com/kiralikbeyin
I'm currently under migration of all my software to AsyncTCP
https://github.com/me-no-dev/ESPAsyncWebServer which also include
WebSocket of @me-no-dev https://github.com/me-no-dev, I've done a
little example with WebSocket2Serial
https://github.com/hallard/WebSocketToSerial

About the code you're talking about, it's not currently released but will
be soon

Charles

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#42 (comment)

hallard · 2016-05-22T18:49:25Z

You're welcome, by the way, nice piece of code ESPimatic ;-)

kiralikbeyin · 2016-06-20T22:52:08Z

@hallard
Can you share bootsrap theme:)

@Links2004
esp8266/Arduino#2121 (comment)

gustavosinbandera1 · 2019-02-14T02:24:42Z

this is a best solution . in order to work with socket.io client library, you need a nodejs server with socket.io https://github.com/dtoki/Socket.io-Esp-client

Links2004 closed this as completed Oct 22, 2016

Websockets compatibility/Implementation HTML5 vs Socket.io #42

Websockets compatibility/Implementation HTML5 vs Socket.io #42

Comments

hallard commented Jan 20, 2016

Links2004 commented Jan 20, 2016

Uh oh!

hallard commented Jan 20, 2016

Uh oh!

Links2004 commented Jan 20, 2016

Uh oh!

hallard commented Jan 20, 2016

Uh oh!

hallard commented Jan 20, 2016

Uh oh!

Links2004 commented Jan 20, 2016

Uh oh!

hallard commented Jan 20, 2016

Uh oh!

Vitorbnc commented Jan 27, 2016

Uh oh!

Links2004 commented Jan 27, 2016

Uh oh!

hallard commented Jan 27, 2016

Uh oh!

hallard commented Jan 27, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

Vitorbnc commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

Vitorbnc commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

Vitorbnc commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

Vitorbnc commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

hallard commented Jan 29, 2016

Uh oh!

Vitorbnc commented Jan 29, 2016

Uh oh!

Links2004 commented Jan 29, 2016

Uh oh!

mkeyno commented Apr 15, 2016

Uh oh!

kiralikbeyin commented May 21, 2016

Uh oh!

hallard commented May 22, 2016

Uh oh!

kiralikbeyin commented May 22, 2016

Uh oh!

hallard commented May 22, 2016

Uh oh!

kiralikbeyin commented Jun 20, 2016

Uh oh!