Skip to content

ESP8266 NONOS SDK 2.1/2.2/2.2.1 #1207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaBecker opened this issue Jul 30, 2017 · 50 comments
Closed

ESP8266 NONOS SDK 2.1/2.2/2.2.1 #1207

MaBecker opened this issue Jul 30, 2017 · 50 comments
Assignees
Labels
enhancement ESP8266 This is only a problem on ESP8266 devices

Comments

@MaBecker
Copy link
Contributor

As version 2.1 is now available since May 2017 let give it a try :)

Overview:

Code changes:

Used libraries: NET, TELNET, GRAPHICS, CRYPTO and NEOPIXEL

make statement

  • RELEASE=1 ESPSDK_2_1=1 make

The result looks promising, freeHeap: 11264

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v93.3119 Copyright 2016 G.Williams 
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:1024/1024, manuf 0xe0 chip 0x4016

>require('ESP8266').logDebug(false);
=undefined
>require("ESP8266").getState(); 
={
  "sdkVersion": "2.1.0(116b762)",
  "cpuFrequency": 160, "freeHeap": 11264, "maxCon": 10,
  "flashMap": "4MB:1024/1024",
  "flashKB": 4096,
  "flashChip": "0xe0 0x4016"
}
>process.env
={
  "VERSION": "1v93.3120",
  "BUILD_DATE": "Jul 30 2017",
  "BUILD_TIME": "11:04:34",
  "GIT_COMMIT": "ed27629121172c7c090cae3a6273feb6227d48ef",
  "BOARD": "ESP8266_4MB",
  "CHIP": "ESP8266",
  "CHIP_FAMILY": "ESP8266",
  "FLASH": 0, "RAM": 81920,
  "SERIAL": "5ccf7fc1-120b",
  "CONSOLE": "Serial1",
  "EXPORTS": { "jsvLock": 1075851304, "jsvLockAgainSafe": 1075851356, "jsvUnLock": 1075851728, "jsvSkipName": 1075860364,
    "jsvMathsOp": 1075865696, "jsvMathsOpSkipNames": 1075867412, "jsvNewFromFloat": 1075858076, "jsvNewFromInteger": 1075857976, "jsvNewFromString": 1075857620,
    "jsvNewFromBool": 1075858044, "jsvGetFloat": 1075863436, "jsvGetInteger": 1075859860, 
    "jsvGetBool": 1075863796, "jspeiFindInScopes": 1075879024,
    "jspReplaceWith": 1075878744, "jspeFunctionCall": 1075889112, "jspGetNamedVariable": 1075880868, "jspGetNamedField": 1075881312, "jspGetVarNamedField": 1075881444,
    "jsvNewWithFlags": 1075857344 }
 }
> 
@MaBecker MaBecker added enhancement ESP8266 This is only a problem on ESP8266 devices labels Jul 30, 2017
@MaBecker
Copy link
Contributor Author

MaBecker commented Aug 3, 2017

Code changes:

@MaBecker
Copy link
Contributor Author

@gfwilliams, please let me know if this can go to Espruino master, so I can prepare the PRs.

@wilberforce
Copy link
Member

@MaBecker

V2.2 is out now.

ESP8266 non-OS SDK V2.2.0 | NON-OS SDK | V2.2.0 | 2018.02.07 |  

https://www.espressif.com/en/support/download/sdks-demos?keys=&field_type_tid%5B%5D=14

@wilberforce
Copy link
Member

We are still getting the V2.0 tool chain from tve's server:

https://github.com/espruino/Espruino/blob/master/scripts/provision.sh#L60-L69

curl -Ls http://s3.voneicken.com/esp_iot_sdk_v2.0.0.p1.tgx | tar Jxf - --no-same-owner
curl -Ls http://s3.voneicken.com/xtensa-lx106-elf-20160330.tgx | tar Jxf - --no-same-owner

Once 2.2 is tested and working, this could be updated and moved to here:
https://github.com/espruino/EspruinoBuildTools

@gfwilliams
Copy link
Member

Eep. Yes, we should definitely be trying to drag stuff from EspruinoBuildTools now :)

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 9, 2018

2.2 has some link issues

@wilberforce
Copy link
Member

wilberforce commented Apr 9, 2018

@MaBecker

2.2 has some link issues

Can you elaborate - Are there library changes that need changes to the makefile?

What tool chain are you using?

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 10, 2018

version:

Marks-MBP:Espruino markbecker$ git log -1
commit 05277faaf109727c973dcb58a23562161083b94e (HEAD -> master, origin/master, origin/HEAD)
Author: Gordon Williams <[email protected]>
Date:   Mon Apr 9 15:54:31 2018 +0100

like this:

/Volumes/Espruino/gcc/xtensa-lx106-elf-4.8.5/bin/../lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld: espruino_esp8266_user2.elf section `.text' will not fit in region `iram1_0_seg'
collect2: error: ld returned 1 exit status
make: *** [espruino_esp8266_user2.elf] Error 1
make: *** Waiting for unfinished jobs....
/Volumes/Espruino/gcc/xtensa-lx106-elf-4.8.5/bin/../lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld: espruino_esp8266_user1.elf section `.text' will not fit in region `iram1_0_seg'
collect2: error: ld returned 1 exit status
make: *** [espruino_esp8266_user1.elf] Error 1

@MaBecker
Copy link
Contributor Author

@MaBecker
Copy link
Contributor Author

Ok, have to replace the LD file with the new version coming with der SDK 2.2.

@MaBecker
Copy link
Contributor Author

There are some issue named for 2.2 at https://github.com/espressif/ESP8266_NONOS_SDK/issues

@MaBecker
Copy link
Contributor Author

SDK 2.2 sounds very promising (release notes)

will use this wifi test created by tve to get a go or nogo for this SDK

@MaBecker
Copy link
Contributor Author

added ESP82666 SDK to Build 2.0, 2.1 and 2.2 to EspruinoBuildTools

@MaBecker
Copy link
Contributor Author

Build and change result for Espruino 1.96 with SDK 2.1 - Status: stable

console.log('\n',process.env,'\n',
            require("ESP8266").getState(),
            '\n',process.memory());

/*

Buid options: RELEASE=1 BOARD_4MB=1
SDK: 2.1 
Changes:
    targets/esp8266/espmissingincludes.h remove entries
    make/family/ESP8266.make: remove CFLAG mfore-l32, because of increase in size around 350K

To disassemble: xtensa-lx106-elf-objdump -d -l -x espruino_esp8266_user2.elf
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  3 .text         000076d4  40100000  40100000  00002d3c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  4 .irom0.text   0006c1e4  40201010  40201010  0000a410  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
To disassemble: xtensa-lx106-elf-objdump -d -l -x espruino_esp8266_user1.elf
xtensa-lx106-elf-objdump -d -l -x espruino_esp8266_user1.elf >espruino_esp8266_user1.lst
  8 -rwxr-xr-x  1 markbecker  staff    2188 15 Apr 07:20 eagle.app.v6.data.bin
872 -rwxr-xr-x  1 markbecker  staff  442852 15 Apr 07:20 eagle.app.v6.irom0text.bin
 24 -rwxr-xr-x  1 markbecker  staff    9164 15 Apr 07:20 eagle.app.v6.rodata.bin
 64 -rwxr-xr-x  1 markbecker  staff   30420 15 Apr 07:20 eagle.app.v6.text.bin
** user1.bin uses  484692 bytes of 962560 available

>
 {
  "VERSION": "1v96.3780",
  "GIT_COMMIT": "34d678da",
  "BOARD": "ESP8266_4MB",
  "FLASH": 0, "RAM": 81920,
  "SERIAL": "5ccf7f05-c100",
  "CONSOLE": "Serial1",
  "MODULES": "Flash,Storage,net,dgram,http,NetworkJS,Wifi,ESP8266,TelnetServer,crypto,neopixel",
  "EXPTR": 1073643636 }
 {
  "sdkVersion": "2.1.0(116b762)",
  "cpuFrequency": 160, "freeHeap": 12016, "maxCon": 10,
  "flashMap": "4MB:1024/1024",
  "flashKB": 4096,
  "flashChip": "0xe0 0x4016"
 }
 { "free": 1500, "usage": 100, "total": 1600, "history": 0,
  "gc": 0, "gctime": 1.324 }
=undefined
>

*/

@MaBecker
Copy link
Contributor Author

Build result for Espruino 1.96 with SDK 2.2 - Status: infinitely reboot loop after flash

using the targets/esp8266/eagle.app.v6.new.2048.ld ld complains:

/Volumes/ESPTools/esp-open-sdk/xtensa-lx106-elf/bin/../lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld: espruino_esp8266_user1.elf section `.text' will not fit in region `iram1_0_seg'
collect2: error: ld returned 1 exit status
make: *** [espruino_esp8266_user1.elf] Error 1
make: *** Waiting for unfinished jobs....
/Volumes/ESPTools/esp-open-sdk/xtensa-lx106-elf/bin/../lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld: espruino_esp8266_user2.elf section `.text' will not fit in region `iram1_0_seg'
collect2: error: ld returned 1 exit status
make: *** [espruino_esp8266_user2.elf] Error 1

@wilberforce
Copy link
Member

@MaBecker
espruino_esp8266_user1.elf section .text' will not fit in region iram1_0_seg'

This means that the RAM parts of firmware are now to big to fit within the RAM memory area.

Are there constants that are not declared as constants that could to ROM?

have you tried comping turning some features of such as Crypto?

@MaBecker
Copy link
Contributor Author

Looks like sdk 2.2 libs Need more space.

https://github.com/espressif/ESP8266_NONOS_SDK/blob/master/lib/readme.md

@MaBecker
Copy link
Contributor Author

Thanks @wilberforce for that hint.

Removing CRYPTO and GRAPHICS does not help....

Looks like Espressif did changes to their libs and now lots of stuff is puled into .irom0.text

increasing Memory

MEMORY
{
  dport0_0_seg :                        org = 0x3FF00000, len = 0x10
  dram0_0_seg :                         org = 0x3FFE8000, len = 0x14000
  iram1_0_seg :                         org = 0x40100000, len = 0x20000         <-  0x8000
  irom0_0_seg :                         org = 0x40201010, len = 0xE0000
}

result:

Idx Name          Size      VMA       LMA       File off  Algn
  3 .text         00016e28  40100000  40100000  00002d4c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  4 .irom0.text   0005c8e4  40201010  40201010  00019b80  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE

Sorry no idea how to handle this.

@MaBecker
Copy link
Contributor Author

Could need some help.

@wilberforce
Copy link
Member

It is ram that is the issue. You are constrained by the amount of physical ram -- so changing sizes won't help.

Try reducing the jsvars to see if for you can get a build.

I seem to recall the libc.a was added to the project at some point - you might want to link against the latest version.

@MaBecker
Copy link
Contributor Author

Well removing CRYPTO, GRAPHICS and shrink variable to 500 does not fix this.

@MaBecker
Copy link
Contributor Author

Change everything back to default and run a test with this ld file

https://github.com/nodemcu/nodemcu-firmware/blob/master/ld/nodemcu.ld

linking now works again

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00006464  40100000  40100000  00001b60  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  6 .irom0.text   0006d900  40210000  40210000  00009000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
To disassemble: xtensa-lx106-elf-objdump -d -l -x espruino_esp8266_user1.elf
xtensa-lx106-elf-objdump -d -l -x espruino_esp8266_user1.elf >espruino_esp8266_user1.lst
  8 -rwxr-xr-x  1 markbecker  staff    2188 17 Apr 16:47 eagle.app.v6.data.bin
880 -rwxr-xr-x  1 markbecker  staff  448768 17 Apr 16:47 eagle.app.v6.irom0text.bin
  8 -rwxr-xr-x  1 markbecker  staff       8 17 Apr 16:47 eagle.app.v6.rodata.bin
 56 -rwxr-xr-x  1 markbecker  staff   25700 17 Apr 16:47 eagle.app.v6.text.bin
** user1.bin uses  476724 bytes of 962560 available

@MaBecker
Copy link
Contributor Author

now way to boot this .....

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 17, 2018

put this 2.2 on hold and run test for esp01 and esp12 with sdk 2.1

@MaBecker MaBecker self-assigned this Aug 2, 2018
@MaBecker
Copy link
Contributor Author

MaBecker commented Aug 2, 2018

SDK after 2.2 do not work with because Espressif decided to compile some libraries with -ffunction-sectons -fdata-sections and modify ld to put this.text.* into .irom0.text

Question is it possible to reverse this with ld?

@gfwilliams
Copy link
Member

Is it bad that they've put the text sections into irom0? I would have thought that'd be good?

@MaBecker MaBecker changed the title ESP8266 NONOS SDK 2.1 ESP8266 NONOS SDK 2.1/2.2 Aug 2, 2018
@wilberforce
Copy link
Member

wilberforce commented Aug 2, 2018

Any idea of much it is too big by?

What size is the 4Mb firmware?

@MaBecker
Copy link
Contributor Author

MaBecker commented Aug 3, 2018

Any idea of much it is too big by?

No, but see attached file for all sections created with
xtensa-lx106-elf-objdump -h espruino_esp8266_partial.o > espruino_esp8266_user1.lst.txt
espruino_esp8266_user1.lst.txt

What size is the 4Mb firmware?

** user1.bin uses  528644 bytes of 962560 available

@MaBecker
Copy link
Contributor Author

MaBecker commented Aug 7, 2018

like to switch to SDK 2.2 or 2.2.1 for 4mb boards.

Gordon, how do you think about it?

@gfwilliams
Copy link
Member

What does it add?

And just for 4MB boards? Any chance we can keep the same SDK for all of them?

@MaBecker
Copy link
Contributor Author

MaBecker commented Aug 7, 2018

Wait, missed to replace eagle.app.v6.new.1024 files, now the SSDK 2.2 works for all ESP8266 boards.

What does it add?

If you mean features: To much to name it, so just added links below

Now starting test for wifi, i2c, spi and neopixel.

Any thing else to test?

@MaBecker MaBecker changed the title ESP8266 NONOS SDK 2.1/2.2 ESP8266 NONOS SDK 2.1/2.2/2.2.1 Aug 7, 2018
@gfwilliams
Copy link
Member

Then yes, I'd definitely be up for changing it.

It'd also be nice to get the relevant files into the EspruinoBuildTools repo so we're not thrashing @tve's S3 account with Travis :)

@MaBecker
Copy link
Contributor Author

How to run all test from folder test?

@gfwilliams
Copy link
Member

./espruino --test-all - there's always espruino --help.

There's no way to run tests on the device itself though.

@MaBecker
Copy link
Contributor Author

MaBecker commented Aug 15, 2018

Well I was thinking of some code like this to do on device testing (odt)

/* onDeviceTest_v0.js

try to run espruino test on device

*/


// for now static
// maybe store all in flash and load from flash test by test
// or load from webserver test by test
// or ....
catalog = [
  ["test_number_comparison.js",
   `var a = 42; result = a==42;`],
  ["test_tostring2.js",
    `0x8000000000000000.toString(2) // could segfault :)
     result = (8446744073709551616).toString(16) == "7538dcfb76180000"`
  ],
  ["test_undefined_behaviour.js",
`var testUndefined;        // variable declared but not defined, set to value of undefined
var testObj = {};  

var a = 0;
if ((""+testUndefined) != "undefined") a = 1; // test variable exists but value not defined, displays undefined
if ((""+testObj.myProp) != "undefined") a = 2; // testObj exists, property does not, displays undefined
if (!(undefined == null)) a = 3;  // unenforced type during check, displays true
if (undefined === null) a = 4;// enforce type during check, displays false
if (null != undefined) a = 5;  // unenforced type during check, displays true
if (null === undefined) a = 6; // enforce type during check, displays false
if (undefined != undefined) a = 7;
if (!(undefined == undefined)) a = 8;
result = a==0;`]
];

function onDeviceTest(test){
  var result, error;
  try {
    result = eval(test[1]);
  } catch(e) {
    error = e;
    result = false;
  }
  console.log(test[0],'[',result,',',error,']');
}

for( i = 0; i< catalog.length;i++)
  onDeviceTest(catalog[i]);

@MaBecker
Copy link
Contributor Author

got Espruino Command-line Tool 0.1.12, but looks like this is not the one to be used for --test-all

./espruino --test-all

Marks-MacBook-Pro:Espruino markbecker$ espruino --test-all --board ESP8266_4MB -p /dev/cu.usbserial -b 115200
/usr/local/Cellar/node@8/8.9.4/lib/node_modules/espruino/bin/espruino-cli.js:139
} else throw new Error("Unknown Argument '"+arg+"', try --help");
^

Error: Unknown Argument '--test-all', try --help
at Object. (/usr/local/Cellar/node@8/8.9.4/lib/node_modules/espruino/bin/espruino-cli.js:139:17)

@wilberforce
Copy link
Member

the espruino in this case is the linux target.

make clean && make

This will make ./espruino in your build folder which you can then run.

@gfwilliams
Copy link
Member

I've been after doing actual on-device testing for some time - this issue really isn't the place to discuss that though. I started a new issue at #1490

@MaBecker
Copy link
Contributor Author

Updated EspruinoBuildTools, now there are all available 2.x SDKS.

Decide which to choose after testing.

@MaBecker
Copy link
Contributor Author

MaBecker commented Sep 1, 2018

add this CFLAG to get approximately 2.5k less in size, found here

-fno-guess-branch-probability -freorder-blocks-and-partition -fno-cse-follow-jumps 

without

** user1.bin uses  458324 bytes of 479232 available        ESP8266_BOARD
** user1.bin uses  516564 bytes of 962560 available        ESP8266_4MB

with

** user1.bin uses  455796 bytes of 479232 available        ESP8266_BOARD
** user1.bin uses  513972 bytes of 962560 available        ESP8266_4MB

@MaBecker
Copy link
Contributor Author

MaBecker commented Sep 2, 2018

will be updated and extended

  • wifi tests created by tve
    ....
  • Wifi station
    • wifi connect to AP
    • events connect / disconnect
    • wifi save / restore / reconnect after reboot
  • Wifi AP
    • startAP / stopAP
    • events stat_joined / stat_left
    • save / restore station settings
  • TCP/IP
    • Espruino WEB IDE over TCP/IP
    • simple http server with page
    • http server turn led on/off using GET and POST
    • mqtt send and receive
    • websocket-server
    • use ./scripts/wiflash.sh to update
  • modules
    • neopixel
    • i2c oled
    • DHT22/AM230x/RHT0x Temperature and RH Sensor
    • DS18B20 Temperature Sensor (OneWire)
      ....

@gfwilliams
Copy link
Member

add this CFLAG to get approximately 2.5k less in size

Perfect - lets definitely do that then!

@MaBecker
Copy link
Contributor Author

Discovered that ESP8266 AP is going down sometimes for some seconds even if stations are connected.

bildschirmfoto 2018-09-10 um 23 35 51
Monitored with Wifi Explorer for Mac OS X

@MaBecker
Copy link
Contributor Author

MaBecker commented Oct 15, 2018

How to preapare Travis build stuff?

Wanted SDK is 2.2.1 and it is uploaded as zip file to folder https://github.com/espruino/EspruinoBuildTools/tree/master/esp8266

@wilberforce
Copy link
Member

@gfwilliams
Copy link
Member

How ready are we to merge this? I'd like to get a new release of Espruino out this week (ideally tomorrow).

Sounds like it's unlikely we'd get all the tools put in and have it tested in the next 2 days?

@MaBecker
Copy link
Contributor Author

Thanks @wilberforce this was exactly what I was looking for.

Ok Gordon lets cross fingers an see how travis will handle this.

@MaBecker
Copy link
Contributor Author

 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 1v99.4205 (c) 2018 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:1024/1024, manuf 0xe0 chip 0x4016
>
 {
  "VERSION": "1v99.4205",
  "GIT_COMMIT": "066cd080",
  "BOARD": "ESP8266_4MB",
  "FLASH": 0, "RAM": 81920,
  "SERIAL": "5ccf7f05-c100",
  "CONSOLE": "Serial1",
  "MODULES": "Flash,Storage,heatshrink,net,dgram,http,NetworkJS,Wifi,ESP8266,TelnetServer,crypto,neopixel",
  "EXPTR": 1073643636 }
 {
  "sdkVersion": "2.2.1(6ab97e9)",
  "cpuFrequency": 160, "freeHeap": 11008, "maxCon": 10,
  "flashMap": "4MB:1024/1024",
  "flashKB": 4096,
  "flashChip": "0xe0 0x4016"
 }
 { "free": 1492, "usage": 108, "total": 1600, "history": 6,
  "gc": 0, "gctime": 1.812 }
[
  "Flash",
  "Storage",
  "heatshrink",
  "net",
  "dgram",
  "http",
  "NetworkJS",
  "Wifi",
  "ESP8266",
  "TelnetServer",
  "crypto",
  "neopixel"
 ]
> 

gfwilliams added a commit that referenced this issue Oct 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement ESP8266 This is only a problem on ESP8266 devices
Projects
None yet
Development

No branches or pull requests

3 participants