Skip to content

Added desription of Arduino IDE OTA process to documentation #984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Nov 10, 2015
Merged

Added desription of Arduino IDE OTA process to documentation #984

merged 9 commits into from
Nov 10, 2015

Conversation

krzychb
Copy link
Contributor

@krzychb krzychb commented Nov 8, 2015

Following http://www.esp8266.com/viewtopic.php?p=33235#p33235
Keep up development of this great platform 😄 👍

@mangelajo
Copy link
Contributor

You now have the ArduinoOTA lib that encapsulates this, no need to use the mDNS examples.

@mangelajo
Copy link
Contributor

good work, by the way!, documentation is always awesome

@igrr igrr merged commit 421206f into esp8266:master Nov 10, 2015
@krzychb
Copy link
Contributor Author

krzychb commented Nov 11, 2015

@ mangelajo - this is good point to use simpler functionality encapsulated in “ArduinoOTA “ library.

I am looking forward to include it in wiki and propose to do it once new json release is published with this library build in (I assume this is your plan).

The other way around is to change instruction from “Installing with Boards Manager” to “Using git version”. Loading “ArduinoOTA” seems to me not enough if I use staging release and "OTALeds.ino" example. I see compiler complaining about missing "MD5Builder.h" that is part of the core on git.

Seeing repetitive questions asked about OTA on esp8266.com forum I prepared this instruction having beginners in mind. My impression is that adding extra complexity to it would discourage people rather than attract. Extended number of steps == more chances to fail and even more questions.

By making it simpler and referring to proven code we should reduce the same questions asked over and over. As result you will be less distracted and could do what I believe you like doing that is adding new exciting functionality to this platform 😄

What do you think?

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

Guys,
just forked the repo to add bargraph to espota.py, then tried what's the documentation for OTA says but seems the sketch described in it https://github.com/esp8266/Arduino/blob/master/doc/ota_updates/ota_updates.md
is not there, when I try
"Start Arduino IDE and load sketch DNS_SD_Arduino_OTA.ino available under File > Examples > ESP8266mDNS"

Looking the repo sktech, it's not there ?https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266mDNS/examples
Did I missed something ?

@igrr
Copy link
Member

igrr commented Nov 11, 2015

@hallard there were two parallel pull requests: this one and another one from @me-no-dev related to OTA improvements. DNS_SD_Arduino_OTA.ino was gone as part of the second one:
a8976a0.
Just need to update docs to use ArduinoOTA library and its examples.

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

@igrr
ok thanks, no problem, will just get this one, test my change in espota.py and do a pull request ;-)

@me-no-dev
Copy link
Collaborator

@hallard what are you adding to espota.py?

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

@me-no-dev
just answered there in the meantime you were asking the question ;-)
a8976a0

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

to be able to have something like that, I always like bargraph ;-)
image

@igrr
Copy link
Member

igrr commented Nov 11, 2015

Does this work fine when output is piped by Arduino to its console? I had issues when i used \r to generate progress indicator.

@me-no-dev
Copy link
Collaborator

no, needs to print to stderr for it to show :)

@me-no-dev
Copy link
Collaborator

and goes miltiline :)

python /Users/ficeto/Desktop/ESP8266/Arduino-Main/build/macosx/work/Arduino.app/Contents/Java/hardware/esp8266com/esp8266/tools/espota.py -i 192.168.254.182 -p 8266 --auth= -f /var/folders/b_/2gy4g72944359d2vj4htbtwm0000gn/T/build6c445534217ff306d2c48161a213eacf.tmp/ESP_FSBrowser.ino.bin 

Uploading: [                                                            ] 0% 
Uploading: [                                                            ] 0% 
Uploading: [=                                                           ] 1% 
Uploading: [=                                                           ] 1% 
Uploading: [=                                                           ] 2% 
Uploading: [==                                                          ] 2% 
Uploading: [==                                                          ] 3% 
Uploading: [==                                                          ] 3% 
Uploading: [===                                                         ] 4% 
Uploading: [===                                                         ] 4% 
Uploading: [===                                                         ] 5% 
Uploading: [====                                                        ] 5% 
Uploading: [====                                                        ] 6% 
Uploading: [====                                                        ] 7% 
Uploading: [=====                                                       ] 7% 
Uploading: [=====                                                       ] 8% 
Uploading: [=====                                                       ] 8% 
Uploading: [======                                                      ] 9% 
Uploading: [======                                                      ] 9% 
Uploading: [======                                                      ] 10% 
Uploading: [======                                                      ] 10% 
Uploading: [=======                                                     ] 11% 
Uploading: [=======                                                     ] 11% 
Uploading: [=======                                                     ] 12% 
Uploading: [========                                                    ] 12% 
Uploading: [========                                                    ] 13% 
Uploading: [========                                                    ] 14% 
Uploading: [=========                                                   ] 14% 
Uploading: [=========                                                   ] 15% 
Uploading: [=========                                                   ] 15% 
Uploading: [==========                                                  ] 16% 
Uploading: [==========                                                  ] 16% 
Uploading: [==========                                                  ] 17% 
Uploading: [===========                                                 ] 17% 
Uploading: [===========                                                 ] 18% 
Uploading: [===========                                                 ] 18% 
Uploading: [============                                                ] 19% 
Uploading: [============                                                ] 20% 
Uploading: [============                                                ] 20% 
Uploading: [=============                                               ] 21% 
Uploading: [=============                                               ] 21% 
Uploading: [=============                                               ] 22% 
Uploading: [==============                                              ] 22% 
Uploading: [==============                                              ] 23% 
Uploading: [==============                                              ] 23% 
Uploading: [===============                                             ] 24% 
Uploading: [===============                                             ] 24% 
Uploading: [===============                                             ] 25% 
Uploading: [================                                            ] 25% 
Uploading: [================                                            ] 26% 
Uploading: [================                                            ] 27% 
Uploading: [=================                                           ] 27% 
Uploading: [=================                                           ] 28% 
Uploading: [=================                                           ] 28% 
Uploading: [==================                                          ] 29% 
Uploading: [==================                                          ] 29% 
Uploading: [==================                                          ] 30% 
Uploading: [===================                                         ] 30% 
Uploading: [===================                                         ] 31% 
Uploading: [===================                                         ] 31% 
Uploading: [===================                                         ] 32% 
Uploading: [====================                                        ] 33% 
Uploading: [====================                                        ] 33% 
Uploading: [====================                                        ] 34% 
Uploading: [=====================                                       ] 34% 
Uploading: [=====================                                       ] 35% 
Uploading: [=====================                                       ] 35% 
Uploading: [======================                                      ] 36% 
Uploading: [======================                                      ] 36% 
Uploading: [======================                                      ] 37% 
Uploading: [=======================                                     ] 37% 
Uploading: [=======================                                     ] 38% 
Uploading: [=======================                                     ] 38% 
Uploading: [========================                                    ] 39% 
Uploading: [========================                                    ] 40% 
Uploading: [========================                                    ] 40% 
Uploading: [=========================                                   ] 41% 
Uploading: [=========================                                   ] 41% 
Uploading: [=========================                                   ] 42% 
Uploading: [==========================                                  ] 42% 
Uploading: [==========================                                  ] 43% 
Uploading: [==========================                                  ] 43% 
Uploading: [===========================                                 ] 44% 
Uploading: [===========================                                 ] 44% 
Uploading: [===========================                                 ] 45% 
Uploading: [============================                                ] 45% 
Uploading: [============================                                ] 46% 
Uploading: [============================                                ] 47% 
Uploading: [=============================                               ] 47% 
Uploading: [=============================                               ] 48% 
Uploading: [=============================                               ] 48% 
Uploading: [==============================                              ] 49% 
Uploading: [==============================                              ] 49% 
Uploading: [==============================                              ] 50% 
Uploading: [===============================                             ] 50% 
Uploading: [===============================                             ] 51% 
Uploading: [===============================                             ] 51% 
Uploading: [===============================                             ] 52% 
Uploading: [================================                            ] 53% 
Uploading: [================================                            ] 53% 
Uploading: [================================                            ] 54% 
Uploading: [=================================                           ] 54% 
Uploading: [=================================                           ] 55% 
Uploading: [=================================                           ] 55% 
Uploading: [==================================                          ] 56% 
Uploading: [==================================                          ] 56% 
Uploading: [==================================                          ] 57% 
Uploading: [===================================                         ] 57% 
Uploading: [===================================                         ] 58% 
Uploading: [===================================                         ] 58% 
Uploading: [====================================                        ] 59% 
Uploading: [====================================                        ] 60% 
Uploading: [====================================                        ] 60% 
Uploading: [=====================================                       ] 61% 
Uploading: [=====================================                       ] 61% 
Uploading: [=====================================                       ] 62% 
Uploading: [======================================                      ] 62% 
Uploading: [======================================                      ] 63% 
Uploading: [======================================                      ] 63% 
Uploading: [=======================================                     ] 64% 
Uploading: [=======================================                     ] 64% 
Uploading: [=======================================                     ] 65% 
Uploading: [========================================                    ] 66% 
Uploading: [========================================                    ] 66% 
Uploading: [========================================                    ] 67% 
Uploading: [=========================================                   ] 67% 
Uploading: [=========================================                   ] 68% 
Uploading: [=========================================                   ] 68% 
Uploading: [==========================================                  ] 69% 
Uploading: [==========================================                  ] 69% 
Uploading: [==========================================                  ] 70% 
Uploading: [===========================================                 ] 70% 
Uploading: [===========================================                 ] 71% 
Uploading: [===========================================                 ] 71% 
Uploading: [============================================                ] 72% 
Uploading: [============================================                ] 73% 
Uploading: [============================================                ] 73% 
Uploading: [============================================                ] 74% 
Uploading: [=============================================               ] 74% 
Uploading: [=============================================               ] 75% 
Uploading: [=============================================               ] 75% 
Uploading: [==============================================              ] 76% 
Uploading: [==============================================              ] 76% 
Uploading: [==============================================              ] 77% 
Uploading: [===============================================             ] 77% 
Uploading: [===============================================             ] 78% 
Uploading: [===============================================             ] 79% 
Uploading: [================================================            ] 79% 
Uploading: [================================================            ] 80% 
Uploading: [================================================            ] 80% 
Uploading: [=================================================           ] 81% 
Uploading: [=================================================           ] 81% 
Uploading: [=================================================           ] 82% 
Uploading: [==================================================          ] 82% 
Uploading: [==================================================          ] 83% 
Uploading: [==================================================          ] 83% 
Uploading: [===================================================         ] 84% 
Uploading: [===================================================         ] 84% 
Uploading: [===================================================         ] 85% 
Uploading: [====================================================        ] 86% 
Uploading: [====================================================        ] 86% 
Uploading: [====================================================        ] 87% 
Uploading: [=====================================================       ] 87% 
Uploading: [=====================================================       ] 88% 
Uploading: [=====================================================       ] 88% 
Uploading: [======================================================      ] 89% 
Uploading: [======================================================      ] 89% 
Uploading: [======================================================      ] 90% 
Uploading: [=======================================================     ] 90% 
Uploading: [=======================================================     ] 91% 
Uploading: [=======================================================     ] 91% 
Uploading: [========================================================    ] 92% 
Uploading: [========================================================    ] 93% 
Uploading: [========================================================    ] 93% 
Uploading: [========================================================    ] 94% 
Uploading: [=========================================================   ] 94% 
Uploading: [=========================================================   ] 95% 
Uploading: [=========================================================   ] 95% 
Uploading: [==========================================================  ] 96% 
Uploading: [==========================================================  ] 96% 
Uploading: [==========================================================  ] 97% 
Uploading: [=========================================================== ] 97% 
Uploading: [=========================================================== ] 98% 
Uploading: [=========================================================== ] 99% 
Uploading: [============================================================] 99% 
Uploading: [============================================================] 100% Done...

@me-no-dev
Copy link
Collaborator

maybe add it as an option?

@igrr
Copy link
Member

igrr commented Nov 11, 2015

Maybe just remove percentage indicator? This way we won't need to use \r
and output will be single line.

On Wed, Nov 11, 2015 at 6:39 PM, Me No Dev [email protected] wrote:

maybe add it as an option?


Reply to this email directly or view it on GitHub
#984 (comment).

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

that's strange it was working fine on my env some months ago, need to check this

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

on my computer under Win10 cmd and cygwin works fine without new lines, I need to test under IDE as soon as I can see my device in network board list
image

@me-no-dev
Copy link
Collaborator

here is the pull with the optional progress: https://github.com/esp8266/Arduino/pull/1003/files
use -r or --progress to see it

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

@me-no-dev
I installed this Arduino build, but unfortunately I still don't see ESP8266 device in network port name
Device just updated with OTA,

  • serial COM6 see the log and display ip address, http server started (sketch running)
  • bonjour browser see the service and name declared WifInfo_fcb401
  • cmd prompt can ping device by name WifInfo_fcb401.local
  • and Arduino IDE just see serial port !!

But don't worry, under windows seeing network device using Arduino IDE has been a pain from beginning, You can't imagine how much time I lost trying to get this working, I tough this time with 1.6.6 and build 1.6.7 will correct, but still not.

image

@me-no-dev
Copy link
Collaborator

Are you using the latest ESP8266 core from git? The changes necessary for it all to work were merged yesterday. There is an updated BasicOTA example in the ArduinoOTA library that has all new code needed/available.

@me-no-dev
Copy link
Collaborator

my bad :) visible in the bonjour browser :) no clue then...

@hallard
Copy link
Contributor

hallard commented Nov 11, 2015

Don't worry I'm doing on command line for months now, already had this issue with dragino and arduino yun, Of course tried with Windows Firewall disabled ;-)
I'm just telling you I can't test newline on espota.py progress bar under win10 and arduino IDE ;-(

@krzychb
Copy link
Contributor Author

krzychb commented Nov 12, 2015

@hallard – your documentation of issue with network port is excellent. Thank you for investing your time and providing such level of details.

I have exactly the same issue and was about to document it as I am preparing an update of documentation to cover new ArduinoOTA library.

My testing was done on 11.11.2015 late afternoon CET and I made fresh installation of Arduino IDE 1.6.5-r5 for Windows and fresh "git clone https://github.com/esp8266/Arduino.git esp8266". I tried both “BasicOTA.ino” as well as “OTALeds.ino” provided among examples. I have exactly the same results as you had – the module successfully joins the network but the OTA network port does not show up in IDE.

Could you, please, point me to your work around / what command line you are using to make the ArduinoOTA work for Windows?

Krzysztof

@hallard
Copy link
Contributor

hallard commented Nov 12, 2015

@krzychb
yeah, no pb, I will do it by the end of the day (French time) I'm in a hurry now ;-)

By the way, I wanted to tell that OTA now rocks !!! I've been updated my device la least 100 times yesterday without any lost, even if I had browser requesting the device in the meantime (before this was causing trouble)
Nice work to all of this team ;-)

@me-no-dev
Copy link
Collaborator

now imagine my disappointment after spending so much time on OTA, that Windows refuses to see the devices... and there is nothing I can do about it

@me-no-dev
Copy link
Collaborator

Any chance this can be caused by Antivirus software?

@hallard
Copy link
Contributor

hallard commented Nov 12, 2015

I've just got MS Windows Defender, that's all
Don't worry @me-no-dev some windows users report this to be working on Arduino Forum, it was working sometimes on my old Windows 8.1 but this is the 1st try I test on windows 10

@me-no-dev
Copy link
Collaborator

found fine on both 8.1 and XP that I have around. Will look for v10 download to give it a shot.
Defender could also be causing this if it's thinking that the mDNS is doing something wrong opening a multicast port

@hallard
Copy link
Contributor

hallard commented Nov 12, 2015

  • Installed Apple Bonjour Service
  • Disabled Windows Defender
  • Disabled Windows firewall

same thing :-( :-(

@me-no-dev
Copy link
Collaborator

Oh... a hunch... maybe remove Bonjour? It could be clashing since they use the same port

@hallard
Copy link
Contributor

hallard commented Nov 12, 2015

I've just installed it to give a try (it was not before last hour), but I have an idea, let me check something

@me-no-dev
Copy link
Collaborator

Default clean install:
screen shot 2015-11-12 at 15 37 19

@krzychb
Copy link
Contributor Author

krzychb commented Nov 12, 2015

@me-no-dev – don’t worry – basing on responses so far I am certain that you and other guys there are a step apart from finding the reason of this issue or a workaround 😄

Looking through new ArduionOTA code it believe it is very versatile and I can’t wait trying out all the functions you have implemented.

To supplement information provided I am using Windows 7. The OTA port shows up when I am using the legacy “DNS_SD_Arduino_OTA.ino” sketch.

Krzysztof

@me-no-dev
Copy link
Collaborator

@krzychb thanks for the good words :)
You can grab the windows build from above and download the latest core as zip from the repo and since the build is portable, you can have it on a side and try the new OTA :)
I have to say, I can't wait for the Arduino team to merge so we can finally have a fully working OTA. Flashing is a breeze now and the result is not only consistent, but I really have not had a failed OTA since the changes :)

@sticilface
Copy link
Contributor

I agree the new OTA, seems a lot faster and I've not had a single fail. Good work @me-no-dev

@me-no-dev
Copy link
Collaborator

to be honest with you all, the change that made OTA work consistently was replacing yield() with delay(1).
The issue manifested itself in a really strange way... the connection hangs and as soon as the computer closes it as a result of the timeout, the data that was supposed to come from the ESP is sent with the first FIN packet. I tried a few things and looked much deeper, then as a last resort tried with some delay instead. And whoa !!!
So keep in mind when you do network projects and have issues, to try something that simple and see if things improve.

@igrr
Copy link
Member

igrr commented Nov 12, 2015

As mentioned by @EUA in #1005 this might have been related to light sleep mode (which is the default one).

I had an issue with network and light sleep in one of my sketches, where I was sending telemetry from a UAV (plane) to ground station. When I disabled light sleep, the number of dropped UDP packets went down by two orders of magnitude.

@me-no-dev
Copy link
Collaborator

unrelated: what range have you tested on the UAV?

@sticilface
Copy link
Contributor

That is interesting, I might have a play with that.
I was / have been trying to implement self/downloading firmware and files for SPIFFS, and adding delay, but not yield fixed it for HTTP only downloads. I thought the code was looping too fast and the buffer being emptied to quickly, so a delay give it time to accumulate. might be total BS though.

@me-no-dev
Copy link
Collaborator

based on how TCP works, one packet with data and one response with data look like this:

// PC -----> DATA ----> ESP
// PC <----- ACK <----- ESP //needs yield
// PC <- RESPONSE <- ESP //needs another yield
// PC -----> ACK ----> ESP //might also need yield

so having lots of data will lead to not enough empty loops/yields to be ale to transmit and receive all packets
If on other hand the situation is like @igrr is saying, the wifi is napping too much and needing more yields to wake up and do it's thing

@krzychb
Copy link
Contributor Author

krzychb commented Nov 12, 2015

Basing on previous posts I did the following steps:

  1. Uninstalled Java from my PC
  2. Uninstalled Arduino IDE 1.6.5-r5
  3. Installed Arduino IDE 1.6.6
  4. Checked for network OTA port - it was there!!!
  5. Installed latest esp8266 cores from Github
  6. Checked for network port - it was gone 😞
  7. Tried to replicate this sequence – the port did not show up again.

So I am back to step zero and keep looking for clues 😄
BTW - this is all on Windows 7 PC

@me-no-dev – I wonder if you tried to load esp module from your Windows set up and if the network port was not gone at some certain point?

Krzysztof

@me-no-dev
Copy link
Collaborator

@krzychb Yes I did. I installed python and did OTA. The device showed as soon as I opened the IDE and pulled the ports menu.

@me-no-dev
Copy link
Collaborator

Also tested on Windows 8.1 and Windows XP with success as well

@igrr
Copy link
Member

igrr commented Nov 12, 2015

@krzychb i'm concerned about your step 3. I think you need to get one of the builds mentioned by @me-no-dev.
The reason is that these IDE builds do not attempt to check port 8266 for being reachable over TCP.

Older version of OTA sample works because it also initializes a TCP server on port 8266, and IDE reachability check passes.

@krzychb
Copy link
Contributor Author

krzychb commented Nov 12, 2015

@igrr – every thing is hunky-dory now!

capture

I will leave this special screen on my PC overnight it took so much sweat and tear 😄

I was indeed intrigued where you got Arduino 1.6.7 from and did not catch that you coin own by yourself. You guys are awesome 👍

Krzysztof

@tzapu
Copy link
Contributor

tzapu commented Nov 18, 2015

i can't get the port to show up in os x either
#1039

@aminbehravan
Copy link

http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-4107-BUILD-421-windows.zip
404 error not found. it cant be download.
actually all links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants