Improvement request: Network upload should use password entered in IDE next to Port ("Set or remove password") when no password is found in the code

[gadolchik]

Do you use the Sloeber product or plugin? BOTH
Sloeber version number Sloeber 4.4.1.202109151310 (same problem in 4.4.0)
OS: windows 10
The url of the json that you used to download the platform (my 3rd party index urls)
https://downloads.arduino.cc/packages/package_index.json
https://raw.githubusercontent.com/jantje/hardware/master/package_jantje_index.json
https://raw.githubusercontent.com/jantje/ArduinoLibraries/master/library_jantje_index.json
https://arduino.esp8266.com/stable/package_esp8266com_index.json
https://downloads.arduino.cc/libraries/library_index.json

Describe the bug

The dialog for set / remove password updates security_login and security_password.

The esp TCP OTA upload only looks for network_password. If the plugin can't find the password in the main file, it fails to authenticate.

It would be very nice if the OTA upload accepted the security_password when network_password can't be found in the source file.

To Reproduce
Try to use TCP OTA upload with no password in the .ino file, setting one in the IDE with 'Set or remove password' makes no difference, authentication fails.

provide the name of the board and the platform version you are using.
NodeMCU 1.0, ESP8266 3.02
Provide the json url and a screenshot of project properties->arduino

[jantje]

As there is no port in the upload port specified the upload can not succeed.
To activate the network upload you need to provide a port with a space.
The part before the space will be used as the ipadress

[gadolchik]

Thanks for replying!

When uploading I enter the IP address in the port field - it was "192.168.1.31 tcpOTA" in this case.

The problem remains the same - cannot use the adjacent 'Set or remove password' dialog to provide a password for the OTA, it tries to run the esp ota tool with --auth=no-password-found-in-code or similar argument, regardless of login info entered in the IDE.

Looking at the plugin code it only seems to look for network_password in the code but does not use the security_password entered in the IDE.

It would be nice if it did, some deployments only have the OTA authentication available at run time.

[jantje]

I tried here and I see indeed Sloeber did not pick up the pwd from the security tab
Starting upload
Uploading project "dd" with "esptool"
no reset because we are using network upload

Launching: e:\platform\arduinoPlugin\packages\esp8266\tools\python3\3.7.2-post1/python3 -I e:\platform\arduinoPlugin\packages\esp8266\hardware\esp8266\3.0.2/tools/espota.py -i 1.1.1.1 -p 8266 --auth=no_pwd_found_in_code -f L:\test\runtime-EclipseApplication_eng\dd\Release/dd.bin 
Output:
The execution of command "3.7.2-post1/python3" is aborted.
failed to upload "dd"

[jantje]

Seems the logon/password was only ever used to do a ssh upload (arduino yun(shield) specific code) and no where else
Also seems I removed the ssh upload :-s #1388
But I agree this could/should be used for ota upload
Main question is: do we prioritize the password set in code or the one set in the properties?

[jantje]

I change the code so it will use the code found password; if no password found in the code it will search the secure storage for a password linked to the port name and if none is found it will use the password "no_pwd_configured_nor_found_in_code"
You should be able to test this with the nightly in about 12 hours

[gadolchik]

Thanks!

Regarding the question - in my opinion the IDE properties are more dynamic than the code - that is, if there is an entry in the IDE, use it regardless of the code.

For example, if part of the code upload is changing the auth password, and we can't override the password in the IDE, we can't authenticate against the old password to perform the upload ...
If the IDE properties override anything found in the code it's much simpler to manage.

[jantje]

Please confirm this works on your side

Note
During testing I found that the options "no" and "never" resulted in no upload. Fixed that as well.

[gadolchik]

I will test the nightly, just interested if you would consider having the IDE properties as an override to code search - that would allow network upload that has new auth password in the code while using the current password in the IDE.

[jantje]

Thanks!

Regarding the question - in my opinion the IDE properties are more dynamic than the code - that is, if there is an entry in the IDE, use it regardless of the code.

For example, if part of the code upload is changing the auth password, and we can't override the password in the IDE, we can't authenticate against the old password to perform the upload ...
If the IDE properties override anything found in the code it's much simpler to manage.

I accept your comments but ....
Using the code password is the more natural way and can easily be disabled (see below).
Changing the password is a hassle in both cases. To do so it is probably easiest to -at some point in time- run the upload command from the command line or use the new target functionality in a custom makefile.init.
Unfortunately I could not get ${tools.${upload.tool}.upload.network_pattern} to work in the makefile.init as the variables are expanded before make is started (in windows)
But this makefile.init worked fine (Well ... I do not have a esp to test but it looks ok)

OTASecureUploadWithBuild : network.password=mySecureNetwork
OTASecureUploadWithBuild : network.port=1234
OTASecureUploadWithBuild:
	@echo trying to build and upload with custom password and upload tool ${upload.tool}
	"${tools.esptool.network_cmd}" -I "${runtime.platform.path}/tools/espota.py" -i "${serial.port}" -p "${network.port}" "--auth=${network.password}" -f "${build.path}/${build.project_name}.bin"

This basically creates the target OTASecureUploadWithBuild which sets 2 environment variables and runs a command. The command is a literal copy of tools.esptool.upload.network_pattern form platform.sloeber.txt.
The easiest way I found to "build the target" OTASecureUploadWithBuild is by

1. copy OTASecureUploadWithBuild to the clipboard

2. open release/makefile.

3. in the outline view search for a target (like all, BurnBootloader,uploadWithBuild...)
   

4. in the outline view right click the target and select add build target

5. Past OTASecureUploadWithBuild into the field targetname

6. press ok
   now you will see the OTASecureUploadWithBuild target in release/build targets and you can double click it to run it
   If you delete the release folder it will reappear after the next build
   If you want a com port to disconnect during the upload put the com port in the project properties->arduino->port field and add OTASecureUploadWithBuild to menu->arduino->preferences->build targets
   

Note that for the code password to work you need to literally use ArduinoOTA.setPassword in setup() in active code (no comments nor defined out).
Simply putting ArduinoOTA.setPassword in a method (for example configureNetwork) called by setup will make the security storage password to be used.
Even defining MY_OTA as ArduinoOTA and using MY_OTA .setPassword in setup() will lead to password not found in the code.
Also note that this has some drawbacks as exactly the same goes for ArduinoOTA.setPort which defaults to 8266 and which can not be set in the gui.

[gadolchik]

I can confirm that entering the IP address in 'Login:' and the OTA password in 'Password:' in the 'Set or remove password' fields works - authentication succeeds and OTA upload completes.

For my use cases this is sufficient without needing makefile modifications.

Thanks for your help.

[jantje]

thanks for the confirmation