Changed ant build script to use http with launcher4j. #5772

Chris--A · 2016-12-28T11:59:50Z

Due to sourceforge using a 307 temporary redirect, the build script now fails to retrieve the launcher package (windows/linux).
This is because the build script uses the https protocol and the redirect uses http.

Error from ant script:

[get] Redirection detected from https to http. Protocol switch unsafe, not allowed.

This is a simple fix as it now requests the download using http, so the redirect is allowed.

Due to sourceforge using a 307 temporary redirect, the build script now fails to retrieve the launcher package (windows/linux). This is because the build script uses the https protocol and the redirect uses http. Error from ant script "[get] Redirection detected from https to http. Protocol switch unsafe, not allowed." This is a simple fix as it now requests the download using http, so the redirect is allowed.

Chris--A · 2016-12-28T12:13:03Z

Just found this commit which changed all downloads to https: 1da9108

@NicoHood

NicoHood · 2016-12-28T12:15:06Z

The build always worked for me. Why does this happen? We should not use http as a solution.

cmaglie · 2016-12-28T12:20:16Z

This is my output:

download-launch4j-linux:

untar-unzip-check:

untar-unzip-download:

untar-unzip-download-web:
      [get] Getting: https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-linux.tgz
      [get] To: /home/megabug/Code/arduino/build/windows/launch4j-3.9-linux.tgz
      [get] https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-linux.tgz moved to https://netcologne.dl.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-linux.tgz
      [get] ....................................................
      [get] ....................................................
      [get] ....................................................
      [get] ....................................................
      [get] ....................................................
      [get] ....................................................
      [get] ....................................................
      [get] ....................................................
      [get] ..............

untar-unzip-download-local:

so it seems that sourceforge doesn't redirect to http.
@Chris--A
can you paste the error message?

Chris--A · 2016-12-28T13:15:11Z

I haven't actually tried the linux build, only windows. However the linux package is also redirected (307).

The build always worked for me. Why does this happen? We should not use http as a solution.

I didn't notice it until I re-built my repo from scratch (previously downloaded launcher/temp build files deleted). The redirect is only recent... I think.

Here is the output:

download-launch4j-windows:

untar-unzip-check:

untar-unzip-download:

untar-unzip-download-web:
      [get] Getting: https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-win32.zip
      [get] To: U:\Arduino\build\windows\launch4j-3.9-win32.zip
      [get] https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-win32.zip moved to http://internode.dl.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-win32.zip
      [get] Redirection detected from https to http. Protocol switch unsafe, not allowed.

untar-unzip-download-local:

untar-unzip-checksum:
     [echo] Testing checksum of "windows/launch4j-3.9-win32.zip"
 [checksum] Could not find file U:\Arduino\build\windows\launch4j-3.9-win32.zip to generate checksum for.

BUILD FAILED
U:\Arduino\build\build.xml:103: The following error occurred while executing this line:
U:\Arduino\build\build.xml:1047: The following error occurred while executing this line:
U:\Arduino\build\build.xml:955: The following error occurred while executing this line:
U:\Arduino\build\build.xml:858: Could not find file U:\Arduino\build\windows\launch4j-3.9-win32.zip to generate checksum for.

Changing to http allows the redirect to occur.

NicoHood · 2016-12-28T13:32:03Z

We better write sourceforge an email to provide https connections instead of using http.

cmaglie · 2016-12-28T16:03:07Z

It seems that some of their mirrors supports only http.

Internode mirror force http:

      [get] https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-win32.zip moved to http://internode.dl.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-win32.zip
      [get] Redirection detected from https to http. Protocol switch unsafe, not allowed.

Netcologne mirror insteal allows https:

      [get] https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-linux.tgz moved to https://netcologne.dl.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-linux.tgz

We better write sourceforge an email to provide https connections instead of using http.

Are you going to write them?
In the meantime what do you suggest?

NicoHood · 2016-12-28T16:58:23Z

@cmaglie I suggest some protest. Such a big site should take their business serious. I can write them, but you as arduino.cc might have some more impact. But I will try. In the meantime you can generate your gpg key #5619 .

It is really important, I cannot say it often enough. Please sign your sources. You do not need to use it yourself in the build script if you dont care, but please give others the chance to do so.

And once more as an explanation:

The download via https does not secure us from source tampering. The initial download can be tampered. The hash we generate from this initial download is trusted by us, but does not guarantee the origin of the source. Therefore we need to use GPG.

The real solution here would be that launch4j signs their sources and we verify them. Then you can create a hash of that download and include it in your build script. But DO NOT blindly download sources, generate a hash and use this.

The use of https (or sftp)over http helps us a bit in the first place of the download. It ensures the server is hopefully the server. While still a lot people do not trust the CA model. If you can trust the hash (it has to be sha256 at a minimum), then you can also download via http or ftp. The reason why we do not want to use http still is a reason of privacy. Which might not be that critical in europe (yet) but in other countries it is.

So:

I go ahead and write sourceforge about the https mirrors [done]
I You go ahead and write launch4j about gpg signatures -> Open a bug at the bugtracker.Or send him a pm I dont have an account.
You go ahead and sign all your sources. Please sign github releases and the arduino.cc downloads
Please also create sha512sums and also use them in build.xml if possible (over sha256)

cmaglie · 2016-12-28T17:42:31Z

The real solution here would be that launch4j signs their sources and we verify them. Then you can create a hash of that download and include it in your build script. But DO NOT blindly download sources, generate a hash and use this.

I still don't understand how can this improve things.
How a user is supposed to verify the signature? Who guarantees that the key I get from a keyserver is the real key used by developers and not a fake one?

Chris--A · 2016-12-28T17:46:35Z

It is really important, I cannot say it often enough. Please sign your sources. You do not need to use it yourself in the build script if you dont care, but please give others the chance to do so.

Are you asking me to sign my commits? (probably not).
Either way, I'm setting up gnupg anyway, might as well sign 'em. I'll amend this commit later.

In the meantime what do you suggest?

We can add what I have changed in this PR.
See if we can disable the redirect protocol restrictions (no safer than using plain http all the time).
Add the download directly into the repo until the situation is sorted out.
We all act out a DDOS on sourceforge, using cyber terrorism to get our way 😄

PaulStoffregen · 2016-12-28T17:53:30Z

Is this a solution in search of a problem?

In an alternate universe where everyone has an abundance of man-hours to implement everything, regarless of its actual value, great.

But here in reality, where projects are constrained by a fixed number of developer hours, is this really worthwhile? Should the human time needed to implement even more than the already-good hashes, and continue doing this extra work on every release really be prioritized over hundreds of other goals, which create tangible benefits for users?

Has there ever been any confirmed case of source tampering in Arduino's 10+ year history?

cmaglie · 2016-12-29T11:48:45Z

While we wait for sourceforge to take action, I've mirrored the files to downloads.arduino.cc that have no problems with https. The files are already checksummed.

Chris--A · 2016-12-29T11:51:42Z

Good compromise.

ChristophBWeber · 2016-12-30T00:43:17Z

Hello all, I am Director of Technology at SourceForge. Great discussion here regarding all the facets that make a download safe and trustworthy. At SourceForge we are in the final stage of moving all our mirrors to HTTPS. This is a bit like herding cats, not unlike managing an open source project. Nevertheless, we will be 100% HTTPS on our mirror network by January 15, 2017. This is part of our ongoing push to move our entire platform to the HTTPS protocol. My apologies if you got caught in the middle of the transition unprepared.

Hope this clears things up, and please let me know if there's anything else we can do to help your project.
Christoph Weber, [email protected]

P.S.: No use to DDoS us, won't get very far. ;-)

NicoHood · 2016-12-30T14:28:35Z

I still don't understand how can this improve things. How a user is supposed to verify the signature?

@cmaglie A user can verify the signatures via GPG. But most users are to lazy to verify those. But we use them for packaging our software (not only arch, also debian (+ubuntu), rpm (suse, redhat) and others). It is the only way for us to provide secure packages of the software that we distribute because the author can be authenticated.

Who guarantees that the key I get from a keyserver is the real key used by developers and not a fake one?

@cmaglie You need to write the full fingerprint on your website and also on github. You have two sources with https where the fingerprint is shown which should give you a good portion of trust. Also GPG is a public trust database. Users can sign other keys, so they get more trust. For example my key is signed with the ArchLinux master keys. So you can ensure that its me, nicohood. Otherwise they would not have signed my public key/fingerprint.

Are you asking me to sign my commits? (probably not).
Either way, I'm setting up gnupg anyway, might as well sign 'em. I'll amend this commit later.

@Chris--A The more people sign their commits, the more authenticated the git history becomes. The more people join, the better it is. And its really simple to activate automated commit signing with git.

In an alternate universe where everyone has an abundance of man-hours to implement everything, regarless of its actual value, great.

@PaulStoffregen You totally missed the point. GPG signing does not even take a minute for every release that happens every month or so. But the value we can gain is authenticated sources. Its the reason why you can trust your system. Security is not something that we do for fun. And yes i think especially the last 3 years showed that is is extremely important to protect the users with all the surveillance and hacks out there. It would not be the first service that got hacked. And if we have GPG signatures this would prevent this risk.

@ChristophBWeber Thanks a lot for this statement, this really clarified things and helps the community. Thanks!

ChristophBWeber · 2017-01-13T22:42:06Z

Just to tie this up: All active SourceForge mirrors are running over HTTPS as of 11:30am PST today. No more HTTP downloads! Hope this helps you folks.

cmaglie closed this in 4f29650 Dec 29, 2016

cmaglie added this to the Release 1.8.1 milestone Dec 29, 2016

NicoHood mentioned this pull request Jan 29, 2017

Various improvements in build script #5778

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed ant build script to use http with launcher4j. #5772

Changed ant build script to use http with launcher4j. #5772

Chris--A commented Dec 28, 2016

Chris--A commented Dec 28, 2016

NicoHood commented Dec 28, 2016

cmaglie commented Dec 28, 2016

Chris--A commented Dec 28, 2016 •

edited

Loading

NicoHood commented Dec 28, 2016

cmaglie commented Dec 28, 2016

NicoHood commented Dec 28, 2016 •

edited

Loading

cmaglie commented Dec 28, 2016

Chris--A commented Dec 28, 2016 •

edited

Loading

PaulStoffregen commented Dec 28, 2016 •

edited

Loading

cmaglie commented Dec 29, 2016

Chris--A commented Dec 29, 2016

ChristophBWeber commented Dec 30, 2016 •

edited

Loading

NicoHood commented Dec 30, 2016

ChristophBWeber commented Jan 13, 2017 •

edited

Loading

Changed ant build script to use http with launcher4j. #5772

Changed ant build script to use http with launcher4j. #5772

Conversation

Chris--A commented Dec 28, 2016

Chris--A commented Dec 28, 2016

NicoHood commented Dec 28, 2016

cmaglie commented Dec 28, 2016

Chris--A commented Dec 28, 2016 • edited Loading

NicoHood commented Dec 28, 2016

cmaglie commented Dec 28, 2016

NicoHood commented Dec 28, 2016 • edited Loading

And once more as an explanation:

cmaglie commented Dec 28, 2016

Chris--A commented Dec 28, 2016 • edited Loading

PaulStoffregen commented Dec 28, 2016 • edited Loading

cmaglie commented Dec 29, 2016

Chris--A commented Dec 29, 2016

ChristophBWeber commented Dec 30, 2016 • edited Loading

NicoHood commented Dec 30, 2016

ChristophBWeber commented Jan 13, 2017 • edited Loading

Chris--A commented Dec 28, 2016 •

edited

Loading

NicoHood commented Dec 28, 2016 •

edited

Loading

Chris--A commented Dec 28, 2016 •

edited

Loading

PaulStoffregen commented Dec 28, 2016 •

edited

Loading

ChristophBWeber commented Dec 30, 2016 •

edited

Loading

ChristophBWeber commented Jan 13, 2017 •

edited

Loading