Skip to content

package_index.json file signature verification failure for some users #8988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
per1234 opened this issue Jun 17, 2019 · 112 comments
Open

package_index.json file signature verification failure for some users #8988

per1234 opened this issue Jun 17, 2019 · 112 comments
Assignees
Labels
Component: Board/Lib Manager Boards Manager or Library Manager Type: Bug

Comments

@per1234
Copy link
Collaborator

per1234 commented Jun 17, 2019

We have had four reports of this error on the forum in the last two days. The usual trick of clearing out the data folder and trying again didn't work for any of them. One of the users, Cheetor, provided the package_index.json and package_index.json.sig files they get from https://downloads.arduino.cc/packages/package_index.json and https://downloads.arduino.cc/packages/package_index.json.sig:

I compared these to the files I download from the same URLs and found that their package_index.json file was missing the entries for Arduino SAMD Boards 1.8.1 and avrdude 6.3.0-arduino17, but no differences other than that. The checksum of their .sig file matches mine.

Cheetor is in New Zealand and one of the other reporting users (DavidBMason) is as well. The other two haven't provided their location. The problem stopped occurring for DavidBMason before I could get the bad package_index.json and package_index.json.sig files from them:
https://forum.arduino.cc/index.php?topic=621637.msg4212584#msg4212584

My hypothesis is that there was a recent update to package_index.json but the new .json file didn't make it to a server that provides the files to people in NZ. However, the new .sig file did make it to that server. So they are getting the old .json file but the new .sig file, thus the signature verification. Further evidence of this is that when Cheetor used TOR with an exit node in the USA they got the new version of package_index.json:
https://forum.arduino.cc/index.php?topic=621811.msg4212512#msg4212512

It would be nice if there was some way to make sure that the .json and .sig files will always hit the servers at the same time. I suspect this delay of days on the .json file is a rare glitch but if we regularly have a delay of even minutes that still is going to cause problems for people, more so because of #8936.

Forum threads:

@per1234 per1234 added Type: Bug Component: Board/Lib Manager Boards Manager or Library Manager labels Jun 17, 2019
@facchinm
Copy link
Member

@rsora or @endorama may you take a look?

@endorama
Copy link
Contributor

@rsora will take care on this. I'm subscribed to the issue, so just ping if you need me!

@rsora
Copy link

rsora commented Jun 17, 2019

I'm reviewing the issue and opening an internal Incident report, I'll keep you posted!

@ilmarhrundel
Copy link

I have the same problem on two PCs and four different internet connections..

@hortynz
Copy link

hortynz commented Jun 17, 2019

I am in NZ too, and I also had the same problem on two different PCs across several web connections. I didn't try tunnelling yet - would have to set something up. I'm still experiencing the issue this morning, and have attached the two files I get.
package_index.json.txt
package_index.json.sig.txt

@hortynz
Copy link

hortynz commented Jun 17, 2019

not sure if searches will scan the text of our issue, but just in case, I'm pasting the error message displayed on my sketch UI. "package_index.json file signature verification failed. File ignored"

@per1234
Copy link
Collaborator Author

per1234 commented Jun 17, 2019

Thanks for sharing those files @hortynz! They have the same issue as the files shared on the forum by Cheetor. package_index.json is missing the entries for Arduino SAMD Boards 1.8.1 and avrdude 6.3.0-arduino17 that are present in my download from https://downloads.arduino.cc/packages/package_index.json but the checksum of the .sig file matches the checksum of the .sig file I get from https://downloads.arduino.cc/packages/package_index.json.sig.

@rsora
Copy link

rsora commented Jun 18, 2019

Hi there,
Can someone provide the http headers that are received when calling both
https://downloads.arduino.cc/packages/package_index.json https://downloads.arduino.cc/packages/package_index.json.sig
in case you are still experiencing the signature error?
Thanks!

@anzas
Copy link

anzas commented Jun 18, 2019

Here are the http headers
package_index_json_header.txt
package_index_json_sig_header.txt

@rsora
Copy link

rsora commented Jun 18, 2019

Hi there,
I' have just launched a CDN refresh can you please verify if you are still experiencing the
"package_index.json file signature verification failed. File ignored"
error?
If yes please reply with geographical location and both headers and files.
Thanks!
cc @hortynz @per1234 @anzas

@anzas
Copy link

anzas commented Jun 18, 2019

That fixed it, at least for me. Location is Finland.
header_json.txt
header_json_sig.txt
package_index.json.sig.txt
package_index.json.txt

@ilmarhrundel
Copy link

Everything is working now! (Estonia)
Thank You very much.

@hortynz
Copy link

hortynz commented Jun 18, 2019

here too, thanks

@cheetor5923
Copy link

Can confirm, CDN refresh has worked. I'm now getting the correct file

@bhavanakrishna
Copy link

http://arduino.esp8266.com/stable/package_esp8266com_index.json file signature verification failed. File ignored.
help me out with this error
i tried different versions of arduino ide and different system but same error

@rsora
Copy link

rsora commented Jun 20, 2019

Hi @bhavanakrishna,
the package_esp8266com_index.json is a third party index that is not served by our services, I suggest you to ask in the https://www.esp8266.com/ forum for hints on how to solve your problem!

Regarding this issue, having received a positive feedback both in the forum and in the previous message, I'll proceed to close and solve this issue.

Thanks to @hortynz @per1234 @anzas @cheetor5923 @ilmarhrundel for providing feedback!

@rsora rsora closed this as completed Jun 20, 2019
@BZ840
Copy link

BZ840 commented Jun 20, 2019

Hi @rsora,
I encountered the same issue on multiple devices since yesterday: https://downloads.arduino.cc/packages/package_index.json file signature verification failed. File ignored.
Here are the files:
package_index.json.txt

Thank you for your help in advance!

@endorama
Copy link
Contributor

endorama commented Jun 21, 2019

Hi @BZ840, please provide the HTTP headers you get when requesting the files, so we can look into it!

@BZ840
Copy link

BZ840 commented Jun 21, 2019

Hi @endorama,
Here is the file
http_header.txt

Thanks!

@endorama
Copy link
Contributor

Hi @BZ840 sorry for not being clearer, but we need both .json and .json.sig requests headers to debug this.

As reference take this previous comment.

May you also share which nation are you connecting from?

Thanks

@BZ840
Copy link

BZ840 commented Jun 21, 2019

Hi @endorama
header_json.txt
header_json.sig.txt
The location is Canada.

@ghost
Copy link

ghost commented Jun 22, 2019

I have the same problem and I'm from Canada.

https://downloads.arduino.cc/packages/package_index.json
https://downloads.arduino.cc/packages/package_index.json.sig

It's causing all MKR boards to not be listed and I can't search them in my boards manager. Every once in a while they will pop up again and be listed, but most of the time they are not. It's random and annoying. All of my university projects rely on the MKR1000.

@red-scorp
Copy link

Same problem
OS: Windows
Location: Germany
OS Language: US

package_index.json.sig.txt
package_index.json.txt

How can I grab the http header?
How can I launched a CDN refresh?
Thanks in advance!

@per1234
Copy link
Collaborator Author

per1234 commented Jun 24, 2019

How can I grab the http header?

  1. Open a new browser tab or window.
  2. Press F12 to open the toolbox.
  3. Click the "Network" tab of the toolbox.
  4. On the next bar down in the toolbox, click "All".
  5. Paste the URL (https://downloads.arduino.cc/packages/package_index.json or https://downloads.arduino.cc/packages/package_index.json.sig) into the URL bar of your browser.
  6. Press Enter.
  7. In the toolbox, click on the line that says "package_index.json" or "package_index.json.sig" (depending on which URL you're currently getting the headers for).
  8. In the pane that appears, click the "Headers" tab.
  9. If using Firefox, switch the "Raw headers" switch to the on position for the "Response headers" section.
  10. Click and drag to select all text in the "Response headers" section.
  11. Press Ctrl + C to copy the selected text to the clipboard.
  12. You can now either paste the copied header text directly into a reply here or save it in a .txt file and attach the .txt file in a reply here.
  13. Repeat the process for the other URL.

@BZ840
Copy link

BZ840 commented Jun 24, 2019

Hi @endorama
Those two header files I provided 3 days ago, were used some http header online service to retrieve by pasting the URL.
Here is what I got if I follow what @per1234 suggested:
header.json.txt
header.json.sig.txt
The location is Canada.

I apologize for any confusion and inconvenience.

@rsora
Copy link

rsora commented Jul 3, 2019

Hi there,
As I mentioned in my previous comment we worked on both Server and Java IDE side:

Server Side:
We forced the injection of a Cache-Control: private header on our package_index.json and related .sig and .gz file. This should prevent unwanted caching between our CDN and your PCs.

Java IDE Side:
We prepared a PR (#9023) for the next release that should mitigate the issue enabling caching for the indexes files, without removing them if something in the signature verification process fails. In addition we implemented a simple logging mechanism that should avoid you to search and download the request headers. Sharing the generated logging files will be sufficient.

So,

you can download the following preview binaries: #9023 (comment), install them, and tell us if it is working for you and solves the signature problem. In case you experience errors using this IDE preview, please do the following steps in order to share with us the IDE logs:

  1. (In the Arduino IDE) File > Preferences
  2. Click the link at the line following "More preferences can be edited directly in the file". This will open the Arduino15 (or similar name depending on OS) folder.
  3. create a zip for the logs folder and upload it as a github comment attachment

this way will be able to see better what gone wrong on your system.
Thanks a lot to you all for helping us in solving this (nasty 😄 ) issue!

edit : updated binary download link after some bug fixing made in the PR

@sl1pkn07
Copy link

sl1pkn07 commented Jul 3, 2019

fixed for me (spain)

@red-scorp
Copy link

In my case (Germany) it is very strange. Sometimes it works and I can update/install boards, and few minutes later it does not work any more.

@rsora
Copy link

rsora commented Jul 4, 2019

Hi @red-scorp, could you please try using this build for the Java IDE: #9023 (comment) and see if it works for you?

As I explained here #8988 (comment) we should be able to see what is happening your side inspecting the IDE logs.

Thanks a lot!

edit: Updated link to binaries

@pdo-smith
Copy link

After another day of testing it still works for me. Many thanks you guys!

@red-scorp
Copy link

red-scorp commented Jul 4, 2019

@rsora I've downloaded http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-9023-BUILD-874-windows.zip and put it in portable mode. When I start arduino.exe it shows me a splash screen and does not start an editor. Either this version is bad or my local antivirus is evil (which can be).
grafik

I'll test on another pc later and let you know..

@red-scorp
Copy link

report from another PC: Avast was not happy to run your nonsigned code. Even with disabled antivirus this version is still not able to start editor window.

@rsora
Copy link

rsora commented Jul 5, 2019

Hi there,
@red-scorp good catch! we fixed the issue you foudt and prepared this new build, I hope that you have the chance to give it a try and let us now how it goes!

To download the updated build use this link: #9023 (comment)

Thanks a lot!

@chrisly42
Copy link

After hours of unsuccessful attempts, changing the run.options to

run.options=-Djava.net.preferIPv4Stack

in preferences.txt fixed the issue for me.

@red-scorp
Copy link

red-scorp commented Jul 7, 2019

@chrisly42 same for me, but only for a single run. Now the same error appears again.

@red-scorp
Copy link

@rsora Now I've tried several times with 1.8.9. I works fine. I can update board packages and stuff.

@red-scorp
Copy link

@rsora still working... Was it a server issue?

@red-scorp
Copy link

@rsora still working... Tried with 1.8.7 and portable 1.8.9. Both are working good.

@sonisonjames
Copy link

Getting the problem in Singapore, error is :
package_index.json file signature verification failed
java.lang.RuntimeException: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:150)
at java.lang.Thread.run(Thread.java:748)
Caused by: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed
at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:91)
at processing.app.BaseNoGui.initPackages(BaseNoGui.java:484)
at processing.app.Base$9.onIndexesUpdated(Base.java:1381)
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:148)
... 1 more

@Sxyther
Copy link

Sxyther commented Jul 31, 2019

Got the same issue TODAY!

But I got latest Windows 10 update yesterday..

java version "1.8.0_221"
Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

and Arduino 1.8.9

Got Java errors when I tried to download code to a SAMD21 MCU over CDC /USB as well and was working fine yesterday...
hope this help

package_index.json file signature verification failed
java.lang.RuntimeException: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:150)
at java.lang.Thread.run(Thread.java:748)
Caused by: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed
at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:91)
at processing.app.BaseNoGui.initPackages(BaseNoGui.java:484)
at processing.app.Base$9.onIndexesUpdated(Base.java:1381)
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:148)
... 1 more

@Xerusial
Copy link

Bump for Arduino IDE 1.8.10 on Ubuntu 18.04 and Location Germany.
Got the same issue today.

@kevteg
Copy link

kevteg commented Mar 28, 2020

I was having this issue with Arduino IDE V 1.812. I was able to solve it changing the Additional Boards Manager URLs to https://downloads.arduino.cc/packages/package_index.json

@stpatrick2016
Copy link

stpatrick2016 commented Apr 29, 2020

Got the same error today (freshly installed Arduino IDE 1.8.12). Here are headers, location is Israel
json.headers.txt
json.sig.headers.txt

Thanks :)

UPD (2020-05-02): working properly now, thank you :)

@erwinbonsma
Copy link

Today I also got this error on a fresh Arduino 1.8.12 install. Here are the headers:
package_index.json.header.txt
package_index.json.sig.header.txt

My location is the Netherlands.

@henriquelino
Copy link

henriquelino commented Jun 9, 2020

Update 12hrs later, now IDE works and problem is solved, in our facebook group many realated same issue, not sure if all of then are solved now.
New and working jsons and headers:
package_index - Copy.json.txt
package_index.json - Copy.sig.txt
working header - sig.txt
working header.txt


Old and was not working, crashing
package_index.json.sig.txt
package_index.json.txt
application.log.txt

I'm in Brazil, this issue started today, deleting files in Arduino15 get me to open IDE again, but as soon as those files are created, my IDE dont even open anymore.

Already reinstalled arduino IDE and java.

Header of https://downloads.arduino.cc/packages/package_index.json
HTTP/2 304 Not Modified date: Tue, 09 Jun 2020 19:53:08 GMT cf-ray: 5a0d65a46a9ef677-GRU age: 659 cache-control: max-age=3600 etag: "bdfb8b1354c75cd2be921c41cd39763c" last-modified: Tue, 09 Jun 2020 15:41:23 GMT vary: Accept-Encoding via: 1.1 2e9033da1cf7b64ac622ab535b39a267.cloudfront.net (CloudFront) cf-cache-status: HIT cf-request-id: 033c3ddac10000f677d70b7200000001 expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" x-amz-cf-id: BRNl8Z41ZGNem4Ze19gghkfl-4TlGjBMdK2esfb2DKyzwm7UHZ2sjA== x-amz-cf-pop: MIA3-C3 x-amz-id-2: mLwSJyc1q3puS52zNu7YBQYgUmXmVuBWWeEtKd4MLnZtOKK2YDJ73rtK/J1ebfh8ooI2vhy2zrY= x-amz-replication-status: PENDING x-amz-request-id: 4630B42FB314822F x-amz-version-id: dCcX8rEv_jnZcFRzEksIFUXBQK3JRI4q x-cache: RefreshHit from cloudfront server: cloudflare alt-svc: h3-27=":443"; ma=86400 X-Firefox-Spdy: h2

And https://downloads.arduino.cc/packages/package_index.json.sig

HTTP/2 200 OK date: Tue, 09 Jun 2020 19:55:13 GMT content-type: application/pgp-signature content-length: 543 cf-ray: 5a0d68b09934f677-GRU accept-ranges: bytes age: 783 cache-control: max-age=3600 etag: "117c06a2897e79ad384744c67b2c89f3" last-modified: Tue, 09 Jun 2020 15:41:23 GMT vary: Accept-Encoding via: 1.1 52e2243a8168629f98bb0607016f7225.cloudfront.net (CloudFront) cf-cache-status: HIT cf-request-id: 033c3fc2620000f677d720f200000001 expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" x-amz-cf-id: rIY35Z0doDB7jG6KGeZzlavTAwNL2feJD7dG4144Gj_Pm29drqkzkg== x-amz-cf-pop: ATL51-C1 x-amz-id-2: aqOzcZTCLWsBXbkQk7lX90W/Hqs0aFKblTUshVgtox+UV9Qx1RF2qAmUfbqjC/yk4EJqSK/pK6I= x-amz-replication-status: COMPLETED x-amz-request-id: F990AA05464FF915 x-amz-version-id: gvUGiZJxSQ2dzeBeLwv_Nt_TVJceWtsN x-cache: RefreshHit from cloudfront server: cloudflare alt-svc: h3-27=":443"; ma=86400 X-Firefox-Spdy: h2

@absalom-muc
Copy link

@kuba989898
Copy link

kuba989898 commented Jun 9, 2020

Also in Poland, run Board Manager killing Arduino installation, not starting again, version trough 1.8.8, 1.8.9, 1.8.12, in Windows and in Linux.
In Ubuntu:

jakub@jakub-F7F:/arduino-1.8.8$ ./arduino
Picked up JAVA_TOOL_OPTIONS:
java.lang.NullPointerException
at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:124)
at processing.app.BaseNoGui.initPackages(BaseNoGui.java:484)
at processing.app.Base.(Base.java:268)
at processing.app.Base.main(Base.java:151)
jakub@jakub-F7F:
/arduino-1.8.8$

@jfduhart
Copy link

Same issue here in Chile (Arduino IDE 1.8.12 for Mac)
Deleting package_index.json and library_index.json from Arduino15 folder is the only way to get the IDE to start without crashing, files are created every time and IDE works fine until it is closed.

package_index.json.txt
library_index.json.txt

@facchinm
Copy link
Member

We had a problem on our package_index.json that caused the error and prevented the IDE to start.
Now the package_index has been fixed, but to recover a working installation you must delete once again the package_index.json with the cache folder and restart the IDE as usual.

Windows:

  • remove file C:\Users\YourUsername\AppData\Local\Arduino15\package_index.json
  • remove folder C:\Users\YourUsername\AppData\Local\Arduino15\cache

Mac:

  • remove file /Users/YourUsername/Library/Arduino15/package_index.json
  • remove folder /Users/YourUsername/Library/Arduino15/cache

Linux:

  • remove file /home/YourUsername/.arduino15/package_index.json
  • remove folder /home/YourUsername/.arduino15/cache

Please note that the problem that prevents the IDE to start has already been fixed and it's ready for the next release, that's the reason why the Nightly/Beta Builds are not affected, and also the reason why we did not detect this problem earlier, sorry about that!

@giulianovaraschin
Copy link

I have the same problem:

age: 2650
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cache-control: public, max-age=3600
cf-cache-status: HIT
cf-ray: 6e11d25969564d4e-GRU
content-encoding: gzip
content-type: application/json
date: Mon, 21 Feb 2022 17:51:15 GMT
etag: W/"6a5d5920c8c262abac717297873088a2"
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
expires: Mon, 21 Feb 2022 18:51:15 GMT
last-modified: Mon, 21 Feb 2022 09:00:48 GMT
server: cloudflare
vary: Accept-Encoding
via: 1.1 978d7ccfdbed8c0e3015142e29dd5c5c.cloudfront.net (CloudFront)
x-amz-cf-id: EQwmbudsNAceY0VEt1T1LcSUZ6-wX2beCnFVuVRjsY4w-OE0PcuKWg==
x-amz-cf-pop: EWR53-P1
x-amz-id-2: S3szxnxMyZUi0v4pAdFCt5xd7FHRmj/6H5/hh5nZ3YI8Ap9GhFLTpwGIxwuZaaGtmBHuh0c2mpQ=
x-amz-replication-status: COMPLETED
x-amz-request-id: KHVD6F97T1A1JDBG
x-amz-version-id: N6O__NNMoIEb.89NO8mU8Zwv3QYpjPgm
x-cache: Miss from cloudfront

@rsora
Copy link

rsora commented Feb 22, 2022

@giulianovaraschin can you tell us

  • the IDE version you are using
  • your OS and version

It would be nice if you could follow this guide and report here your findings to have a clear understanding of what's going on your side

Thanks!

@rsora rsora reopened this Feb 22, 2022
@rsora
Copy link

rsora commented Feb 22, 2022

FYI we forced a cleanup on the CDN Arduino side, this should help to resolve your issue @giulianovaraschin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Board/Lib Manager Boards Manager or Library Manager Type: Bug
Projects
None yet
Development

No branches or pull requests