Teensy JSON fails Board Platform install due to unsupported tar.zst archives

[mattbaker-digital]

Sloeber 4.4.2 Product Bundle
OS: Windows 10
Platform JSON: https://www.pjrc.com/teensy/package_teensy_index.json

Project properties arduino showing the board settings:

Sloeber settings:

Describe the bug
The archive files in the Teensy package JSON are not supported (tar.zst file)

To Reproduce
Steps to reproduce the behavior:

1. Go to Window -> Preferences
2. Click on Sloeber -> Third party index url's section
3. Add https://www.pjrc.com/teensy/package_teensy_index.json
4. Click Apply and Close to force the board list update.
5. Go to Window -> Preferences
6. Click on Sloeber -> Platforms and Boards section
7. Tick teensy -> Teensy -> 1.58.1
8. Click Apply

Expected behavior
Board to be installed correctly

Data to back up your claim
If applicable, add screenshots, console output and/or logs to help explain your problem.

Additional context
As a workaround I tried the outdated (now installed to %LOCALAPPDATA% not Program files) instructions here:
http://eclipse.baeyens.it/how_to.shtml#/i

This failed as it tried to call "/arm/bin/arm-none-eabi-g++" instead of using the absolute path "C:\\Users\\matt.baker\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-compile\\11.3.1/arm/bin/arm-none-eabi-g++"

Error message:

sh: /precompile_helper: Permission denied
make: *** [subdir.mk:20: sloeber.ino.cpp.o] Error 126

[jantje]

I confirm this is an issue.
It looks as if @PaulStoffregen made a new version of https://www.pjrc.com/teensy/package_teensy_index.json using a archive format that Sloeber does not (yet) supports and in one go changed all old platform versions (which you should not do) making it impossible to add any Teensy platform version to Sloeber.
I will add the archive format to Sloeber but I'm currently caught up in other work 😒

[PaulStoffregen]

Yes, confirm, we switched to zstd compression. Arduino IDE has supported zstd since version 2.0.4.

It gives significantly smaller files to download and decompresses much faster, especially on Windows.

[PaulStoffregen]

Teensy will also soon start using Arduino's new pre_uninstall script feature.

arduino/arduino-cli#2277

So far no stable Arduino IDE release has this feature, but it is in the nightly builds.

I hope for your long term planning, consider checking for this script and run it before uninstall. The script allows cleanup to be done. In the short term, Teensy will use this to cause any lingering Teensy Loader application to automatically shut down. If the installed files are to be deleted, this especially helps for Windows where the file can not be deleted if any instance of the program is still running.

[PaulStoffregen]

The pre_uninstall script official documentation can be found in newer copies of Arduino's platform specification.

https://arduino.github.io/arduino-cli/dev/platform-specification/#pre-uninstall-script

[jantje]

@PaulStoffregen
Thanks for the confirmation and early warning.

[jantje]

I'm in a bit of a pickle here.
Teensy install worked with commit #1612 in branch support_for_tar_zst_#1611
However the tests for the pull request failed due to "Sloeber needs a newer maven than github build uses" 😒
Due to this issue the github actions for Sloeber build has been down for a couple of months now.
The only way I can think of to get this build right now is to build the branch support_for_tar_zst_#1611 yourself

[PaulStoffregen]

Is using an older version of the plugin an option? On this release notes file, looks like version 3.0.5 might be the last before it started requiring maven 3.9.

[jantje]

After some more investigation in this issue:
from actions/runner-images#8034

As of version 4.0.0, the Eclipse Tycho plugin for Maven requires Maven 3.9.0 or later:

So basically github actions is very aware and the situations for Sloeber is:

1. Eclipse started a cleanup to remove packages from their dependency list and repositories (Which they have every right to do)
2. Sloeber installs no longer work due to missing packages.
3. I fixed this by referencing maven repositories in a .target file=>Sloeber maven build needs tycho 4.0.0 or later to build
4. Tycho 4.0.0 requires maven 3.9 or later
5. Maven 3.9 breaks many github actions (due to removal of dependencies to "old packages" -Where did I see this before😒 - )
6. Github actions have bad experiences with upgrading to maven 3.9 (due to point 5) so they do not feel like doing it again.
7. I find myself in the situation where I need to bet on waiting for github actions to upgrade maven or on investing time learning the proposed alternative solutions.

=>I'm in a bit of a pickle here.

[PaulStoffregen]

Maybe I'm reading too much into this situation, but kinda feels like Github doesn't care much for Java?

[jantje]

or eclipse?
or actions?
or ...
but .... that thought did cross my mind when reading about this issue.

[mattbaker-digital]

@jantje thanks for looking into this. This is my first time using the plugin in several years.

[mattbaker-digital]

Is there a way for me to temporarily fix this by pointing to the files installed by the Arduino IDE and somehow correcting the path to /arm/bin/arm-none-eabi-g++?

[jantje]

EDITED: As pointed out the branch name was wrong so I changed it

Is there a way for me to temporarily fix this by pointing to the files installed by the Arduino IDE and somehow correcting the path to /arm/bin/arm-none-eabi-g++?

There probably is but I think the easiest way it to compile sloeber yourself

You can find good instructions to build the Master branch and find the results here: https://github.com/Sloeber/arduino-eclipse-plugin/tree/master

You need change the build a bit to build the branch "support_for_tar_zst_#161"
So the build command becomes

git clone https://github.com/Sloeber/arduino-eclipse-plugin sloeber
cd sloeber
git branch support_for_tar_zst_#1611
git switch support_for_tar_zst_#1611
mvn clean verify -DskipTests=true

[jantje]

@mattbaker-digital

Did you succeed in building and installing/upgrading Sloeber ?
And if so did it fix the issue?

[mattbaker-digital]

@jantje I managed to build the Sloeber for Windows in the end. As you probably already know from my deleted posts, I used the instructions above verbatim which didn't checkout the fixed branch. Using git checkout support_for_tar_zst_#1611 worked (not sure where the 23 prefix to the branch name in the original comes from?).

It fixed the issue for me. Thank you for the quick patch.

Hope you manage to get he automated builds to work again without too much work - it sounds like a real pain 😕.

On an unrelated note I did encounter missing dependency errors when attempting to build the SDK version for development / improvements to Sloeber.

[jantje]

As you probably already know from my deleted posts

That would have been an assumption and only me knowing that.
I asked to confirm so everybody who sees this issue knows you got it to work.

it sounds like a real pain

It is. Thanks for noticing.

On an unrelated note I did encounter missing dependency errors when attempting to build the SDK version for development / improvements to Sloeber.

I confirm the issue. I should make SDK part of the normal build so it doesn't get left out.

[jantje]

I got the actions build back up and running.

[jantje]

FYI
I'm currently release testing this version https://github.com/Sloeber/arduino-eclipse-plugin/actions/runs/7764103167
So people who have this issue can download the product (and update repository) from there.