Boards Manager - unable to install 3rd party boards

[rogerclarkmelbourne]

The restructure of the hardware and tools files into %appdata% (on PC) causes third party boards which use the AVR or ARM compilor to no longer function - because the compilor path has changed

I suggest that a method is needed to add 3rd party board packages, so that the existing compilors can be used for boards which use AVR and ARM

This change file structure change doesnt seem to have taken into consideration anyone using all sorts of other AVR boards like the Mega1280p and ATTiny devices

I suspect than none of the existing third party board packages now work

[sej7278]

I also don't see how Linux packages (e.g. rpm, deb) are going to work with the compilers and cores being stored inside the user's home directory instead of a system directory such as /usr/share/arduino

For a multi-user system you're going to end up with multiple copies of the same files and no way to centrally manage them.

[rogerclarkmelbourne]

Sorry. I didnt mean use Linux packages.

The IDE now has defined their own special package package_index file which is JSON

I was trying to describe the process where by on Linux, its possible to add external Repo's that the package manager can then search in

Not that Linux packages should be added

I mean a feature where the boards manager has an "Add Source" button, and allows you to type in a URL where a different package_index.json file can be downloaded from, to support whatever third party hardware the user chooses to install

This could be for boards like the Might1280P, ATTiny, STM32, TI Launchpad etc etc

After all the Arduino team introduced the Third party hardware stuff for 1.5 but now seem to have broken it, unless you supply you own compilor - but if the board is AVR this is a total waste of the user's space,and the same applies for ARM if they have installed the Due

[sej7278]

Right, but what I'm saying is how are Fedora, Debian, Ubuntu etc. going to manage this when packages must install applications like compilers and cores (boards) into one system directory and not multiple home directories.

The default cores should be stored in /usr/share/arduino or somesuch, then users can supplement those with local hardware directories kind of how 1.0 does it.

All compilers should be in system directories like /usr/bin not home directories.

[ffissore]

Hi @rogerclarkmelbourne, indeed we did a mistake in not providing a feedback for those cores that use the "old" avr-gcc path. We just coded a fix (available with the nightly builds) that will sooooon be released

[PaulStoffregen]

Doh! I just added the AVR toolchain to my installer for Teesny. Enlarges the download by nearly 20 Mbytes.

[rogerclarkmelbourne]

Simon,

I've not looked at where the new location is for the compiler on Linux or OSX.

One person has already contacted me about STM32 on Linux, but he is an experienced Linux user and has fixed things himself

@ffissore
Thanks

Unfortunately removing ARM from the default install is also an issue for me, as I'm using ARM boards other than the Due (Well, I have 2 Due's but I'm not actively using them. )

I'm looking at various other 3rd party boards, mainly using STM32,but also other ARM based architectures, all of which are impacted :-(

[ffissore]

Sorry @PaulStoffregen , this stuff is keeping me from properly communicate on the devs list. I promise I'll sum up everything when the big ones will be fixed

[tolson2000]

Or just leave the gcc stuff in the Arduino IDE like before. I put Arduino in /opt and have several versions there. Now the 1.6.2 puts the gcc stuff in the users .arduino15 directory of the user who installed it. Ugh!

Just keep all the Arduino support stuff in the Arduino tree. Leave the user chosen specific libraries and preferences in the users home directory.

[rogerclarkmelbourne]

For official Arduino baords, I can't see why the compilers are not installed for both AVR and ARM.
(Except of course for size), but I'm not sure anyone has complained so far from 1.5.0 to 1.6.1 about the size

Other third party stuff that uses MIPS compilers is not such an issue, as they always have to provide their own compiler as part of their "hardware" files

Still... As the Boards Manager is supposed to manage boards, its strange we can't add additional external source URL's

[PaulStoffregen]

@ffissore - Yeah, I'm sure you've got your hands full with urgent stuff. It impacts me much less, since I already have my installer in place to manage toolchains and other details. In fact, I even had time to read the new source and prepare #2850 on the new codebase! :) I know you probably won't have time to merge it if you're rushing to get a 1.6.3 fix pushed out. But please take a look before lots of other stuff changes and it won't automatically merge (like the prior 2 pull requests I've sent in the last month).

[ffissore]

@rogerclarkmelbourne indeed size is the main issue

[ffissore]

Latest fix is available as a nightly build.
Cores using paths like {runtime.ide.path}/hardware/tools/avr/bin/ or {runtime.ide.path}/hardware/tools/gcc-arm-none-eabi-4.8.3-2014q1/bin/ are automatically converted (that happens in-memory, no file is changed). Here is the list of property values that get rewritten.
Hardware folder content has been restored and it contains the AVR bundle.
I've used sparkfun core and a core nicohood submitted a couple of weeks ago and they both work fine

[sej7278]

do we really need the version numbers there, its going to make it hard to parse, and it seems gcc-arm-none-eabi-4.8.3-2014q1 is not symlinked to gcc-arm-none-eabi-

[ffissore]

You mean platform.keys.rewrite.txt? If so, that's the content of the latest platform.txt for the sam platform before the switch to boards manager and its issues. [1] Of course we can add as many other keys as we want.
[1]

Arduino/hardware/arduino/sam/platform.txt

Line 13 in c46e667

compiler.path={runtime.ide.path}/hardware/tools/gcc-arm-none-eabi-4.8.3-2014q1/bin/

[ffissore]

keys = key/value pairs to rewrite

[tolson2000]

Just download nightly build.
Looks like gcc-arm is still being dumped into ~/.arduino15 instead of the IDE hardware/tools folder!

[PaulStoffregen]

As someone who installs his own toolchain, this recipe rewriting worries me. I really do not want the IDE to second guess my instructions!

Is there a recommended way to disabled it for my platform.txt file? Or perhaps some option could be added in boards.txt or platform.txt to disable any rewriting?

[ffissore]

@tolson2000 yes, that's on purpose, hardware folder is now back to what it was, so existing docs/tuts are still valid. However, additional boards are still installed into user's home folder
@PaulStoffregen not atm, but your request makes sense: what about an optional key/value pair in your platform.txt like rewriting=disabled or something like that?

[ffissore]

I'll code rewriting=disabled for now and change it later if you come up with a better name

[sej7278]

will the SAM boards be bundled in 1.6.3 also? the currently nightly seems to work again like 1.6.1 did on linux for AVR, but board manager still seems to want to download SAM to to ~/.arduino15 and still uses the ide version in the path which complicates matters for people who want to use the cli instead of the ide.

so now we have a mixture of /usr/share/arduino/hardware/arduino/avr/ and ~/.arduino15/packages/arduino/hardware/sam/1.6.3/ and a 1.6.4, 1.6.5....

instead of what previously was just /usr/share/arduino/hardware/arduino/sam (or avr)/

just seems like you're making things more difficult for no benefit.

why not just put sam back with avr and remove the versioning and home directory shenanigans? leave the home directory for sketches and 3rd party libraries/cores like 1.0-1.6.1 and put the toolchains back in system directories with the ide.

[tolson2000]

I agree with @sej7278. All this complexity is confounding. Keep it simple like it was. Jumping on the json this and json that band wagon really is not needed.

[rogerclarkmelbourne]

I think there is also the undesirable side effect, the time taken to unpack the packages when the user first runs the IDE.- this was very noticeable on my core i5 Windows 7 machine, and I did wonder what on earth it was doing.

But I've since realized it must be messing around with the packages and actually continuing the install during the first run.

Unfortunately its now broken (and I will post another issue), but... its not clear to the user that "Installing" a new board could potentially mean they are going to then download hundreds of MB files.

Some people may be on restricted bandwidth/ or data quota (even in first world countries - me included)
So at the very minimum it needs to be made clear what size of download is going to occur.

[rogerclarkmelbourne]

Edit
I created a new issue

#2871

I'm not sure if the latest version of 1.6.2 has this issue or not. I don't have the time or data quota to download today's version of 1.6.2 to see if the dev team has changed the 1.6.2 binaries without changing the version number (this would not be good practice).

But in the 1.6.2 I downloaded on Sunday 29th March, I can no longer install the Due

[ffissore]

SAM boards won't be part of the bundle, to save everyone's bandwidth. Downloading the avr bundle and installing sam boards sum up to a similar bandwidth consumption as before. After having installed SAM boards, you can upgrade the IDE without having to reinstall SAM again: they will remain in your home folder. That means an overall reduced bandwidth consumption.
A command line interface for installing additional boards is in the making.
Folders used as version numbers will be removed in next version or the one after.
First start slowness has been fixed in current nightly by restoring content of hardware folder.

[sej7278]

ok well removal of version numbers in directories is a good thing.

shame about the sam toolchain being in the home directory, that means no linux distro is going to support it as it won't be packageable.

on a multi-user system you're going to waste more bandwidth with every user downloading the toolchain to their home directories rather than a single installation in /usr/share/arduino

i'd have thought the way to do it would be to offer a complete download zip file from the website or a partial download with the ability to install to home directory if you really want to.

[ffissore]

One initial idea was to provide a "nacked" idea together with the avr bundle. We chose not to as that may confuse novices

[ffissore]

1.6.3 is out. Thank you for your feedback. I hope you'll find it more reliable than 1.6.2