arm-none-eabi-gcc build for armhf

[ShorTie8]

Thankz @cmaglie for the test_package_arm_index.json file!
Sorry for all the frustration relief, But I truly am just a stupid Chicken Farmer and not some super coder like you guys are.

This is really not a issue or a bug, But was wondering if your script to build the arm-none-eabi-gcc toolchain is around any where ?? Can't seem to find it .. :/~

Have A Great Day
ShirTie

[cmaglie]

Sorry for all the frustration relief, But I truly am just a stupid Chicken Farmer and not some super coder like you guys are.

No problem, and don't worry, we all learn new stuff every day, for me there are no "super coder" nor "stupid Chicken" but only good and positive contributors, so let's forget about that and move on :-).

This is really not a issue or a bug, But was wondering if your script to build the arm-none-eabi-gcc toolchain is around any where ?? Can't seem to find it ..

We always used the unpatched pre-compiled binaries provided from https://launchpad.net/gcc-arm-embedded so we didn't build it directly like we do with al the other tools (that often have been patched to work properly).

On their website there is a .pdf document that describes "how to build", and recently ARM added a script that should work with the Ubuntu distro (they suggest to use 8.04 IIRC). By the way I must warn you that the last time I tried it didn't worked out of the box (I don't remember the exact details sorry, it was too much time ago).

Also please note that we need the exact version 4.8.3-2014q1, this is a very important detail.

In any case if you're going to try something on your side I suggest to send frequent updates of your progress here and arduino/Arduino#4457 to avoid duplicating efforts (@PaulStoffregen seems to be also looking at that).

[PaulStoffregen]

I am indeed working on this. Well, right now my little Raspberry Pi 1 Model B is working on it.... the build takes many hours.

You can find the modified scripts here:
https://github.com/PaulStoffregen/ARM_Toolchain_2014q1_Source

If you look at the commit log, this repository started with an exact copy of 2014q1. The important edits are in these 2 commits: PaulStoffregen/ARM_Toolchain_2014q1_Source@5902fd0 PaulStoffregen/ARM_Toolchain_2014q1_Source@3824906

[PaulStoffregen]

If you're curious about why these particular edits, the reason stems from slightly different (and incompatible) naming conventions used by Debian, ARM's build script, and GNU.

Because these scripts are meant to be able to build a Canadian cross compiler (eg, a Linux machine builds a compiler that runs on Windows, which compiles code for ARM EABI), they pass all 3 specs to every configure script, for --build, --host and --target.

The problem stems from use of "uname" and some assumptions in the scripts to create the --build and --host names. On Raspberry Pi, the script will use "armv6l-linux-gnu". However, none of the GNU configure scripts recognize that particular combination.

Fortunately, the configure scripts do know how to detect the build and host settings. Because we're not building a "Canadian" (this has to be actually built on a Raspberry Pi), the simple solution is to just not force the --build and --host settings. Then each configure script automatically detects "armv6l-unknown-linux-gnueabihf" and is able to build the code. ... or at least so far, so good... it's been running for about 11 hours and made it through the first stage gcc build... update: got through newlib... now in 2nd stage gcc build... looking good...

I know there's always some level of anxiety about using a hacked toolchain. My earlier efforts involved replacing components with newer versions that were updated to understand "armv6l-linux-gnu". If you google the errors, that's the advice you find, because later versions added "armv6l-linux-gnu" to their configure scripts. But that would have changed quite a bit of the toolchain code. This way, the build is all the same toolchain code from 2014q1. It should result in a build that's a perfect match to the 2014q1 toolchain Arduino is currently using on the 4 other platforms.

My intention is to do some testing to verify the compiler actually works, and then re-run the entire build a 2nd time, just to be absolutely sure it really compiles cleanly from the files on https://github.com/PaulStoffregen/ARM_Toolchain_2014q1_Source Even though this may look easy from only the commits I've pushed to Github, I can assure you I did quite a lot of fiddling yesterday. I want to make sure this really is correct before contributing the final result.

Of course I'll post a final build for Arduino to use. Given the very slow speed of building on Raspberry Pi, it will very likely be 2 to 3 days to complete.

[PaulStoffregen]

Just a quick update, my first test build is still running, now going more than 38 hours! It seems to have successfully completed both stages of the gcc build and is now building other stuff like gdb. My hope is to do the first round of testing sometime this weekend.

[PaulStoffregen]

Good news. The build completed. I compiled several test programs. The .HEX files are a perfect match to the ones produced by Arduino's Linux 64 bit toolchain.

@cmaglie The build process produces a final file named "gcc-arm-none-eabi-4_8-2014q1-20160129-linux.tar.bz2". When I complete another build, the "29" will become "31", because building takes 2 days. When extracted, the directory name is "gcc-arm-none-eabi-4_8-2014q1". That's slightly different than "4.8.3-2014q1". Internally, the subdirs and file structure is identical, and at least on the 6 small projects I tested, the compiled .HEX output is a perfect match. Hopefully this minor difference in the top level folder is ok? Or you'll rename it?

[ShorTie8]

Your having better luck then me, lol.
After looking more at this I can remember from years gone by playing with it and not having any success then either.
Some if not all of my problems seem to be from using gcc-4.9 to compile it.
@PaulStoffregen what version of gcc are you using on your system ??
Oh, I'm playing on a Ordoid if that matters because I only have 8gig sdcards for the pi.

@cmaglie since your using precompiled binaries from someone else with no patches, what is the harm of using the system package ??
I understand to fit into the grand scheme of things it has to be just that name for you guys. Not sure why it has to be just that version though. But the old /usr/bin plateform hack still seems to work just fine .. :/~
Would it not save a lot of time and effort if the system stuff was just used ??
Not by you guys, but alot of peeps for years have been doing it with no problems.

[PaulStoffregen]

what version of gcc are you using on your system ??

The native (to Raspberry Pi) gcc I'm using to build the toolchain is:

paul@raspberrypi ~ $ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/4.6/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Debian 4.6.3-14+rpi1' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --disable-sjlj-exceptions --with-arch=armv6 --with-fpu=vfp --with-float=hard --enable-checking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 4.6.3 (Debian 4.6.3-14+rpi1)

Your having better luck then me, lol.
I'm playing on a Ordoid if that matters

Well, did you try using the repository I published. I put the exact commands you need to type in the readme file, so it should be extremely easy. Truly, it's just a matter of downloading those files and typing those commands... and waiting!

what is the harm of using the system package ?? .... Would it not save a lot of time and effort if the system stuff was just used ??

It sure would save a lot of time! (though I'm almost done now....)

Does any system provide a pre-compiled package for the 2014q1 ARM embedded toolchain? It's quite different from the toolchains provided by the Linux distros. Just because it's for "ARM" doesn't mean it works for embedded systems. Arduino can't use the native toolchain on your ARM board, because it produces programs that run on your ARM Linux-based board, not on an ARM-based Arduino.

Even among the embedded tools, significant differences exist between the many versions published over the last few years. In the long run, doing a sloppy job by not having a precise match to the 2014q1 toolchain Arduino uses on the other platforms will be result in anything but "save a lot of time and effort". Sure, it may get things running on ARM Linux sooner, but long-term all Arduino users on those platforms will suffer and Arduino developers will have to contend with frustrating issues if the toolchain doesn't very closely matched what's used on the PC & Mac platforms.

That's why I'm taking a couple extra days to rebuild again and test carefully. I'll have this completed within a day, so please be patient and trust me. I do have quite a lot of experience in these matters and I want to make sure this is done properly.

[PaulStoffregen]

Opps, looks like I made a small mistake... which causes it to fail after 2 days of building. D'oh!! That's why I decided to rebuild, to make sure it really works from the files I published (rather than the fiddling I did last week).

I'm starting another build now. It'll be Wednesday or Thursday before I have this done, due to the slow speed of my Raspberry Pi.

If you downloaded that repository, please grab the latest copy. Perhaps click the "Watch" and "Star" buttons.

[ShorTie8]

Oops, to late, lol.
I fork yours and been working on a script.

gcc version 4.6.3 (Debian 4.6.3-14+rpi1)

hmmm, real old .. :/~
Think I figured out how to switch to gcc-4.8 with update-alternatives.
Started it a few hours ago, Still compiling strong, So I'll let it crash out.

This is the stuff the hack uses

 dpkg -l | grep arm-none-eabi
ii  binutils-arm-none-eabi                2.24.51.20140604-3+5                      armhf        GNU assembler, linker and binary utilities for ARM Cortex-A/R/M processors
ii  gcc-arm-none-eabi                     4.8.4-1+11-1                              armhf        GCC cross compiler for ARM Cortex-A/R/M processors
ii  libnewlib-arm-none-eabi               2.1.0+git20140818.1a8323b-2               all          C library and math library compiled for bare metal using Cortex A/R/M
ii  libstdc++-arm-none-eabi-newlib        4.8.3-9+4                                 all          GNU Standard C++ Library v3 for ARM Cortex-A/R/M processors (newlib)

which is like what we need isn't it ??
binutils might be a little newer

Ps: when did you last update your pi ??

[ShorTie8]

hmmm,
craped out @ some readme.txt

Task [III-11] /armv7l-linux-gnu/package_tbz2/
+ rm -f /usr/local/ARM_Toolchain_2014q1_Source/pkg/gcc-arm-none-eabi-4_8-2014q1-20160201-linux.tar.bz2
+ pushd /usr/local/ARM_Toolchain_2014q1_Source/build-native
/usr/local/ARM_Toolchain_2014q1_Source/build-native /usr/local/ARM_Toolchain_2014q1_Source/src
+ rm -f gcc-arm-none-eabi-4_8-2014q1
+ cp /usr/local/ARM_Toolchain_2014q1_Source/release.txt /usr/local/ARM_Toolchain_2014q1_Source/install-native/share/doc/gcc-arm                              -none-eabi/
+ cp /usr/local/ARM_Toolchain_2014q1_Source/readme.txt /usr/local/ARM_Toolchain_2014q1_Source/install-native/share/doc/gcc-arm-                              none-eabi/
cp: cannot stat '/usr/local/ARM_Toolchain_2014q1_Source/readme.txt': No such file or directory
OdroidC1 ARM_Toolchain_2014q1_Source #

there is a bunch of stuff in the install-native directory
lets try

cp readme.md readme.txt

did seem to take 10gigs
I'll spin it up again now

[cmaglie]

@PaulStoffregen thanks for the time you put into this one!

@cmaglie The build process produces a final file named "gcc-arm-none-eabi-4_8-2014q1-20160129-linux.tar.bz2". When I complete another build, the "29" will become "31", because building takes 2 days. When extracted, the directory name is "gcc-arm-none-eabi-4_8-2014q1". That's slightly different than "4.8.3-2014q1". Internally, the subdirs and file structure is identical, and at least on the 6 small projects I tested, the compiled .HEX output is a perfect match. Hopefully this minor difference in the top level folder is ok? Or you'll rename it?

It's ok, the first level folder is ignored by the IDE. Moreover I know that the .bz2 file contains a lot of very heavy .pdf documents that I'm going to remove anyway, so... yes, that's fine.

@cmaglie since your using precompiled binaries from someone else with no patches, what is the harm of using the system package ??

The problem is in the small subtle differences between different compiler versions. Just to give you an idea: we used to run the system provided avr-gcc on linux systems, but I remember that debian at some point shipped a buggy version of avr-gcc that broken a lot of sketches using the Ethernet library. The annoying thing is that only linux systems were affected because mac and win used a bundled avr-gcc. This always lead to difficult to diagnose bug. As @PaulStoffregen said, I prefer to spend more time now, but have a coherent behaviour between the various OS.

Some if not all of my problems seem to be from using gcc-4.9 to compile it.

I tried to run the build on a Rpi2 and the build failed on binutils with a:
warnings treated as errors

I guess it's the same error encountered by @ShorTie8, the newer gcc seems to be able to detect more warnings and since binutils is compiled with the -Werror flag set the build fails.

I found the following configuration files that adds the -Werror flag:

pi@raspberrypi:ARM_Toolchain_2014q1_Source/src $ grep GCC_WARN_CFLAGS=\"\$GCC_WARN_CFLAGS\ -Werror\" -r *
binutils/gprof/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/ld/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/opcodes/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/binutils/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/gold/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/gas/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/bfd/warning.m4:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
binutils/bfd/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
gdb/opcodes/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
gdb/bfd/warning.m4:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"
gdb/bfd/configure:    GCC_WARN_CFLAGS="$GCC_WARN_CFLAGS -Werror"

I've commented out all the lines with:

pi@raspberrypi:ARM_Toolchain_2014q1_Source/src $ sed -e '/GCC_WARN_CFLAGS -Werror/s/^/#/' -i */*/configure
pi@raspberrypi:ARM_Toolchain_2014q1_Source/src $ sed -e '/GCC_WARN_CFLAGS -Werror/s/^/#/' -i */*/warning.m4

and restarted the build. For now it's going without errors... let's see how it goes.
A lot of builds running in parallel now :-)

[ShorTie8]

The problem is in the small subtle differences between different compiler versions. 

Maybe that is why using gcc-4.8 would be best.
Ya, -Werror with binutils was the problem.
Like I said before, In Gentoo a new compiler is compiled with the old 1 and then recompiled again with the new 1. I guess so it tries to take all those little things out.

[PaulStoffregen]

Moreover I know that the .bz2 file contains a lot of very heavy .pdf documents that I'm going to remove anyway

FWIW, I disabled the that in build script.

[ShorTie8]

Task [V-1] /md5_checksum/
+ pushd /usr/local/ARM_Toolchain_2014q1_Source/pkg
/usr/local/ARM_Toolchain_2014q1_Source/pkg /usr/local/ARM_Toolchain_2014q1_Source/src
+ rm -rf md5.txt
+ md5sum -b gcc-arm-none-eabi-4_8-2014q1-20160201-linux.tar.bz2
+ '[' xyes '!=' xyes ']'
+ md5sum -b gcc-arm-none-eabi-4_8-2014q1-20160201-src.tar.bz2
+ popd
/usr/local/ARM_Toolchain_2014q1_Source/src
+ echo Mon Feb 1 15:11:40 EST 2016
Mon Feb 1 15:11:40 EST 2016
+ date
Mon Feb  1 20:54:02 EST 2016
+ exit 0

about 5 1/2 hours to compile

ls -l pkg
total 191032
-rw-r--r-- 1 root staff 43849879 Feb 1 20:37 gcc-arm-none-eabi-4_8-2014q1-20160201-linux.tar.bz2
-rw-r--r-- 1 root staff 151758766 Feb 1 20:53 gcc-arm-none-eabi-4_8-2014q1-20160201-src.tar.bz2
-rw-r--r-- 1 root staff 170 Feb 1 20:54 md5.txt

[ShorTie8]

?? @cmaglie

I found the following configuration files that adds the -Werror flag:

Would not all those little errors cause problems, or the potential too...
Would it not be better to leave that and take the time to fix them instead ??
Wonder if it would not make toolchain-avr more stable to use gcc-4.8 on it too ??
Instead of ignoring the errors.

[cmaglie]

Would not all those little errors cause problems, or the potential too...

We are talking about warnings not errors (the build stopped because of the presence of -Werror flag that forced the compiler to stop on both errors and warnings).
From your build I deduce that the same binutils codebase compiles fine with gcc<=4.8 but not with the latest gcc-4.9 that is able to detect more types of warnings that went unnoticed until 4.8.

Wonder if it would not make toolchain-avr more stable to use gcc-4.8 on it too ??

The toolchain obtained from the build is perfectly equivalent, despite the compiler version used to build it.

Would it not be better to leave that and take the time to fix them instead ??

The warnings are probably already fixed upstream, it would be better to use a more recent version of the arm-gcc-none-eabi toolchain instead. BTW this is another task that requires more work (I'm quite sure that some patching on the cores SAM and SAMD is needed and other 3rd party cores may be affected as well), this is a long task and maybe another issue.

[ShorTie8]

Pretty much done with the build script.
did a pull request for it
Have a look

[PaulStoffregen]

Only 5.5 hours is so much faster! Is that an armv6 platform?

I've been building on the very slow Raspberry Pi 1 Model B board, with the assumption we want compatibility with all Raspberry Pi models.

[ShorTie8]

That is on a Odroid C1, which is armv7.
It's a faster board the the rPi2.
I do believe the foundation has move the rPi2 up to armv7

 uname -a
Linux pi 4.1.13-v7+ #826 SMP PREEMPT Fri Nov 13 20:19:03 GMT 2015 armv7l GNU/Linux

the old pi is the only armv6 I do believe

[ShorTie8]

And then you have the ODROID-C2 : 64bit quad-core SBC
http://forum.odroid.com/viewtopic.php?f=135&t=18683

[PaulStoffregen]

Does building on those boards result in a toolchain that can't run on the armv6-based Raspberry Pi Zero & version 1 model B?

[cmaglie]

It seems that compiling on a Raspberry PI2 produces binaries that works on a Raspberry PI 1 (I checked that yesterday with the help of @sandeepmistry not sure the exact model or PI 1 he has BTW).

This is the compiler installed on my RPi2:

pi@raspberrypi:~ $ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/4.9/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Raspbian 4.9.2-10' --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libitm --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-armhf/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-armhf --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-armhf --with-arch-directory=arm --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-sjlj-exceptions --with-arch=armv6 --with-fpu=vfp --with-float=hard --enable-checking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 4.9.2 (Raspbian 4.9.2-10) 

The --with-arch=armv6 flag makes me think that the produced binaries uses armv6 instruction set.

Instead I discovered that cross-compiling on x86 for arm doesn't work on the Pi1 (only Pi2), I had to recompile ctags and liblistserial on the Pi2 to make them work on the Pi1 too. Maybe tweaking the compile flags will make the cross-compiler produce code for armv6 but I don't have the time to experiment with this right now.

I've been building on the very slow Raspberry Pi 1 Model B board, with the assumption we want compatibility with all Raspberry Pi models.

I have a build made on the Pi2 but, even if it may work on the Pi1, since you're doing it I prefer to use your build because it has been made on an older environment it doesn't depends on more recent glibc and it guarantees a wider compatibility.

[ShorTie8]

Just checked on irc with

[10:38:55] <ShorTie> gcc -v gives --with-arch=armv6, does that mean even though it is on a armv7, it's producing armv6 stuff ??
[10:41:20] <shiftplusone> correct

So, ya, pi's only produce armv6 stuff.
he works at the foundation so...

[PaulStoffregen]

I have finally finished. Here's the ready-to-use toolchain:

https://github.com/PaulStoffregen/ARM_Toolchain_2014q1_Source/tree/master/pkg

This actually completed several hours ago, but I spent some extra time to verify. The small collection of test programs I used earlier produced identical results, every HEX file a perfect match, compared to the 2014q1 toolchain in the Linux 64 bit version of Arduino 1.6.7. Initially, I saw a discrepancy when compiling a much larger project. After much digging through objdump listings, it turned out my build process was getting the list of files from the operating system in a different order. The order each file is compiled does not matter, but their order on the final linker command does matter. Once I used an identical linker command, I got a perfect match on the HEX files.

I feel confident this toolchain is ready to be used by Arduino on ARM.

[cmaglie]

I've uploaded the toolchain and updated test_package_arm_index.json.

Thanks to everyone!