Build path inside sketch folder breaks

[sej7278]

Something in 1.6.7 is breaking continuous integration, as i have to delete the build-path directory every time i re-run arduino-builder, otherwise it seems to try to link to the temp files in build-path.

Seems like recursion gone mad as it creates subdirectories of the new build inside the old build or something:

$ tree -L 5 build-esp8266/
build-esp8266/
├── arduino.ar
├── build.options.json
├── core
│   ├── abi.cpp.d
│   ├── abi.cpp.o
│   ├── blah....
│   ├── WString.cpp.d
│   └── WString.cpp.o
├── gizwits_test.ino.bin
├── libraries
├── preproc
│   └── ctags_target_for_gcc_minus_e.cpp
└── sketch
    ├── build-esp8266
    │   ├── preproc
    │   │   ├── ctags_target_for_gcc_minus_e.cpp
    │   │   ├── ctags_target_for_gcc_minus_e.cpp.d
    │   │   └── ctags_target_for_gcc_minus_e.cpp.o
    │   └── sketch
    │       ├── gizwits_test.ino.cpp
    │       ├── gizwits_test.ino.cpp.d
    │       └── gizwits_test.ino.cpp.o
    ├── gizwits_test.ino.cpp
    ├── gizwits_test.ino.cpp.d
    └── gizwits_test.ino.cpp.o

it then exits with "multiple definition of setup" errors such as:

~/arduino16/gizwits_test/build-esp8266/sketch/build-esp8266/preproc/ctags_target_for_gcc_minus_e.cpp.o: In function `HardwareSerial::begin(unsigned long)':

~/arduino16/hardware/esp8266com/esp8266/cores/esp8266/HardwareSerial.h:77: multiple definition of `setup'

the ide seems to work ok oddly enough and outputs "Using previously compiled file: /tmp/build...." rather than trying to reproduce the whole thing again in a subdirectory (unless it finds the build-path is in the same directory as the sketch, in which case it breaks).

it worked fine in arduino-builder 1.0.5 bundled with ide 1.6.6

doesn't require any special sketch or board to reproduce, just that build-path is in the same directory as the sketch i guess, which seems like an ok requirement to me - its nice to be able to commit builds to git, rather than stashing them in /tmp or deleting them all the time.

full build output

[matthijskooijman]

I've changed the title of your issue, since I think the problem is triggered by setting the build path inside the sketch folder. I agree that this setup should be supported, I use it with a Makefile as well. I guess there's two parts of this problem: That all files and directories inside the sketch are copied into the build folder (including the build folder itself), and that sketches are now recursively compiled (which is also the cause of #65). I suspect it would be good to fix both of these parts, even though either one would already prevent this issue.

[sej7278]

i help maintain arduino-mk and we've been putting the equivalent to build-path inside the sketch folder for years without trouble - but we don't support recursively compiling sketches in subdirectories, which arduino-builder didn't either until 1.6.7, so i suspect that is the problem.

you can work around it by calling a make clean all recipe to delete the existing directory before building a new one, but its a shame not to reuse pre-compiled files, and it causes havoc with XP/CI if you nuke the build-path all the time.

[Pimmetje]

I had some trouble building using arduino-builder 1.6.8. Turns out that i am not allowed to build inside the scetch folder. This is unhandy for ci services.

[matthijskooijman]

I've discussed this issue with @cmaglie yesterday, and we agreed the best course would be to disable recursive compilation for sketches again. It seemed like a good idea, but it has side effects like these.

Since recursive compilation in a sketch can be useful, especially for more complex sketches, we're considering to allow a src directory inside the sketch which will be recursively compiled (similar to how libraries are treated). .ino files would still be limited to the root directory, so src can contain e.g. .c, .cpp, .h, etc. files.

[sej7278]

@matthijskooijman did the revert get into 1.6.9 ? seems there's a similar/related issue #139

edit: tested and seems not

[matthijskooijman]

@sej7278, Nope, not yet. I've been working on an implementation as part of some other improvements, but ran into some things that needed refactoring and cleanup beforehand, so it's not finished yet. We considered just reverting recursive compilation for 1.6.9, but then decided against it - we want to remove recursive compilation for the root folder in the same release where we introduce recursive compilation for the src folder, which would provided the cleanest upgrade path for existing sketches. For now this does mean this issue stays unresolved in this release. IIUC there are plans for another release relatively soon, which should contain this change.

[PaulStoffregen]

Building sketch files that can't be viewed or edited in the IDE window is a terrible recipe for user confusion & frustration.

[sej7278]

it does seem a niche request, most people are just using a single .ino file not a full c++ setup

[matthijskooijman]

@PaulStoffregen, good point, so some changes will be needed on the IDE side of things too.