From 322ed0bd1996ad360f2921de439dfdd91deb476c Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 5 Feb 2020 16:28:45 +0100 Subject: [PATCH 001/100] Branch for the implementation of the automagical IntelliSense configuration --- BRANCHNOTES.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 BRANCHNOTES.md diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md new file mode 100644 index 00000000..b346ffe4 --- /dev/null +++ b/BRANCHNOTES.md @@ -0,0 +1,41 @@ +# IntelliSense Autoconfiguration Branch +## Problem +This branch more or less adresses [these](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) issues. + +It implements a parser which parses the output from Arduino's build process to generate a very precise `c_cpp_properties.json` which in turn hopefully renders any user interaction with this file obsolete + +## Branch Goals +### Build Output Parser +The parser which parses the relevant includes, defines, compiler paths and flags from Arduino's build output +### `c_cpp_properties.json` Generator +The generator takes the parser's output and transforms it into a valid `c_cpp_properties.json` file. + +### Configuration Flags +Provide a configuration flag which allows the user to turn this feature off - this is useful for the cases in which this magic fails or the user has a very specific setup. Although this branch tries to eliminate most of the latter cases. + +### Status +| | Tasks | +|-----:|:--------| +| **Build output parser** | :heavy_check_mark: Basic parser working (not committed yet) | +| | :white_check_mark: Support for different boards | +| | :white_check_mark: X-platform support | +| **`c_cpp_properties.json` generator** | :white_check_mark: | +| **Configuration flags** | :white_check_mark: | +| **Unit tests** | :white_check_mark: Basic parser | + +## Motivation +I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives. The old and dysfunct Arduino CDT extension for eclipse somehow stalled (even if it was promising), Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). + +Then remains vscode-arduino. It seems that it isn't completely dead - but almost. Most of the core functionality seems to work (I used it a few days now). But the biggest show stopper is the bad IntelliSense support. + +## Beer Money +You can chip in some beer money to keep me motivated - this is really appreciated. + +[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=PVLCSRZHBJ28G&source=url) + + + +## Useful Links +* [IntelliSense issues on vscode-arduino](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) +* [`c_cpp_properties.json` reference](https://code.visualstudio.com/docs/cpp/c-cpp-properties-schema-reference) +* [Interactive regex debugger](https://regex101.com/) \ No newline at end of file From fd398c755b62ebd7ab2669ce7d32b46951a7d69e Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 5 Feb 2020 21:33:16 +0100 Subject: [PATCH 002/100] * More documentation on this project * Tried to identify most of the tasks * Added beer money support option --- BRANCHNOTES.md | 75 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 69 insertions(+), 6 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index b346ffe4..1a7ed9ab 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -13,29 +13,92 @@ The generator takes the parser's output and transforms it into a valid `c_cpp_pr ### Configuration Flags Provide a configuration flag which allows the user to turn this feature off - this is useful for the cases in which this magic fails or the user has a very specific setup. Although this branch tries to eliminate most of the latter cases. +### Global Tasks in vscode-arduino +Places where `c_cpp_properties.json` gets altered (list in progress) +``` +src/extension.ts + 260, 53: arduinoContextModule.default.arduinoApp.tryToUpdateIncludePaths(); +src/arduino/arduino.ts + 328, 12: public tryToUpdateIncludePaths() { + +``` + ### Status + | | Tasks | |-----:|:--------| -| **Build output parser** | :heavy_check_mark: Basic parser working (not committed yet) | -| | :white_check_mark: Support for different boards | -| | :white_check_mark: X-platform support | -| **`c_cpp_properties.json` generator** | :white_check_mark: | +| **Build output parser** | :heavy_check_mark: Basic parser working* | +| | :white_check_mark: Support for different boards | +| | :white_check_mark: X-platform support | +| **`c_cpp_properties.json` generator** | :heavy_check_mark: Basic objects* | +| | :heavy_check_mark: Basic setting of parsing result* | +| | :heavy_check_mark: Basic file input* | +| | :heavy_check_mark: Basic file output* | +| | :white_check_mark: Merging of parsing result and existing file content | | **Configuration flags** | :white_check_mark: | | **Unit tests** | :white_check_mark: Basic parser | +| | :white_check_mark: JSON input | +| | :white_check_mark: JSON output | +| | :white_check_mark: Configuration merging | +| **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | +* not committed to branch yet ## Motivation I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives. The old and dysfunct Arduino CDT extension for eclipse somehow stalled (even if it was promising), Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). Then remains vscode-arduino. It seems that it isn't completely dead - but almost. Most of the core functionality seems to work (I used it a few days now). But the biggest show stopper is the bad IntelliSense support. -## Beer Money +## Beer Money :beers: You can chip in some beer money to keep me motivated - this is really appreciated. [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=PVLCSRZHBJ28G&source=url) +I will list every supporter here, thanks! + +### Supporters +5$ -> 1 :beer: +1h coding -> 20$ -> 4 :beers: +2020-02-04 Elektronik Workshop: 32 :beers: (8h coding) +2020-02-05 Elektronik Workshop: 40 :beers: (10h coding) + + ## Useful Links * [IntelliSense issues on vscode-arduino](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) * [`c_cpp_properties.json` reference](https://code.visualstudio.com/docs/cpp/c-cpp-properties-schema-reference) -* [Interactive regex debugger](https://regex101.com/) \ No newline at end of file +* [Interactive regex debugger](https://regex101.com/) +* [Git branch management](https://blog.scottlowe.org/2015/01/27/using-fork-branch-git-workflow/) + +## Future Work +* Proper interactive serial terminal (this is the second major show stopper in my opinion) +* Lots of redundand code + * e.g. "upload is a superset of "verify" + * general lack of modularity - the above is the result +* It seems that this extension is pretty chaotic. Most probably some refactoring is necessary. + +---- + +## Implementation + +### Settings +Global user settings, on linux under `~/.config/Code/User/settings.json`, for instance: +```json +{ + "arduino.additionalUrls": "", + "arduino.logLevel": "verbose", + "C_Cpp.default.cppStandard": "c++11", + "C_Cpp.default.cStandard": "c11", + "arduino.disableTestingOpen": true, + "workbench.editor.enablePreview": false +} +``` +Project settings in `.vscode/arduino.json` +``` +{ + "board": "arduino:avr:nano", + "configuration": "cpu=atmega328old", + "sketch": "examples/lcdpong-butenc/lcdpong-butenc.ino", + "port": "/dev/ttyUSB0" +} +``` From 0e4a47baff07c71a4ac3b8d31f9ff276baf95320 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 01:06:03 +0100 Subject: [PATCH 003/100] * More investigations and documentation, especially on intrinsic/built-in compiler header include paths --- BRANCHNOTES.md | 48 ++++++++++++++++++- .../compilerinfo/avr-gcc_built_in_specs.txt | 33 +++++++++++++ .../xtensa-esp32-elf-gcc_built_in_specs.txt | 32 +++++++++++++ 3 files changed, 112 insertions(+), 1 deletion(-) create mode 100644 doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt create mode 100644 doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 1a7ed9ab..f52467fe 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -1,6 +1,6 @@ # IntelliSense Autoconfiguration Branch ## Problem -This branch more or less adresses [these](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) issues. +This branch more or less addresses [these](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) issues (about seven). It implements a parser which parses the output from Arduino's build process to generate a very precise `c_cpp_properties.json` which in turn hopefully renders any user interaction with this file obsolete @@ -24,6 +24,7 @@ src/arduino/arduino.ts ``` ### Status +**2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) | | Tasks | |-----:|:--------| @@ -35,6 +36,7 @@ src/arduino/arduino.ts | | :heavy_check_mark: Basic file input* | | | :heavy_check_mark: Basic file output* | | | :white_check_mark: Merging of parsing result and existing file content | +| | :white_check_mark: Getting intrinsic gcc include paths (partly done)| | **Configuration flags** | :white_check_mark: | | **Unit tests** | :white_check_mark: Basic parser | | | :white_check_mark: JSON input | @@ -81,6 +83,50 @@ I will list every supporter here, thanks! ## Implementation +### `c_cpp_properties.json` Generator +#### Intrinsic Include Paths +Some include paths are built into gcc and don't have to be specified on the command line. This requires that we have to get them from the compiler. + +Just searching the compiler installation directory with something like +```bash +find ~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/ -name "include*" +``` +won't do since not all include directorys are named `include`. Fortunately gcc can be queried about its configuration ([source](https://stackoverflow.com/a/6666338)): +```bash +# generally for C++ +gcc -xc++ -E -v - +# for esp32 +~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc -xc++ -E -v - < /dev/null 2>&1 | tee xtensa-esp32-elf-gcc_built_in_specs.txt +# avr +~/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-gcc -xc++ -E -v - < /dev/null 2>&1 | tee avr-gcc_built_in_specs.txt +``` +The result can be inspected here: +* [xtensa-esp32-elf-gcc_built_in_specs.txt](doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt) +* [avr-gcc_built_in_specs.txt](doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt) +To show the most interesting section in the output of the above commands, for ESP32: +``` +#include "..." search starts here: +#include <...> search starts here: + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0 + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/xtensa-esp32-elf + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/backward + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/include + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/include-fixed + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/usr/include +End of search list. +``` +for AVR: +``` +#include "..." search starts here: +#include <...> search starts here: + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/include + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/include-fixed + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include +End of search list. +``` +As one can see with the ESP32-gcc not all include directories are named `include`. Parsing of this output is pretty trivial though. + ### Settings Global user settings, on linux under `~/.config/Code/User/settings.json`, for instance: ```json diff --git a/doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt b/doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt new file mode 100644 index 00000000..950f0c06 --- /dev/null +++ b/doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt @@ -0,0 +1,33 @@ +Using built-in specs. +Reading specs from /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/device-specs/specs-avr2 +COLLECT_GCC=/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-gcc +Target: avr +Configured with: ../gcc/configure --enable-fixed-point --enable-languages=c,c++ --prefix=/home/jenkins/workspace/avr-gcc-staging/label/debian7-x86_64/objdir --disable-nls --disable-libssp --disable-libada --disable-shared --with-avrlibc=yes --with-dwarf2 --disable-doc --target=avr +Thread model: single +gcc version 7.3.0 (GCC) +COLLECT_GCC_OPTIONS='-E' '-v' '-specs=device-specs/specs-avr2' + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../libexec/gcc/avr/7.3.0/cc1plus -E -quiet -v -iprefix /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/ - -mn-flash=6 -mskip-bug -mn-flash=6 -mskip-bug -fno-rtti -fno-enforce-eh-specs -fno-exceptions +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/avr" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/backward" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/sys-include" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/avr" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/backward" +ignoring duplicate directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/include" +ignoring duplicate directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/include-fixed" +ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/sys-include" +ignoring duplicate directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include" +#include "..." search starts here: +#include <...> search starts here: + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/include + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/include-fixed + /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include +End of search list. +# 1 "" +# 1 "" +# 1 "" +# 1 "" +COMPILER_PATH=/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../libexec/gcc/avr/7.3.0/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../libexec/gcc/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/bin/ +LIBRARY_PATH=/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/lib/ +COLLECT_GCC_OPTIONS='-E' '-v' '-specs=device-specs/specs-avr2' diff --git a/doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt b/doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt new file mode 100644 index 00000000..7e39fe2e --- /dev/null +++ b/doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt @@ -0,0 +1,32 @@ +Using built-in specs. +COLLECT_GCC=/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc +Target: xtensa-esp32-elf +Configured with: /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/configure --build=x86_64-build_pc-linux-gnu --host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf --prefix=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf --with-local-prefix=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/sysroot --with-sysroot=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/sysroot --with-newlib --enable-threads=no --disable-shared --with-pkgversion='crosstool-NG crosstool-ng-1.22.0-80-g6c4433a' --disable-__cxa_atexit --enable-cxx-flags='-fno-rtti -ffunction-sections' --with-gmp=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-mpfr=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-mpc=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-isl=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-cloog=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-libelf=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --enable-lto --enable-target-optspace --without-long-double-128 --disable-libgomp --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libquadmath-support --disable-nls --enable-languages=c,c++ --disable-libstdcxx-verbose --enable-threads=posix --enable-gcov-custom-rtio +Thread model: posix +gcc version 5.2.0 (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) +COLLECT_GCC_OPTIONS='-E' '-v' + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../libexec/gcc/xtensa-esp32-elf/5.2.0/cc1plus -E -quiet -v -iprefix /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/ -isysroot /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot - +ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0" +ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/xtensa-esp32-elf" +ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/backward" +ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/include" +ignoring nonexistent directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/sysroot/include" +ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/include-fixed" +ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include" +#include "..." search starts here: +#include <...> search starts here: + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0 + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/xtensa-esp32-elf + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/backward + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/include + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/include-fixed + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include + /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/usr/include +End of search list. +# 1 "" +# 1 "" +# 1 "" +# 1 "" +COMPILER_PATH=/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../libexec/gcc/xtensa-esp32-elf/5.2.0/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../libexec/gcc/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ +LIBRARY_PATH=/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/lib/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/lib/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/usr/lib/ +COLLECT_GCC_OPTIONS='-E' '-v' From 2e290ee742268af641a67596b9e112bb7cb51c52 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 21:48:34 +0100 Subject: [PATCH 004/100] * Added IntelliSense compiler parsing engine code * First injection of the compiler command parser and IntelliSense auto-configuration. Currently injected into "verify" only. * Updated branch documentation to reflect the current state of this project --- BRANCHNOTES.md | 38 ++-- src/arduino/arduino.ts | 24 ++- src/arduino/intellisense.ts | 360 ++++++++++++++++++++++++++++++++++++ src/common/util.ts | 10 +- 4 files changed, 414 insertions(+), 18 deletions(-) create mode 100644 src/arduino/intellisense.ts diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index f52467fe..0afff62a 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -24,26 +24,30 @@ src/arduino/arduino.ts ``` ### Status -**2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) - +**2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) +**2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. | | Tasks | |-----:|:--------| -| **Build output parser** | :heavy_check_mark: Basic parser working* | -| | :white_check_mark: Support for different boards | +| **Build output parser** | :heavy_check_mark: Basic parser working | +| | :heavy_check_mark: Support for different boards (done for AVR, ESP32, ESP8266) -- The code has been designed such that it is easy to write/add new parser engines (for non gcc compilers for instance) | +| | :heavy_check_mark: Getting intrinsic gcc include paths | +| | :heavy_check_mark: Handling quoted arguments | | | :white_check_mark: X-platform support | -| **`c_cpp_properties.json` generator** | :heavy_check_mark: Basic objects* | -| | :heavy_check_mark: Basic setting of parsing result* | -| | :heavy_check_mark: Basic file input* | -| | :heavy_check_mark: Basic file output* | +| **`c_cpp_properties.json` generator** | :heavy_check_mark: Basic objects | +| | :heavy_check_mark: Basic setting of parsing result | +| | :heavy_check_mark: Basic file input | +| | :heavy_check_mark: Basic file output | | | :white_check_mark: Merging of parsing result and existing file content | -| | :white_check_mark: Getting intrinsic gcc include paths (partly done)| +| | :white_check_mark: Handling inexistent files and folders | | **Configuration flags** | :white_check_mark: | -| **Unit tests** | :white_check_mark: Basic parser | +| **Unit tests** | :white_check_mark: Basic parser (known boards, match/no match)| +| | :white_check_mark: Querying of compiler built-in includes | +| | :white_check_mark: Throwing arbitrary data at parser engines | | | :white_check_mark: JSON input | | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | -* not committed to branch yet +`*` not committed to branch yet ## Motivation I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives. The old and dysfunct Arduino CDT extension for eclipse somehow stalled (even if it was promising), Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). @@ -63,6 +67,7 @@ I will list every supporter here, thanks! 1h coding -> 20$ -> 4 :beers: 2020-02-04 Elektronik Workshop: 32 :beers: (8h coding) 2020-02-05 Elektronik Workshop: 40 :beers: (10h coding) +2020-02-06 Elektronik Workshop: 36 :beers: (9h coding) @@ -83,7 +88,7 @@ I will list every supporter here, thanks! ## Implementation -### `c_cpp_properties.json` Generator +### Build Output Parser #### Intrinsic Include Paths Some include paths are built into gcc and don't have to be specified on the command line. This requires that we have to get them from the compiler. @@ -96,9 +101,9 @@ won't do since not all include directorys are named `include`. Fortunately gcc c # generally for C++ gcc -xc++ -E -v - # for esp32 -~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc -xc++ -E -v - < /dev/null 2>&1 | tee xtensa-esp32-elf-gcc_built_in_specs.txt +~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc -xc++ -E -v - < /dev/null > xtensa-esp32-elf-gcc_built_in_specs.txt 2>&1 # avr -~/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-gcc -xc++ -E -v - < /dev/null 2>&1 | tee avr-gcc_built_in_specs.txt +~/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-gcc -xc++ -E -v - < /dev/null > avr-gcc_built_in_specs.txt 2>&1 ``` The result can be inspected here: * [xtensa-esp32-elf-gcc_built_in_specs.txt](doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt) @@ -127,6 +132,9 @@ End of search list. ``` As one can see with the ESP32-gcc not all include directories are named `include`. Parsing of this output is pretty trivial though. +### `c_cpp_properties.json` Generator + + ### Settings Global user settings, on linux under `~/.config/Code/User/settings.json`, for instance: ```json @@ -148,3 +156,5 @@ Project settings in `.vscode/arduino.json` "port": "/dev/ttyUSB0" } ``` + +### Global Tasks in vscode-arduino diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index b1d1f64e..a01c492c 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -23,6 +23,7 @@ import { ArduinoWorkspace } from "../common/workspace"; import { SerialMonitor } from "../serialmonitor/serialMonitor"; import { UsbDetector } from "../serialmonitor/usbDetector"; import { ProgrammerManager } from "./programmerManager"; +import { CompilerCmdParserEngineGcc, CompilerCmdParser, CCppProperties } from "./intellisense"; /** * Represent an Arduino application based on the official Arduino IDE. @@ -299,7 +300,27 @@ export class ArduinoApp { arduinoChannel.show(); // we need to return the result of verify try { - await util.spawn(this._settings.commandPath, arduinoChannel.channel, args); + var gccParserEngine = new CompilerCmdParserEngineGcc(dc.sketch); + let compilerParser = new CompilerCmdParser([gccParserEngine]); + + await util.spawn(this._settings.commandPath, + arduinoChannel.channel, + args, + {}, + compilerParser.callback); + + // Write compiler command parser result to IntelliSense + // configuration file in case parsing was successful. + if (compilerParser.result) { + var cppPropsPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); + var cppProps = new CCppProperties(cppPropsPath); + cppProps.merge(compilerParser.result); + cppProps.write(); + arduinoChannel.info("IntelliSense configuration generated successfully."); + } else { + arduinoChannel.warning("Failed to generate IntelliSense configuration."); + } + arduinoChannel.end(`Finished verify sketch - ${dc.sketch}${os.EOL}`); return true; } catch (reason) { @@ -311,7 +332,6 @@ export class ArduinoApp { arduinoChannel.error(msg); return false; } - } public tryToUpdateIncludePaths() { diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts new file mode 100644 index 00000000..68d5d9b1 --- /dev/null +++ b/src/arduino/intellisense.ts @@ -0,0 +1,360 @@ + +/* Automatic IntelliSense setup through compiler command parsing. + * + * + * Some Arduino compiler commands from different board packages: + * AVR + * /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-g++ -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10811 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR -I/home/uli/.arduino15/packages/arduino/hardware/avr/1.8.2/cores/arduino -I/home/uli/.arduino15/packages/arduino/hardware/avr/1.8.2/variants/eightanaloginputs -I/home/uli/Projects/arduino/libraries/NewLiquidCrystal_lib -I/home/uli/Projects/arduino/libraries/LcdPong/src -I/home/uli/Projects/arduino/libraries/Encoder -I/home/uli/Projects/arduino/libraries/Button -I/home/uli/.arduino15/packages/arduino/hardware/avr/1.8.2/libraries/Wire/src /tmp/arduino_build_776874/sketch/lcdpong-butenc.ino.cpp -o /tmp/arduino_build_776874/sketch/lcdpong-butenc.ino.cpp.o + * ESP32 + * /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-g++ -DESP_PLATFORM "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" -DHAVE_CONFIG_H -DGCC_NOT_5_2_0=0 -DWITH_POSIX -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/config -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_trace -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_update -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/asio -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bootloader_support -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/coap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/console -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/driver -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-tls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_adc_cal -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_event -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_client -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_server -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_https_ota -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_ringbuf -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ethernet -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/expat -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fatfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freemodbus -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freertos -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/heap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/idf_test -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/jsmn -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/json -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/libsodium -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/log -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/lwip -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mbedtls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mdns -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/micro-ecc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mqtt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/newlib -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nghttp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nvs_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/openssl -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protobuf-c -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protocomm -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/pthread -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/sdmmc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/smartconfig_ack -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/soc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spi_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spiffs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcp_transport -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcpip_adapter -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ulp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/vfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wear_levelling -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wifi_provisioning -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wpa_supplicant -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/xtensa-debug-module -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32-camera -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fb_gfx -std=gnu++11 -Os -g3 -Wpointer-arith -fexceptions -fstack-protector -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -mlongcalls -nostdlib -Wall -Werror=all -Wextra -Wno-error=maybe-uninitialized -Wno-error=unused-function -Wno-error=unused-but-set-variable -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wno-unused-parameter -Wno-unused-but-set-parameter -Wno-missing-field-initializers -Wno-sign-compare -fno-rtti -MMD -c -DF_CPU=240000000L -DARDUINO=10811 -DARDUINO_ESP32_DEV -DARDUINO_ARCH_ESP32 "-DARDUINO_BOARD=\"ESP32_DEV\"" "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" -DESP32 -DCORE_DEBUG_LEVEL=0 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/cores/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/variants/doitESP32devkitV1 -I/home/uli/Projects/arduino/libraries/NewLiquidCrystal_lib -I/home/uli/Projects/arduino/libraries/LcdPong/src -I/home/uli/Projects/arduino/libraries/Encoder -I/home/uli/Projects/arduino/libraries/Button -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/Wire/src /tmp/arduino_build_744383/sketch/lcdpong-butenc.ino.cpp -o /tmp/arduino_build_744383/sketch/lcdpong-butenc.ino.cpp.o + * ESP8266 + * /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-g++ -DESP_PLATFORM "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" -DHAVE_CONFIG_H -DGCC_NOT_5_2_0=0 -DWITH_POSIX -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/config -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_trace -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_update -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/asio -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bootloader_support -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/coap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/console -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/driver -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-tls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_adc_cal -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_event -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_client -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_server -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_https_ota -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_ringbuf -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ethernet -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/expat -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fatfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freemodbus -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freertos -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/heap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/idf_test -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/jsmn -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/json -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/libsodium -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/log -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/lwip -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mbedtls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mdns -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/micro-ecc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mqtt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/newlib -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nghttp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nvs_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/openssl -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protobuf-c -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protocomm -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/pthread -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/sdmmc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/smartconfig_ack -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/soc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spi_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spiffs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcp_transport -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcpip_adapter -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ulp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/vfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wear_levelling -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wifi_provisioning -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wpa_supplicant -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/xtensa-debug-module -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32-camera -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fb_gfx -std=gnu++11 -Os -g3 -Wpointer-arith -fexceptions -fstack-protector -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -mlongcalls -nostdlib -Wall -Werror=all -Wextra -Wno-error=maybe-uninitialized -Wno-error=unused-function -Wno-error=unused-but-set-variable -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wno-unused-parameter -Wno-unused-but-set-parameter -Wno-missing-field-initializers -Wno-sign-compare -fno-rtti -MMD -c -DF_CPU=240000000L -DARDUINO=10811 -DARDUINO_ESP32_DEV -DARDUINO_ARCH_ESP32 "-DARDUINO_BOARD=\"ESP32_DEV\"" "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" -DESP32 -DCORE_DEBUG_LEVEL=0 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/cores/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/variants/doitESP32devkitV1 -I/home/uli/Projects/arduino/libraries/NewLiquidCrystal_lib -I/home/uli/Projects/arduino/libraries/LcdPong/src -I/home/uli/Projects/arduino/libraries/Encoder -I/home/uli/Projects/arduino/libraries/Button -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/Wire/src /tmp/arduino_build_470717/sketch/lcdpong-butenc.ino.cpp -o /tmp/arduino_build_470717/sketch/lcdpong-butenc.ino.cpp.o + */ + +import * as path from "path"; +import * as fs from "fs"; +import { spawnSync } from "child_process"; + +/** Data structure carrying the output from a parsed compiler command. + * All compiler specific option prefixes are removed for includes and + * defines. + */ +export class CompilerCmdParserResult { + includes: Array = []; + defines: Array = []; + options: Array = []; + compiler: string = ""; + /** Dropped arguments like -c -Ox */ + trash: Array = []; +}; + +export type CompilerCmdParserSearchType = string | RegExp; + +/** Base class for any compiler command parser engine. + * If someone needs to write an engine: this is the base class. + * For further inspiration take a look at the implementation of + * CompilerCmdParserEngineGcc. + */ +export abstract class CompilerCmdParserEngine +{ + /** This array should contain the patterns which should match on + * a valid compiler command line to identify the compiler command. + * To be set by the derived class. + */ + protected _match: CompilerCmdParserSearchType[]; + /** This array should contain the patterns which should _NOT_ + * match on a valid compiler command line to identify the + * compiler command. + * To be set by the derived class. + */ + protected _nomatch: CompilerCmdParserSearchType[]; + + /** The parsing function of a matched compiler command line. + * If all conditions hold true (all _match are found and all _nomatch + * are not found), this parsing function is invoked. + * + * Here the derived class has to implement its parsing magic + * to extract the desired includes, defines, compiler flags + * and the compiler command. + * + * @param line A string containing a compiler command line candidate. + * @returns A valid parsing result in case parsing was successful + * and undefined in case it failed fatally. + */ + protected abstract parse(line: string): CompilerCmdParserResult; + /** This function checks if the command line matches the + * requirements given through _match and _nomatch and invokes + * the parse function in case of a match. + * @returns If match was found and parsing was successful + * it returns the result else undefined. + */ + public match(line: string): CompilerCmdParserResult + { + // check for regexes that must match + for (var re of this._match) { + if (line.search(re) == -1) { + return undefined; + } + } + // check for regexes that mustn't match + for (var re of this._nomatch) { + if (line.search(re) != -1) { + return undefined; + } + } + return this.parse(line); + } +} + +/** Compiler command parsing engine for gcc compilers. + * + * + */ +export class CompilerCmdParserEngineGcc + extends CompilerCmdParserEngine +{ + constructor(sketch:string) + { + super(); + + // TODO: windows and osx variants + + this._nomatch = + [ + // make sure Arduino's not testing libraries + /-o\s\/dev\/null/, + ]; + this._match = + [ + // make sure we're running g++ + /-g\+\+\s+/, + // make sure we're compiling + /\s+-c\s+/, + // check if we're compiling the main sketch + path.basename(sketch) + '.cpp.o', + ]; + } + protected parse(line: string): CompilerCmdParserResult + { + var result = new CompilerCmdParserResult(); + var args = line.split(/\s+/); + + for (var arg of args) { + + // drop empty arguments + if (!arg.length) { + continue; + } + + // unpack quoted elements like + // + // "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" + // "-DARDUINO_BOARD=\"ESP32_DEV\"" + // "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" + var packed = arg.match(/^"(.+)"$/); + if (packed) { + arg = packed[1]; + } + + // extract defines + var define = arg.match(/^-D(.+)/); + if (define) { + result.defines.push(define[1]); + continue; + } + + // extract includes + var include = arg.match(/^-I(.+)/); + if (include) { + result.includes.push(include[1]); + continue; + } + + // extract the compiler executable + var c = arg.match(/g\+\+$/); + if (c) { + result.compiler = arg; + continue; + } + + // filter out option trash + var t = arg.match(/^-o|^-O|^-g|^-c|cpp(?:\.o){0,1}$/); + if (t) { + result.trash.push(arg); + continue; + } + + // collect options + var o = arg.match(/^-/); + if (o) { + result.options.push(arg); + continue; + } + + // collect the rest + result.trash.push(arg); + } + + // Query compiler for intrinsic/built-in include paths + if (result.compiler.length > 0) { + + // TODO: Windows + + // Spawn synchronous child process and run bash command + // Source: https://stackoverflow.com/a/6666338 + var compilerinfocmd = `${result.compiler} -xc++ -E -v - < /dev/null 2>&1`; + var child = spawnSync("bash", ["-c", compilerinfocmd], { encoding : 'utf8' }); + + if (child.error || child.status != 0) { + // TODO: report the execution failure + } else { + // Now we look for + // + // #include "..." search starts here: + // #include <...> search starts here: + // ...(include directories list)... + // End of search list. + // + // and extract the include directory list. Could be that some gcc + // even lists something under + // + // #include "..." search starts here: + // + // but I havn't seen it so far. + var includeregex = /^#include\s+<\.\.\.>\ssearch\sstarts\shere\:$(.+)^End\sof\ssearch\slist\.$/ms; + var match = child.stdout.match(includeregex); + if (match) { + // Split list by newlines. Should be platform independent + var lines = match[1].split(/\s*(?:\r|\n)\s*/); + // Filter out empty elements (in most cases only the last element) + lines = lines.filter(function (val:string){return val != ""}); + // Add built-in includes to command line includes + result.includes = [...result.includes, ...lines]; + } else { + // TODO: issue info that include section has not been found + } + } + } + return result; + } +} + +/** A compiler command parser. + * Takes compiler commands line by line and tries to find the compile command + * for the main .ino sketch. From that it tries to extract all includes, + * defines, options and the compiler command itself. + * + * TODO: Make it more generic to support other compilers than gcc + */ +export class CompilerCmdParser +{ + private _result: CompilerCmdParserResult; + private _engines: CompilerCmdParserEngine[]; + + /** Create a compiler command parser. + * Sets up parsing operation. + * @param engines Parsing engines for different compilers + */ + constructor(engines:CompilerCmdParserEngine[]) + { + this._engines = engines; + } + /** Returns the parsing result. + * Returns undefined when the parser fails or when the + * parser didn't run. + */ + get result(): CompilerCmdParserResult + { + return this._result; + } + /** Takes a command line and tries to parse it. + * + * @param line Compiler command line candidate. + * @returns The parsing result if the command line was parsed + * successfully. It returns undefined if no match was found or + * parsing failed. + */ + public parse(line: string): boolean + { + for (var engine of this._engines) { + this._result = engine.match(line); + if (this._result) { + return true; + } + } + return false; + } + /** Returns a callback which can be passed on to other functions + * to call. For instance from stdout callbacks of child processes. + */ + public get callback(): (line: string) => void + { + return (line:string) => { + if (!this._result) { + this.parse(line); + } + } + } +} + +/** Class representing the contents of the IntelliSense + * c_cpp_properties.json configuration file. + * + * @see https://code.visualstudio.com/docs/cpp/c-cpp-properties-schema-reference + */ +export class CCppPropertiesConfiguration { + name: string = "Arduino"; + compilerPath: string = ""; + compilerArgs: string [] = []; + intelliSenseMode: string = "gcc-x64"; // since we're using arduino's compiler + includePath: string[] = []; + forcedInclude: string[] = []; + cStandard: string = "c11"; + cppStandard: string = "c++11"; // as of 1.8.11 arduino is on C++11 + defines: string[] = []; + + constructor(result: CompilerCmdParserResult) + { + this.compilerPath = result.compiler; + this.compilerArgs = result.options; + this.includePath = result.includes; + this.defines = result.defines; + } +} + +export class CCppPropertiesContent +{ + configurations: Array + + constructor(result: CompilerCmdParserResult) + { + this.configurations = [new CCppPropertiesConfiguration(result)]; + } +}; + +export class CCppProperties +{ + proppath: string; + propFileContent: CCppPropertiesContent; + + constructor(proppath: string) + { + this.proppath = proppath; + } + + public read() + { + if (!fs.existsSync(this.proppath)) { + return; + } + const propFileContentPlain = fs.readFileSync(this.proppath, "utf8"); + + // NOTE: in JSON backslashes are escaped to \\\\ + + this.propFileContent = JSON.parse(propFileContentPlain) as CCppPropertiesContent; + } + public merge(result: CompilerCmdParserResult) + { + var pc = new CCppPropertiesContent(result); + + // TODO: + // * merge with existing configuration if desired + // * check if contents changed after merging + // + + this.propFileContent = pc; + } + + public write() + { + // NOTE: in JSON backslashes are escaped to \\\\ + // TODO: + // * check if path exists, create if necessary + // * write file only if modified + + if (this.propFileContent) { + var content = JSON.stringify(this.propFileContent, null, 4); + fs.writeFileSync(this.proppath, content); + } + } +} diff --git a/src/common/util.ts b/src/common/util.ts index 2af4b83b..a11980db 100644 --- a/src/common/util.ts +++ b/src/common/util.ts @@ -200,7 +200,9 @@ export function isArduinoFile(filePath): boolean { return fileExistsSync(filePath) && (path.extname(filePath) === ".ino" || path.extname(filePath) === ".pde"); } -export function spawn(command: string, outputChannel: vscode.OutputChannel, args: string[] = [], options: any = {}): Thenable { +type StdOutCallback = (msg: string)=>void; + +export function spawn(command: string, outputChannel: vscode.OutputChannel, args: string[] = [], options: any = {}, stdoutCallback?: StdOutCallback): Thenable { return new Promise((resolve, reject) => { const stdout = ""; const stderr = ""; @@ -221,7 +223,11 @@ export function spawn(command: string, outputChannel: vscode.OutputChannel, args if (outputChannel) { child.stdout.on("data", (data: Buffer) => { - outputChannel.append(decodeData(data, codepage)); + let decoded = decodeData(data, codepage); + outputChannel.append(decoded); + if (stdoutCallback) { + stdoutCallback(decoded); + } }); child.stderr.on("data", (data: Buffer) => { outputChannel.append(decodeData(data, codepage)); From 064d97ac5cb86598edf0f0a936494b29f4d09501 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 21:54:40 +0100 Subject: [PATCH 005/100] * Fixed some formatting issues (vscode preview renders differently like it seems) --- BRANCHNOTES.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 0afff62a..38d20f98 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -26,6 +26,7 @@ src/arduino/arduino.ts ### Status **2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) **2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. + | | Tasks | |-----:|:--------| | **Build output parser** | :heavy_check_mark: Basic parser working | @@ -47,6 +48,7 @@ src/arduino/arduino.ts | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | + `*` not committed to branch yet ## Motivation From 23feb5a9ad8586beefb1d069908d568430b03ee0 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 22:08:05 +0100 Subject: [PATCH 006/100] Fixed some spelling mistakes --- BRANCHNOTES.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 38d20f98..15c1747b 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -52,7 +52,7 @@ src/arduino/arduino.ts `*` not committed to branch yet ## Motivation -I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives. The old and dysfunct Arduino CDT extension for eclipse somehow stalled (even if it was promising), Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). +I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives. The old and dysfunctional Arduino CDT extension for eclipse somehow stalled (even if it was promising), Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). Then remains vscode-arduino. It seems that it isn't completely dead - but almost. Most of the core functionality seems to work (I used it a few days now). But the biggest show stopper is the bad IntelliSense support. @@ -81,7 +81,7 @@ I will list every supporter here, thanks! ## Future Work * Proper interactive serial terminal (this is the second major show stopper in my opinion) -* Lots of redundand code +* Lots of redundant code * e.g. "upload is a superset of "verify" * general lack of modularity - the above is the result * It seems that this extension is pretty chaotic. Most probably some refactoring is necessary. @@ -98,7 +98,7 @@ Just searching the compiler installation directory with something like ```bash find ~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/ -name "include*" ``` -won't do since not all include directorys are named `include`. Fortunately gcc can be queried about its configuration ([source](https://stackoverflow.com/a/6666338)): +won't do since not all include directories are named `include`. Fortunately gcc can be queried about its configuration ([source](https://stackoverflow.com/a/6666338)): ```bash # generally for C++ gcc -xc++ -E -v - From 1623ba8316b435b8bec84bf2178db964494ecc5c Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 23:29:32 +0100 Subject: [PATCH 007/100] Improved line splitting of built-in include parser after commenting on a pull request :) --- src/arduino/intellisense.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts index 68d5d9b1..2ac795d8 100644 --- a/src/arduino/intellisense.ts +++ b/src/arduino/intellisense.ts @@ -205,7 +205,7 @@ export class CompilerCmdParserEngineGcc var match = child.stdout.match(includeregex); if (match) { // Split list by newlines. Should be platform independent - var lines = match[1].split(/\s*(?:\r|\n)\s*/); + var lines = match[1].split(/\s*(?:\r|\r\n|\n)\s*/); // Filter out empty elements (in most cases only the last element) lines = lines.filter(function (val:string){return val != ""}); // Add built-in includes to command line includes From e0cc57dd1c9b37db2589aabe4362380e61200970 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 23:30:36 +0100 Subject: [PATCH 008/100] * Moved pre-build command into separate member function to reduce code replication, better maintainablility and readability * Added pre-build command to "upload using programmer" since it was (probably unintentional) missing there --- src/arduino/arduino.ts | 55 +++++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 20 deletions(-) diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index a01c492c..5a33f1f0 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -94,6 +94,33 @@ export class ArduinoApp { } } + /** Runs the pre build command. + * Usually before one of + * * verify + * * upload + * * upload using programmer + * @param dc Device context prepared during one of the above actions + * @returns True if successful, false on error. + */ + protected async runPreBuildCommand(dc:DeviceContext): Promise + { + if (dc.prebuild) { + arduinoChannel.info(`Running pre-build command: ${dc.prebuild}`); + const prebuildargs = dc.prebuild.split(" "); + const prebuildCommand = prebuildargs.shift(); + try { + await util.spawn(prebuildCommand, + arduinoChannel.channel, + prebuildargs, + { shell: true, cwd: ArduinoWorkspace.rootPath }); + } catch (ex) { + arduinoChannel.error(`Running pre-build command failed: ${os.EOL}${ex.error}`); + return false; + } + } + return true; + } + public async upload() { const dc = DeviceContext.getInstance(); const boardDescriptor = this.getBoardBuildString(); @@ -129,16 +156,8 @@ export class ArduinoApp { UsbDetector.getInstance().pauseListening(); await vscode.workspace.saveAll(false); - if (dc.prebuild) { - arduinoChannel.info(`Run prebuild command: ${dc.prebuild}`); - const prebuildargs = dc.prebuild.split(" "); - const prebuildCommand = prebuildargs.shift(); - try { - await util.spawn(prebuildCommand, arduinoChannel.channel, prebuildargs, { shell: true, cwd: ArduinoWorkspace.rootPath }); - } catch (ex) { - arduinoChannel.error(`Run prebuild failed: \n${ex.error}`); - return; - } + if (!this.runPreBuildCommand(dc)) { + return; } const appPath = path.join(ArduinoWorkspace.rootPath, dc.sketch); @@ -214,6 +233,10 @@ export class ArduinoApp { UsbDetector.getInstance().pauseListening(); await vscode.workspace.saveAll(false); + if (!this.runPreBuildCommand(dc)) { + return; + } + const appPath = path.join(ArduinoWorkspace.rootPath, dc.sketch); const args = ["--upload", "--board", boardDescriptor, "--port", dc.port, "--useprogrammer", "--pref", "programmer=" + selectProgrammer, appPath]; @@ -265,16 +288,8 @@ export class ArduinoApp { arduinoChannel.start(`Verify sketch - ${dc.sketch}`); - if (dc.prebuild) { - arduinoChannel.info(`Run prebuild command: ${dc.prebuild}`); - const prebuildargs = dc.prebuild.split(" "); - const prebuildCommand = prebuildargs.shift(); - try { - await util.spawn(prebuildCommand, arduinoChannel.channel, prebuildargs, { shell: true, cwd: ArduinoWorkspace.rootPath }); - } catch (ex) { - arduinoChannel.error(`Run prebuild failed: \n${ex.error}`); - return; - } + if (!this.runPreBuildCommand(dc)) { + return; } const appPath = path.join(ArduinoWorkspace.rootPath, dc.sketch); From c8792c4379011588625b6a8b049eb5cfd1e9703c Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 6 Feb 2020 23:34:40 +0100 Subject: [PATCH 009/100] * Notes regarding settings * Notes where to run the auto-generation --- BRANCHNOTES.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 15c1747b..33a052b0 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -48,6 +48,7 @@ src/arduino/arduino.ts | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | +| | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies | `*` not committed to branch yet @@ -150,7 +151,7 @@ Global user settings, on linux under `~/.config/Code/User/settings.json`, for in } ``` Project settings in `.vscode/arduino.json` -``` +```json { "board": "arduino:avr:nano", "configuration": "cpu=atmega328old", @@ -158,5 +159,10 @@ Project settings in `.vscode/arduino.json` "port": "/dev/ttyUSB0" } ``` - +The global settings are [here](src/arduino/vscodeSettings.ts) +```ts +if (VscodeSettings.getInstance().logLevel === "verbose") { + args.push("--verbose"); +} +``` ### Global Tasks in vscode-arduino From 2cff69f8f94685e519980e8e8f20f162baf9db22 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 7 Feb 2020 02:08:35 +0100 Subject: [PATCH 010/100] Fixed spelling. --- package.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/package.json b/package.json index 8c3b9587..c4782f50 100644 --- a/package.json +++ b/package.json @@ -233,7 +233,7 @@ }, "targetArchitecture": { "type": "string", - "description": "The architecture of the debuggee.", + "description": "The architecture of the debugger.", "default": "arm" }, "cwd": { @@ -555,7 +555,6 @@ "eslint-plugin-require-path-exists": "^1.1.7", "eslint-plugin-standard": "^3.0.1", "fancy-log": "^1.3.3", - "gulp": "^4.0.2", "gulp-eslint": "^6.0.0", "gulp-sourcemaps": "^2.4.1", "gulp-tslint": "^8.1.4", @@ -575,6 +574,7 @@ "eventemitter2": "^4.1.0", "express": "^4.14.1", "glob": "^7.1.1", + "gulp": "^4.0.2", "iconv-lite": "^0.4.18", "impor": "^0.1.1", "properties": "^1.2.1", From 2d7feb50208abf175025486b1e5a371a5e7c529d Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 7 Feb 2020 02:55:26 +0100 Subject: [PATCH 011/100] * Added a global configuration switch which allows the IntelliSense auto-configuration to be turned off * Prepared the compiler parser code to be injected into "upload" and "upload using programmer" without overhead * Updated intellisense.ts to use only "let" instead of "var" which is much safer * Updated branch documentation --- BRANCHNOTES.md | 35 +++++++++++---------- package.json | 7 ++++- src/arduino/arduino.ts | 59 +++++++++++++++++++++++++---------- src/arduino/intellisense.ts | 50 +++++++++++++++++------------ src/arduino/vscodeSettings.ts | 6 ++++ 5 files changed, 104 insertions(+), 53 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 33a052b0..edd9007c 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -40,7 +40,9 @@ src/arduino/arduino.ts | | :heavy_check_mark: Basic file output | | | :white_check_mark: Merging of parsing result and existing file content | | | :white_check_mark: Handling inexistent files and folders | -| **Configuration flags** | :white_check_mark: | +| **Configuration flags** | :heavy_check_mark: Disable flag for IntelliSense auto-config | +| | :white_check_mark: Perhaps a general IntelliSense flag `{off/manual, auto, oldstyle}` whereas the old can be removed at some point | +| | :white_check_mark: Fine grained IntelliSense control: Global en-/disable and project override. This is probably more useful since the most boards will hopefully work and for the very special cases the user can disable the feature for this single project but still can enjoy it within his regular projects. | | **Unit tests** | :white_check_mark: Basic parser (known boards, match/no match)| | | :white_check_mark: Querying of compiler built-in includes | | | :white_check_mark: Throwing arbitrary data at parser engines | @@ -48,7 +50,7 @@ src/arduino/arduino.ts | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | -| | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies | +| | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? | `*` not committed to branch yet @@ -139,30 +141,31 @@ As one can see with the ESP32-gcc not all include directories are named `include ### Settings -Global user settings, on linux under `~/.config/Code/User/settings.json`, for instance: +#### Global Settings +Under linux at `~/.config/Code/User/settings.json`, for instance: ```json { "arduino.additionalUrls": "", "arduino.logLevel": "verbose", - "C_Cpp.default.cppStandard": "c++11", - "C_Cpp.default.cStandard": "c11", "arduino.disableTestingOpen": true, "workbench.editor.enablePreview": false } ``` -Project settings in `.vscode/arduino.json` +Code: [src/arduino/arduinoSettings.ts](src/arduino/arduinoSettings.ts) +Code: [src/arduino/vscodeSettings.ts](src/arduino/vscodeSettings.ts) +Validator: [package.json](package.json) + +#### Project Settings +Path in project `.vscode/arduino.json` ```json { - "board": "arduino:avr:nano", - "configuration": "cpu=atmega328old", - "sketch": "examples/lcdpong-butenc/lcdpong-butenc.ino", - "port": "/dev/ttyUSB0" -} -``` -The global settings are [here](src/arduino/vscodeSettings.ts) -```ts -if (VscodeSettings.getInstance().logLevel === "verbose") { - args.push("--verbose"); + "board": "arduino:avr:nano", + "configuration": "cpu=atmega328old", + "sketch": "examples/lcdpong-butenc/lcdpong-butenc.ino", + "port": "/dev/ttyUSB0" } ``` +Code: [src/deviceContext.ts](src/deviceContext.ts) +Validator: [misc/arduinoValidator.json](misc/arduinoValidator.json) + ### Global Tasks in vscode-arduino diff --git a/package.json b/package.json index c4782f50..2f697f52 100644 --- a/package.json +++ b/package.json @@ -489,6 +489,11 @@ "arduino.defaultBaudRate": { "type": "number", "default": 115200 + }, + "arduino.disableIntelliSenseAutoGen": { + "type": "boolean", + "default": false, + "description": "When disabled vscode-arduino will not auto-generate an IntelliSense configuration (i.e. c_cpp_properties.json) by analyzing the compiler output." } } }, @@ -555,6 +560,7 @@ "eslint-plugin-require-path-exists": "^1.1.7", "eslint-plugin-standard": "^3.0.1", "fancy-log": "^1.3.3", + "gulp": "^4.0.2", "gulp-eslint": "^6.0.0", "gulp-sourcemaps": "^2.4.1", "gulp-tslint": "^8.1.4", @@ -574,7 +580,6 @@ "eventemitter2": "^4.1.0", "express": "^4.14.1", "glob": "^7.1.1", - "gulp": "^4.0.2", "iconv-lite": "^0.4.18", "impor": "^0.1.1", "properties": "^1.2.1", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index 5a33f1f0..0a7321d7 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -313,28 +313,17 @@ export class ArduinoApp { } arduinoChannel.show(); - // we need to return the result of verify + try { - var gccParserEngine = new CompilerCmdParserEngineGcc(dc.sketch); - let compilerParser = new CompilerCmdParser([gccParserEngine]); + let compilerParserContext = this.makeCompilerParserContext(dc); await util.spawn(this._settings.commandPath, arduinoChannel.channel, args, {}, - compilerParser.callback); - - // Write compiler command parser result to IntelliSense - // configuration file in case parsing was successful. - if (compilerParser.result) { - var cppPropsPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); - var cppProps = new CCppProperties(cppPropsPath); - cppProps.merge(compilerParser.result); - cppProps.write(); - arduinoChannel.info("IntelliSense configuration generated successfully."); - } else { - arduinoChannel.warning("Failed to generate IntelliSense configuration."); - } + compilerParserContext.callback); + + compilerParserContext.conclude(); arduinoChannel.end(`Finished verify sketch - ${dc.sketch}${os.EOL}`); return true; @@ -349,6 +338,44 @@ export class ArduinoApp { } } + /** Creates a context which is used for compiler command parsing + * during building (verify, upload, ...). + * + * This context makes sure that it can be used in those sections + * without having to check whether this feature is en- or disabled + * and keeps the calling context more readable. + * + * @param dc The device context of the caller. + */ + private makeCompilerParserContext(dc: DeviceContext) + : { callback: (s:string)=>void; conclude: ()=>void; } + { + if (!VscodeSettings.getInstance().disableIntelliSenseAutoGen) { + + // setup the parser with its engines + let gccParserEngine = new CompilerCmdParserEngineGcc(dc.sketch); + let compilerParser = new CompilerCmdParser([gccParserEngine]); + + // set up the function to be called after parsing + let _conclude = () => { + let cppPropsPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); + if (compilerParser.processResult(cppPropsPath)) { + arduinoChannel.info("IntelliSense configuration generated successfully."); + } else { + arduinoChannel.warning("Failed to generate IntelliSense configuration."); + } + }; + return { + callback: compilerParser.callback, + conclude: _conclude, + }; + } + return { + callback: undefined, + conclude: ()=>void{}, + }; + }; + public tryToUpdateIncludePaths() { const configFilePath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); if (!fs.existsSync(configFilePath)) { diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts index 2ac795d8..0343fc50 100644 --- a/src/arduino/intellisense.ts +++ b/src/arduino/intellisense.ts @@ -24,7 +24,7 @@ export class CompilerCmdParserResult { defines: Array = []; options: Array = []; compiler: string = ""; - /** Dropped arguments like -c -Ox */ + /** Dropped arguments like -c -Ox -o, the input and output file. */ trash: Array = []; }; @@ -71,13 +71,13 @@ export abstract class CompilerCmdParserEngine public match(line: string): CompilerCmdParserResult { // check for regexes that must match - for (var re of this._match) { + for (let re of this._match) { if (line.search(re) == -1) { return undefined; } } // check for regexes that mustn't match - for (var re of this._nomatch) { + for (let re of this._nomatch) { if (line.search(re) != -1) { return undefined; } @@ -116,10 +116,10 @@ export class CompilerCmdParserEngineGcc } protected parse(line: string): CompilerCmdParserResult { - var result = new CompilerCmdParserResult(); - var args = line.split(/\s+/); + let result = new CompilerCmdParserResult(); + let args = line.split(/\s+/); - for (var arg of args) { + for (let arg of args) { // drop empty arguments if (!arg.length) { @@ -131,41 +131,41 @@ export class CompilerCmdParserEngineGcc // "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" // "-DARDUINO_BOARD=\"ESP32_DEV\"" // "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" - var packed = arg.match(/^"(.+)"$/); + let packed = arg.match(/^"(.+)"$/); if (packed) { arg = packed[1]; } // extract defines - var define = arg.match(/^-D(.+)/); + let define = arg.match(/^-D(.+)/); if (define) { result.defines.push(define[1]); continue; } // extract includes - var include = arg.match(/^-I(.+)/); + let include = arg.match(/^-I(.+)/); if (include) { result.includes.push(include[1]); continue; } // extract the compiler executable - var c = arg.match(/g\+\+$/); + let c = arg.match(/g\+\+$/); if (c) { result.compiler = arg; continue; } // filter out option trash - var t = arg.match(/^-o|^-O|^-g|^-c|cpp(?:\.o){0,1}$/); + let t = arg.match(/^-o|^-O|^-g|^-c|cpp(?:\.o){0,1}$/); if (t) { result.trash.push(arg); continue; } // collect options - var o = arg.match(/^-/); + let o = arg.match(/^-/); if (o) { result.options.push(arg); continue; @@ -182,8 +182,8 @@ export class CompilerCmdParserEngineGcc // Spawn synchronous child process and run bash command // Source: https://stackoverflow.com/a/6666338 - var compilerinfocmd = `${result.compiler} -xc++ -E -v - < /dev/null 2>&1`; - var child = spawnSync("bash", ["-c", compilerinfocmd], { encoding : 'utf8' }); + let compilerinfocmd = `${result.compiler} -xc++ -E -v - < /dev/null 2>&1`; + let child = spawnSync("bash", ["-c", compilerinfocmd], { encoding : 'utf8' }); if (child.error || child.status != 0) { // TODO: report the execution failure @@ -201,11 +201,11 @@ export class CompilerCmdParserEngineGcc // #include "..." search starts here: // // but I havn't seen it so far. - var includeregex = /^#include\s+<\.\.\.>\ssearch\sstarts\shere\:$(.+)^End\sof\ssearch\slist\.$/ms; - var match = child.stdout.match(includeregex); + let includeregex = /^#include\s+<\.\.\.>\ssearch\sstarts\shere\:$(.+)^End\sof\ssearch\slist\.$/ms; + let match = child.stdout.match(includeregex); if (match) { // Split list by newlines. Should be platform independent - var lines = match[1].split(/\s*(?:\r|\r\n|\n)\s*/); + let lines = match[1].split(/\s*(?:\r|\r\n|\n)\s*/); // Filter out empty elements (in most cases only the last element) lines = lines.filter(function (val:string){return val != ""}); // Add built-in includes to command line includes @@ -256,7 +256,7 @@ export class CompilerCmdParser */ public parse(line: string): boolean { - for (var engine of this._engines) { + for (let engine of this._engines) { this._result = engine.match(line); if (this._result) { return true; @@ -275,6 +275,16 @@ export class CompilerCmdParser } } } + public processResult(configPath: string): boolean + { + if (this._result) { + let cppProps = new CCppProperties(configPath); + cppProps.merge(this._result); + cppProps.write(); + return true; + } + return false; + } } /** Class representing the contents of the IntelliSense @@ -335,7 +345,7 @@ export class CCppProperties } public merge(result: CompilerCmdParserResult) { - var pc = new CCppPropertiesContent(result); + let pc = new CCppPropertiesContent(result); // TODO: // * merge with existing configuration if desired @@ -353,7 +363,7 @@ export class CCppProperties // * write file only if modified if (this.propFileContent) { - var content = JSON.stringify(this.propFileContent, null, 4); + let content = JSON.stringify(this.propFileContent, null, 4); fs.writeFileSync(this.proppath, content); } } diff --git a/src/arduino/vscodeSettings.ts b/src/arduino/vscodeSettings.ts index f66e268b..9192a656 100644 --- a/src/arduino/vscodeSettings.ts +++ b/src/arduino/vscodeSettings.ts @@ -15,6 +15,7 @@ const configKeys = { IGNORE_BOARDS: "arduino.ignoreBoards", SKIP_HEADER_PROVIDER: "arduino.skipHeaderProvider", DEFAULT_BAUD_RATE: "arduino.defaultBaudRate", + DISABLE_INTELLISENSE_AUTO_GEN: "arduino.disableIntelliSenseAutoGen", }; export interface IVscodeSettings { @@ -28,6 +29,7 @@ export interface IVscodeSettings { ignoreBoards: string[]; skipHeaderProvider: boolean; defaultBaudRate: number; + disableIntelliSenseAutoGen: boolean; updateAdditionalUrls(urls: string | string[]): void; } @@ -87,6 +89,10 @@ export class VscodeSettings implements IVscodeSettings { return this.getConfigValue(configKeys.SKIP_HEADER_PROVIDER); } + public get disableIntelliSenseAutoGen(): boolean { + return this.getConfigValue(configKeys.DISABLE_INTELLISENSE_AUTO_GEN); + } + public async updateAdditionalUrls(value) { await this.setConfigValue(configKeys.ADDITIONAL_URLS, value, true); } From cce13503daaae3891913c73f47d380ea8b3c1d53 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 7 Feb 2020 03:11:14 +0100 Subject: [PATCH 012/100] Added two more TODOs to branch documentation --- BRANCHNOTES.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index edd9007c..6914971f 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -51,6 +51,8 @@ src/arduino/arduino.ts | | :white_check_mark: Configuration merging | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | | | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? | +| | :white_check_mark: Document configuration settings in [README.md](README.md) | +| | :white_check_mark: Document features in [README.md](README.md) | `*` not committed to branch yet From 8655f261ec23f6964cec3ec030c5af992afad074 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 7 Feb 2020 03:15:53 +0100 Subject: [PATCH 013/100] Added even more TODOs to branch documentation --- BRANCHNOTES.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 6914971f..d63710a3 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -49,10 +49,12 @@ src/arduino/arduino.ts | | :white_check_mark: JSON input | | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | +| | :white_check_mark: Write configuration on change only | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | | | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? | | | :white_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) | +| | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet From 0067092293b73b056839ffb5604a71630749aaf6 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 7 Feb 2020 03:19:09 +0100 Subject: [PATCH 014/100] Moved TODO in branch documentation --- BRANCHNOTES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index d63710a3..a04b2875 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -40,6 +40,7 @@ src/arduino/arduino.ts | | :heavy_check_mark: Basic file output | | | :white_check_mark: Merging of parsing result and existing file content | | | :white_check_mark: Handling inexistent files and folders | +| | :white_check_mark: Write configuration on change only | | **Configuration flags** | :heavy_check_mark: Disable flag for IntelliSense auto-config | | | :white_check_mark: Perhaps a general IntelliSense flag `{off/manual, auto, oldstyle}` whereas the old can be removed at some point | | | :white_check_mark: Fine grained IntelliSense control: Global en-/disable and project override. This is probably more useful since the most boards will hopefully work and for the very special cases the user can disable the feature for this single project but still can enjoy it within his regular projects. | @@ -49,7 +50,6 @@ src/arduino/arduino.ts | | :white_check_mark: JSON input | | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | -| | :white_check_mark: Write configuration on change only | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | | | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? | | | :white_check_mark: Document configuration settings in [README.md](README.md) | From 8b11b31bb80f4d06f11e509ffcf770bfcca1a549 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 03:10:06 +0100 Subject: [PATCH 015/100] Documenting ... --- BRANCHNOTES.md | 68 ++++++++++++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 24 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index a04b2875..4a1eecfd 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -14,14 +14,6 @@ The generator takes the parser's output and transforms it into a valid `c_cpp_pr Provide a configuration flag which allows the user to turn this feature off - this is useful for the cases in which this magic fails or the user has a very specific setup. Although this branch tries to eliminate most of the latter cases. ### Global Tasks in vscode-arduino -Places where `c_cpp_properties.json` gets altered (list in progress) -``` -src/extension.ts - 260, 53: arduinoContextModule.default.arduinoApp.tryToUpdateIncludePaths(); -src/arduino/arduino.ts - 328, 12: public tryToUpdateIncludePaths() { - -``` ### Status **2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) @@ -41,19 +33,21 @@ src/arduino/arduino.ts | | :white_check_mark: Merging of parsing result and existing file content | | | :white_check_mark: Handling inexistent files and folders | | | :white_check_mark: Write configuration on change only | -| **Configuration flags** | :heavy_check_mark: Disable flag for IntelliSense auto-config | -| | :white_check_mark: Perhaps a general IntelliSense flag `{off/manual, auto, oldstyle}` whereas the old can be removed at some point | -| | :white_check_mark: Fine grained IntelliSense control: Global en-/disable and project override. This is probably more useful since the most boards will hopefully work and for the very special cases the user can disable the feature for this single project but still can enjoy it within his regular projects. | +| | :white_check_mark: Option to backup old configurations? | +| **Configuration flags** | :heavy_check_mark: Provide global disable flag for IntelliSense auto-config | +| | :white_check_mark: Provide project specific override for the global flag - most users will likely use the default setup and disable auto-generation for very specific projects | | **Unit tests** | :white_check_mark: Basic parser (known boards, match/no match)| | | :white_check_mark: Querying of compiler built-in includes | | | :white_check_mark: Throwing arbitrary data at parser engines | | | :white_check_mark: JSON input | | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | -| **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` | -| | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? | +| **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense. (Partially done - documented in the [General Tasks](#General-Tasks) section | +| | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | | | :white_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) | +| | :white_check_mark: How to handle compilation failure? Only set if more comprehensive | +| | :white_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and tests can be run without the heavy vscode-arduino rucksack | | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet @@ -85,27 +79,35 @@ I will list every supporter here, thanks! * [`c_cpp_properties.json` reference](https://code.visualstudio.com/docs/cpp/c-cpp-properties-schema-reference) * [Interactive regex debugger](https://regex101.com/) * [Git branch management](https://blog.scottlowe.org/2015/01/27/using-fork-branch-git-workflow/) +* [Collapsible Markdown](https://gist.githubusercontent.com/joyrexus/16041f2426450e73f5df9391f7f7ae5f/raw/f774f242feff6bae4a5be7d6c71aa5df2e3fcb0e/README.md) +## Issues Concerning this Project + * https://github.com/Microsoft/vscode-cpptools/issues/1750 + * Problems with IntelliSense itself https://github.com/microsoft/vscode-cpptools/issues/1034 + * Logging for IntelliSense https://code.visualstudio.com/docs/cpp/enable-logging-cpp ## Future Work * Proper interactive serial terminal (this is the second major show stopper in my opinion) * Lots of redundant code * e.g. "upload is a superset of "verify" * general lack of modularity - the above is the result * It seems that this extension is pretty chaotic. Most probably some refactoring is necessary. - +* Possibility to jump to compilation errors from compiler output and highlight compiler errors ---- ## Implementation +**Note** Check this vscode feature: +``` +Configuration provider +The ID of a VS Code extension that can provide IntelliSense configuration information for source files. For example, use the VS Code extension ID ms-vscode.cmake-tools to provide configuration information from the CMake Tools extension. +``` ### Build Output Parser #### Intrinsic Include Paths -Some include paths are built into gcc and don't have to be specified on the command line. This requires that we have to get them from the compiler. - -Just searching the compiler installation directory with something like +Some include paths are built into gcc and don't have to be specified on the command line. Just searching the compiler installation directory with something like ```bash find ~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/ -name "include*" ``` -won't do since not all include directories are named `include`. Fortunately gcc can be queried about its configuration ([source](https://stackoverflow.com/a/6666338)): +won't do since not all include directories are named `include`. Fortunately gcc can be queried about its configuration ([source](https://stackoverflow.com/a/6666338)) -- built-in include paths are part of it: ```bash # generally for C++ gcc -xc++ -E -v - @@ -117,6 +119,7 @@ gcc -xc++ -E -v - The result can be inspected here: * [xtensa-esp32-elf-gcc_built_in_specs.txt](doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt) * [avr-gcc_built_in_specs.txt](doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt) + To show the most interesting section in the output of the above commands, for ESP32: ``` #include "..." search starts here: @@ -155,9 +158,9 @@ Under linux at `~/.config/Code/User/settings.json`, for instance: "workbench.editor.enablePreview": false } ``` -Code: [src/arduino/arduinoSettings.ts](src/arduino/arduinoSettings.ts) -Code: [src/arduino/vscodeSettings.ts](src/arduino/vscodeSettings.ts) -Validator: [package.json](package.json) +Code: [src/arduino/arduinoSettings.ts](src/arduino/arduinoSettings.ts) +Code: [src/arduino/vscodeSettings.ts](src/arduino/vscodeSettings.ts) +Validator: [package.json](package.json) #### Project Settings Path in project `.vscode/arduino.json` @@ -169,7 +172,24 @@ Path in project `.vscode/arduino.json` "port": "/dev/ttyUSB0" } ``` -Code: [src/deviceContext.ts](src/deviceContext.ts) -Validator: [misc/arduinoValidator.json](misc/arduinoValidator.json) +Code: [src/deviceContext.ts](src/deviceContext.ts) +Validator: [misc/arduinoValidator.json](misc/arduinoValidator.json) -### Global Tasks in vscode-arduino +### General Tasks +#### Removing existing Attempts which mess with c_cpp_properties.json or Intellisense + +Remove these as they are helpless attempts to get IntelliSense working: +```ts +//src/arduino/arduino.ts + tryToUpdateIncludePaths() + addLibPath(libraryPath: string) + getDefaultForcedIncludeFiles() + // parts in + openExample() + + //probably not needed anymore: + getDefaultPackageLibPaths() + +``` +Remove this as this messes in an unpredictable and helpless way with Intellisense +[src/langService/completionProvider.ts](src/langService/completionProvider.ts) From 55d183274918419d8337fb203a23fa669fffd593 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 12:28:25 +0100 Subject: [PATCH 016/100] Moved compiler parser to cocopa package for better testability and maintainability --- BRANCHNOTES.md | 6 +- package.json | 3 +- src/arduino/arduino.ts | 8 +- src/arduino/intellisense.ts | 370 ------------------------------------ 4 files changed, 9 insertions(+), 378 deletions(-) delete mode 100644 src/arduino/intellisense.ts diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 4a1eecfd..cea437a4 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -36,8 +36,8 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: Option to backup old configurations? | | **Configuration flags** | :heavy_check_mark: Provide global disable flag for IntelliSense auto-config | | | :white_check_mark: Provide project specific override for the global flag - most users will likely use the default setup and disable auto-generation for very specific projects | -| **Unit tests** | :white_check_mark: Basic parser (known boards, match/no match)| -| | :white_check_mark: Querying of compiler built-in includes | +| **Unit tests** | :heavy_check_mark: Basic parser (known boards, match/no match)| +| | :heavy_check_mark: Querying of compiler built-in includes (Note: to be changed to generic compiler such that Arduino is not necessary for unit testing) | | | :white_check_mark: Throwing arbitrary data at parser engines | | | :white_check_mark: JSON input | | | :white_check_mark: JSON output | @@ -47,7 +47,7 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) | | | :white_check_mark: How to handle compilation failure? Only set if more comprehensive | -| | :white_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and tests can be run without the heavy vscode-arduino rucksack | +| | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet diff --git a/package.json b/package.json index 2f697f52..63ce9390 100644 --- a/package.json +++ b/package.json @@ -570,12 +570,13 @@ "plugin-error": "^1.0.1", "tslint": "^5.20.1", "typemoq": "^1.6.0", - "typescript": "^2.2.1", + "typescript": "^3.7.5", "vscode": "^1.1.36", "webpack": "^4.41.2" }, "dependencies": { "body-parser": "^1.16.1", + "cocopa": "0.0.3", "compare-versions": "^3.4.0", "eventemitter2": "^4.1.0", "express": "^4.14.1", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index 0a7321d7..384da933 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -6,6 +6,7 @@ import * as glob from "glob"; import * as os from "os"; import * as path from "path"; import * as vscode from "vscode"; +import * as ccp from "cocopa"; import * as constants from "../common/constants"; import * as util from "../common/util"; @@ -23,7 +24,6 @@ import { ArduinoWorkspace } from "../common/workspace"; import { SerialMonitor } from "../serialmonitor/serialMonitor"; import { UsbDetector } from "../serialmonitor/usbDetector"; import { ProgrammerManager } from "./programmerManager"; -import { CompilerCmdParserEngineGcc, CompilerCmdParser, CCppProperties } from "./intellisense"; /** * Represent an Arduino application based on the official Arduino IDE. @@ -353,8 +353,8 @@ export class ArduinoApp { if (!VscodeSettings.getInstance().disableIntelliSenseAutoGen) { // setup the parser with its engines - let gccParserEngine = new CompilerCmdParserEngineGcc(dc.sketch); - let compilerParser = new CompilerCmdParser([gccParserEngine]); + let gccParserEngine = new ccp.ParserGcc(dc.sketch); + let compilerParser = new ccp.Runner([gccParserEngine]); // set up the function to be called after parsing let _conclude = () => { @@ -366,7 +366,7 @@ export class ArduinoApp { } }; return { - callback: compilerParser.callback, + callback: compilerParser.callback(), conclude: _conclude, }; } diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts deleted file mode 100644 index 0343fc50..00000000 --- a/src/arduino/intellisense.ts +++ /dev/null @@ -1,370 +0,0 @@ - -/* Automatic IntelliSense setup through compiler command parsing. - * - * - * Some Arduino compiler commands from different board packages: - * AVR - * /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-g++ -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10811 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR -I/home/uli/.arduino15/packages/arduino/hardware/avr/1.8.2/cores/arduino -I/home/uli/.arduino15/packages/arduino/hardware/avr/1.8.2/variants/eightanaloginputs -I/home/uli/Projects/arduino/libraries/NewLiquidCrystal_lib -I/home/uli/Projects/arduino/libraries/LcdPong/src -I/home/uli/Projects/arduino/libraries/Encoder -I/home/uli/Projects/arduino/libraries/Button -I/home/uli/.arduino15/packages/arduino/hardware/avr/1.8.2/libraries/Wire/src /tmp/arduino_build_776874/sketch/lcdpong-butenc.ino.cpp -o /tmp/arduino_build_776874/sketch/lcdpong-butenc.ino.cpp.o - * ESP32 - * /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-g++ -DESP_PLATFORM "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" -DHAVE_CONFIG_H -DGCC_NOT_5_2_0=0 -DWITH_POSIX -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/config -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_trace -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_update -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/asio -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bootloader_support -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/coap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/console -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/driver -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-tls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_adc_cal -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_event -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_client -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_server -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_https_ota -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_ringbuf -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ethernet -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/expat -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fatfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freemodbus -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freertos -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/heap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/idf_test -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/jsmn -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/json -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/libsodium -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/log -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/lwip -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mbedtls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mdns -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/micro-ecc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mqtt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/newlib -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nghttp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nvs_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/openssl -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protobuf-c -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protocomm -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/pthread -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/sdmmc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/smartconfig_ack -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/soc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spi_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spiffs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcp_transport -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcpip_adapter -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ulp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/vfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wear_levelling -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wifi_provisioning -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wpa_supplicant -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/xtensa-debug-module -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32-camera -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fb_gfx -std=gnu++11 -Os -g3 -Wpointer-arith -fexceptions -fstack-protector -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -mlongcalls -nostdlib -Wall -Werror=all -Wextra -Wno-error=maybe-uninitialized -Wno-error=unused-function -Wno-error=unused-but-set-variable -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wno-unused-parameter -Wno-unused-but-set-parameter -Wno-missing-field-initializers -Wno-sign-compare -fno-rtti -MMD -c -DF_CPU=240000000L -DARDUINO=10811 -DARDUINO_ESP32_DEV -DARDUINO_ARCH_ESP32 "-DARDUINO_BOARD=\"ESP32_DEV\"" "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" -DESP32 -DCORE_DEBUG_LEVEL=0 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/cores/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/variants/doitESP32devkitV1 -I/home/uli/Projects/arduino/libraries/NewLiquidCrystal_lib -I/home/uli/Projects/arduino/libraries/LcdPong/src -I/home/uli/Projects/arduino/libraries/Encoder -I/home/uli/Projects/arduino/libraries/Button -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/Wire/src /tmp/arduino_build_744383/sketch/lcdpong-butenc.ino.cpp -o /tmp/arduino_build_744383/sketch/lcdpong-butenc.ino.cpp.o - * ESP8266 - * /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-g++ -DESP_PLATFORM "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" -DHAVE_CONFIG_H -DGCC_NOT_5_2_0=0 -DWITH_POSIX -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/config -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_trace -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/app_update -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/asio -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bootloader_support -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/bt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/coap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/console -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/driver -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-tls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_adc_cal -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_event -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_client -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_http_server -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_https_ota -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp_ringbuf -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ethernet -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/expat -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fatfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freemodbus -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/freertos -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/heap -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/idf_test -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/jsmn -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/json -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/libsodium -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/log -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/lwip -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mbedtls -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mdns -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/micro-ecc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/mqtt -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/newlib -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nghttp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/nvs_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/openssl -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protobuf-c -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/protocomm -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/pthread -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/sdmmc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/smartconfig_ack -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/soc -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spi_flash -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/spiffs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcp_transport -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/tcpip_adapter -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/ulp -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/vfs -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wear_levelling -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wifi_provisioning -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/wpa_supplicant -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/xtensa-debug-module -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp32-camera -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/esp-face -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/tools/sdk/include/fb_gfx -std=gnu++11 -Os -g3 -Wpointer-arith -fexceptions -fstack-protector -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -mlongcalls -nostdlib -Wall -Werror=all -Wextra -Wno-error=maybe-uninitialized -Wno-error=unused-function -Wno-error=unused-but-set-variable -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wno-unused-parameter -Wno-unused-but-set-parameter -Wno-missing-field-initializers -Wno-sign-compare -fno-rtti -MMD -c -DF_CPU=240000000L -DARDUINO=10811 -DARDUINO_ESP32_DEV -DARDUINO_ARCH_ESP32 "-DARDUINO_BOARD=\"ESP32_DEV\"" "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" -DESP32 -DCORE_DEBUG_LEVEL=0 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/cores/esp32 -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/variants/doitESP32devkitV1 -I/home/uli/Projects/arduino/libraries/NewLiquidCrystal_lib -I/home/uli/Projects/arduino/libraries/LcdPong/src -I/home/uli/Projects/arduino/libraries/Encoder -I/home/uli/Projects/arduino/libraries/Button -I/home/uli/.arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/Wire/src /tmp/arduino_build_470717/sketch/lcdpong-butenc.ino.cpp -o /tmp/arduino_build_470717/sketch/lcdpong-butenc.ino.cpp.o - */ - -import * as path from "path"; -import * as fs from "fs"; -import { spawnSync } from "child_process"; - -/** Data structure carrying the output from a parsed compiler command. - * All compiler specific option prefixes are removed for includes and - * defines. - */ -export class CompilerCmdParserResult { - includes: Array = []; - defines: Array = []; - options: Array = []; - compiler: string = ""; - /** Dropped arguments like -c -Ox -o, the input and output file. */ - trash: Array = []; -}; - -export type CompilerCmdParserSearchType = string | RegExp; - -/** Base class for any compiler command parser engine. - * If someone needs to write an engine: this is the base class. - * For further inspiration take a look at the implementation of - * CompilerCmdParserEngineGcc. - */ -export abstract class CompilerCmdParserEngine -{ - /** This array should contain the patterns which should match on - * a valid compiler command line to identify the compiler command. - * To be set by the derived class. - */ - protected _match: CompilerCmdParserSearchType[]; - /** This array should contain the patterns which should _NOT_ - * match on a valid compiler command line to identify the - * compiler command. - * To be set by the derived class. - */ - protected _nomatch: CompilerCmdParserSearchType[]; - - /** The parsing function of a matched compiler command line. - * If all conditions hold true (all _match are found and all _nomatch - * are not found), this parsing function is invoked. - * - * Here the derived class has to implement its parsing magic - * to extract the desired includes, defines, compiler flags - * and the compiler command. - * - * @param line A string containing a compiler command line candidate. - * @returns A valid parsing result in case parsing was successful - * and undefined in case it failed fatally. - */ - protected abstract parse(line: string): CompilerCmdParserResult; - /** This function checks if the command line matches the - * requirements given through _match and _nomatch and invokes - * the parse function in case of a match. - * @returns If match was found and parsing was successful - * it returns the result else undefined. - */ - public match(line: string): CompilerCmdParserResult - { - // check for regexes that must match - for (let re of this._match) { - if (line.search(re) == -1) { - return undefined; - } - } - // check for regexes that mustn't match - for (let re of this._nomatch) { - if (line.search(re) != -1) { - return undefined; - } - } - return this.parse(line); - } -} - -/** Compiler command parsing engine for gcc compilers. - * - * - */ -export class CompilerCmdParserEngineGcc - extends CompilerCmdParserEngine -{ - constructor(sketch:string) - { - super(); - - // TODO: windows and osx variants - - this._nomatch = - [ - // make sure Arduino's not testing libraries - /-o\s\/dev\/null/, - ]; - this._match = - [ - // make sure we're running g++ - /-g\+\+\s+/, - // make sure we're compiling - /\s+-c\s+/, - // check if we're compiling the main sketch - path.basename(sketch) + '.cpp.o', - ]; - } - protected parse(line: string): CompilerCmdParserResult - { - let result = new CompilerCmdParserResult(); - let args = line.split(/\s+/); - - for (let arg of args) { - - // drop empty arguments - if (!arg.length) { - continue; - } - - // unpack quoted elements like - // - // "-DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\"" - // "-DARDUINO_BOARD=\"ESP32_DEV\"" - // "-DARDUINO_VARIANT=\"doitESP32devkitV1\"" - let packed = arg.match(/^"(.+)"$/); - if (packed) { - arg = packed[1]; - } - - // extract defines - let define = arg.match(/^-D(.+)/); - if (define) { - result.defines.push(define[1]); - continue; - } - - // extract includes - let include = arg.match(/^-I(.+)/); - if (include) { - result.includes.push(include[1]); - continue; - } - - // extract the compiler executable - let c = arg.match(/g\+\+$/); - if (c) { - result.compiler = arg; - continue; - } - - // filter out option trash - let t = arg.match(/^-o|^-O|^-g|^-c|cpp(?:\.o){0,1}$/); - if (t) { - result.trash.push(arg); - continue; - } - - // collect options - let o = arg.match(/^-/); - if (o) { - result.options.push(arg); - continue; - } - - // collect the rest - result.trash.push(arg); - } - - // Query compiler for intrinsic/built-in include paths - if (result.compiler.length > 0) { - - // TODO: Windows - - // Spawn synchronous child process and run bash command - // Source: https://stackoverflow.com/a/6666338 - let compilerinfocmd = `${result.compiler} -xc++ -E -v - < /dev/null 2>&1`; - let child = spawnSync("bash", ["-c", compilerinfocmd], { encoding : 'utf8' }); - - if (child.error || child.status != 0) { - // TODO: report the execution failure - } else { - // Now we look for - // - // #include "..." search starts here: - // #include <...> search starts here: - // ...(include directories list)... - // End of search list. - // - // and extract the include directory list. Could be that some gcc - // even lists something under - // - // #include "..." search starts here: - // - // but I havn't seen it so far. - let includeregex = /^#include\s+<\.\.\.>\ssearch\sstarts\shere\:$(.+)^End\sof\ssearch\slist\.$/ms; - let match = child.stdout.match(includeregex); - if (match) { - // Split list by newlines. Should be platform independent - let lines = match[1].split(/\s*(?:\r|\r\n|\n)\s*/); - // Filter out empty elements (in most cases only the last element) - lines = lines.filter(function (val:string){return val != ""}); - // Add built-in includes to command line includes - result.includes = [...result.includes, ...lines]; - } else { - // TODO: issue info that include section has not been found - } - } - } - return result; - } -} - -/** A compiler command parser. - * Takes compiler commands line by line and tries to find the compile command - * for the main .ino sketch. From that it tries to extract all includes, - * defines, options and the compiler command itself. - * - * TODO: Make it more generic to support other compilers than gcc - */ -export class CompilerCmdParser -{ - private _result: CompilerCmdParserResult; - private _engines: CompilerCmdParserEngine[]; - - /** Create a compiler command parser. - * Sets up parsing operation. - * @param engines Parsing engines for different compilers - */ - constructor(engines:CompilerCmdParserEngine[]) - { - this._engines = engines; - } - /** Returns the parsing result. - * Returns undefined when the parser fails or when the - * parser didn't run. - */ - get result(): CompilerCmdParserResult - { - return this._result; - } - /** Takes a command line and tries to parse it. - * - * @param line Compiler command line candidate. - * @returns The parsing result if the command line was parsed - * successfully. It returns undefined if no match was found or - * parsing failed. - */ - public parse(line: string): boolean - { - for (let engine of this._engines) { - this._result = engine.match(line); - if (this._result) { - return true; - } - } - return false; - } - /** Returns a callback which can be passed on to other functions - * to call. For instance from stdout callbacks of child processes. - */ - public get callback(): (line: string) => void - { - return (line:string) => { - if (!this._result) { - this.parse(line); - } - } - } - public processResult(configPath: string): boolean - { - if (this._result) { - let cppProps = new CCppProperties(configPath); - cppProps.merge(this._result); - cppProps.write(); - return true; - } - return false; - } -} - -/** Class representing the contents of the IntelliSense - * c_cpp_properties.json configuration file. - * - * @see https://code.visualstudio.com/docs/cpp/c-cpp-properties-schema-reference - */ -export class CCppPropertiesConfiguration { - name: string = "Arduino"; - compilerPath: string = ""; - compilerArgs: string [] = []; - intelliSenseMode: string = "gcc-x64"; // since we're using arduino's compiler - includePath: string[] = []; - forcedInclude: string[] = []; - cStandard: string = "c11"; - cppStandard: string = "c++11"; // as of 1.8.11 arduino is on C++11 - defines: string[] = []; - - constructor(result: CompilerCmdParserResult) - { - this.compilerPath = result.compiler; - this.compilerArgs = result.options; - this.includePath = result.includes; - this.defines = result.defines; - } -} - -export class CCppPropertiesContent -{ - configurations: Array - - constructor(result: CompilerCmdParserResult) - { - this.configurations = [new CCppPropertiesConfiguration(result)]; - } -}; - -export class CCppProperties -{ - proppath: string; - propFileContent: CCppPropertiesContent; - - constructor(proppath: string) - { - this.proppath = proppath; - } - - public read() - { - if (!fs.existsSync(this.proppath)) { - return; - } - const propFileContentPlain = fs.readFileSync(this.proppath, "utf8"); - - // NOTE: in JSON backslashes are escaped to \\\\ - - this.propFileContent = JSON.parse(propFileContentPlain) as CCppPropertiesContent; - } - public merge(result: CompilerCmdParserResult) - { - let pc = new CCppPropertiesContent(result); - - // TODO: - // * merge with existing configuration if desired - // * check if contents changed after merging - // - - this.propFileContent = pc; - } - - public write() - { - // NOTE: in JSON backslashes are escaped to \\\\ - // TODO: - // * check if path exists, create if necessary - // * write file only if modified - - if (this.propFileContent) { - let content = JSON.stringify(this.propFileContent, null, 4); - fs.writeFileSync(this.proppath, content); - } - } -} From 11d247391b938a75e78355fa59055111d7a7e1c7 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 12:49:39 +0100 Subject: [PATCH 017/100] Fixed tslint complaints --- src/arduino/arduino.ts | 140 +++++++++++++++++++++-------------------- src/common/util.ts | 10 ++- 2 files changed, 78 insertions(+), 72 deletions(-) diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index 384da933..34cac628 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -1,12 +1,12 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. +import * as ccp from "cocopa"; import * as fs from "fs"; import * as glob from "glob"; import * as os from "os"; import * as path from "path"; import * as vscode from "vscode"; -import * as ccp from "cocopa"; import * as constants from "../common/constants"; import * as util from "../common/util"; @@ -94,33 +94,6 @@ export class ArduinoApp { } } - /** Runs the pre build command. - * Usually before one of - * * verify - * * upload - * * upload using programmer - * @param dc Device context prepared during one of the above actions - * @returns True if successful, false on error. - */ - protected async runPreBuildCommand(dc:DeviceContext): Promise - { - if (dc.prebuild) { - arduinoChannel.info(`Running pre-build command: ${dc.prebuild}`); - const prebuildargs = dc.prebuild.split(" "); - const prebuildCommand = prebuildargs.shift(); - try { - await util.spawn(prebuildCommand, - arduinoChannel.channel, - prebuildargs, - { shell: true, cwd: ArduinoWorkspace.rootPath }); - } catch (ex) { - arduinoChannel.error(`Running pre-build command failed: ${os.EOL}${ex.error}`); - return false; - } - } - return true; - } - public async upload() { const dc = DeviceContext.getInstance(); const boardDescriptor = this.getBoardBuildString(); @@ -315,7 +288,7 @@ export class ArduinoApp { arduinoChannel.show(); try { - let compilerParserContext = this.makeCompilerParserContext(dc); + const compilerParserContext = this.makeCompilerParserContext(dc); await util.spawn(this._settings.commandPath, arduinoChannel.channel, @@ -323,8 +296,9 @@ export class ArduinoApp { {}, compilerParserContext.callback); - compilerParserContext.conclude(); - + if (compilerParserContext.conclude) { + compilerParserContext.conclude(); + } arduinoChannel.end(`Finished verify sketch - ${dc.sketch}${os.EOL}`); return true; } catch (reason) { @@ -338,44 +312,6 @@ export class ArduinoApp { } } - /** Creates a context which is used for compiler command parsing - * during building (verify, upload, ...). - * - * This context makes sure that it can be used in those sections - * without having to check whether this feature is en- or disabled - * and keeps the calling context more readable. - * - * @param dc The device context of the caller. - */ - private makeCompilerParserContext(dc: DeviceContext) - : { callback: (s:string)=>void; conclude: ()=>void; } - { - if (!VscodeSettings.getInstance().disableIntelliSenseAutoGen) { - - // setup the parser with its engines - let gccParserEngine = new ccp.ParserGcc(dc.sketch); - let compilerParser = new ccp.Runner([gccParserEngine]); - - // set up the function to be called after parsing - let _conclude = () => { - let cppPropsPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); - if (compilerParser.processResult(cppPropsPath)) { - arduinoChannel.info("IntelliSense configuration generated successfully."); - } else { - arduinoChannel.warning("Failed to generate IntelliSense configuration."); - } - }; - return { - callback: compilerParser.callback(), - conclude: _conclude, - }; - } - return { - callback: undefined, - conclude: ()=>void{}, - }; - }; - public tryToUpdateIncludePaths() { const configFilePath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); if (!fs.existsSync(configFilePath)) { @@ -797,6 +733,72 @@ Please make sure the folder is not occupied by other procedures .`); this._programmerManager = value; } + /** + * Runs the pre build command. + * Usually before one of + * * verify + * * upload + * * upload using programmer + * @param dc Device context prepared during one of the above actions + * @returns True if successful, false on error. + */ + protected async runPreBuildCommand(dc: DeviceContext): Promise { + if (dc.prebuild) { + arduinoChannel.info(`Running pre-build command: ${dc.prebuild}`); + const prebuildargs = dc.prebuild.split(" "); + const prebuildCommand = prebuildargs.shift(); + try { + await util.spawn(prebuildCommand, + arduinoChannel.channel, + prebuildargs, + { shell: true, cwd: ArduinoWorkspace.rootPath }); + } catch (ex) { + arduinoChannel.error(`Running pre-build command failed: ${os.EOL}${ex.error}`); + return false; + } + } + return true; + } + + /** + * Creates a context which is used for compiler command parsing + * during building (verify, upload, ...). + * + * This context makes sure that it can be used in those sections + * without having to check whether this feature is en- or disabled + * and keeps the calling context more readable. + * + * @param dc The device context of the caller. + */ + private makeCompilerParserContext(dc: DeviceContext) + : { callback: (s: string) => void; conclude: () => void; } { + if (!VscodeSettings.getInstance().disableIntelliSenseAutoGen) { + + // setup the parser with its engines + const gccParserEngine = new ccp.ParserGcc(dc.sketch); + const compilerParser = new ccp.Runner([gccParserEngine]); + + // set up the function to be called after parsing + const _conclude = () => { + const cppPropsPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); + if (compilerParser.processResult(cppPropsPath)) { + arduinoChannel.info("IntelliSense configuration generated successfully."); + } else { + arduinoChannel.warning("Failed to generate IntelliSense configuration."); + } + }; + return { + callback: compilerParser.callback(), + conclude: _conclude, + } + } +// const donothing = () => void {}; + return { + callback: undefined, + conclude: undefined, + } + }; + private getProgrammerString(): string { const selectProgrammer = this.programmerManager.currentProgrammer; if (!selectProgrammer) { diff --git a/src/common/util.ts b/src/common/util.ts index a11980db..ada3e462 100644 --- a/src/common/util.ts +++ b/src/common/util.ts @@ -200,9 +200,13 @@ export function isArduinoFile(filePath): boolean { return fileExistsSync(filePath) && (path.extname(filePath) === ".ino" || path.extname(filePath) === ".pde"); } -type StdOutCallback = (msg: string)=>void; +type StdOutCallback = (msg: string) => void; -export function spawn(command: string, outputChannel: vscode.OutputChannel, args: string[] = [], options: any = {}, stdoutCallback?: StdOutCallback): Thenable { +export function spawn(command: string, + outputChannel: vscode.OutputChannel, + args: string[] = [], + options: any = {}, + stdoutCallback?: StdOutCallback): Thenable { return new Promise((resolve, reject) => { const stdout = ""; const stderr = ""; @@ -223,7 +227,7 @@ export function spawn(command: string, outputChannel: vscode.OutputChannel, args if (outputChannel) { child.stdout.on("data", (data: Buffer) => { - let decoded = decodeData(data, codepage); + const decoded = decodeData(data, codepage); outputChannel.append(decoded); if (stdoutCallback) { stdoutCallback(decoded); From 1d74d5766f037de163b9cc629402a8bde9fc4bc5 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 13:05:30 +0100 Subject: [PATCH 018/100] Fixed misspelled function name --- webpack.config.js | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/webpack.config.js b/webpack.config.js index b63e6aad..81693ed1 100644 --- a/webpack.config.js +++ b/webpack.config.js @@ -17,7 +17,7 @@ function getEntry() { fs.copySync(p, 'out/node_modules/' + mod); } - const list = getDependeciesFromNpm(mod); + const list = getDependenciesFromNpm(mod); const moduleList = list.filter((value, index, self) => { return self.indexOf(value) === index && unbundledModule.indexOf(value) === -1 && !/^@types\//.test(value); }); @@ -29,7 +29,7 @@ function getEntry() { return entry; } -function getDependeciesFromNpm(mod) { +function getDependenciesFromNpm(mod) { let list = []; const deps = mod.dependencies; if (!deps) { @@ -37,7 +37,7 @@ function getDependeciesFromNpm(mod) { } for (const m of Object.keys(deps)) { list.push(m); - list = list.concat(getDependeciesFromNpm(deps[m])); + list = list.concat(getDependenciesFromNpm(deps[m])); } return list; } From cc3f1b06d6f35c49727ff46b37332f94156e2439 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 13:08:59 +0100 Subject: [PATCH 019/100] Updated earned :beers: --- BRANCHNOTES.md | 1 + 1 file changed, 1 insertion(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index cea437a4..be6f11ab 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -71,6 +71,7 @@ I will list every supporter here, thanks! 2020-02-04 Elektronik Workshop: 32 :beers: (8h coding) 2020-02-05 Elektronik Workshop: 40 :beers: (10h coding) 2020-02-06 Elektronik Workshop: 36 :beers: (9h coding) +2020-02-07 Elektronik Workshop: 48 :beers: (12h coding) From 2b62236211954ac45ffe95b1102efc97797e13fa Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 13:40:08 +0100 Subject: [PATCH 020/100] Fixed package.json programming language complaint --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index 63ce9390..e15d0306 100644 --- a/package.json +++ b/package.json @@ -20,7 +20,7 @@ }, "homepage": "https://github.com/Microsoft/vscode-arduino/blob/master/README.md", "categories": [ - "Languages", + "Programming Languages", "Debuggers", "Snippets", "Formatters" From e07d3a5954bb97429ef0079c005326de74ff29d4 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 16:12:58 +0100 Subject: [PATCH 021/100] Added IntelliSense setup auto-generation project (sketch-context) configuration flag which can override the global flag --- BRANCHNOTES.md | 21 ++++++++++++++------- README.md | 12 +++++++++++- misc/arduinoValidator.json | 12 +++++++++++- src/arduino/arduino.ts | 7 ++++++- src/deviceContext.ts | 20 ++++++++++++++++++++ 5 files changed, 62 insertions(+), 10 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index be6f11ab..6f3b1ec4 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -17,7 +17,9 @@ Provide a configuration flag which allows the user to turn this feature off - th ### Status **2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) -**2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. +**2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. +**2020-02-07** Wrote compiler command parser npm package [cocopa](https://www.npmjs.com/package/cocopa) and began writing a test framework for it. Added a global configuration switch which allows the IntelliSense configuration generation to be turned off. +**2020-02-08** Integrated `cocopa` into vscode-arduino. Added project configuration flag which can override the global flag in both ways (forced off, forced on). Made code tslint compliant. Began some documentation in [README.md](README.md) | | Tasks | |-----:|:--------| @@ -35,7 +37,7 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: Write configuration on change only | | | :white_check_mark: Option to backup old configurations? | | **Configuration flags** | :heavy_check_mark: Provide global disable flag for IntelliSense auto-config | -| | :white_check_mark: Provide project specific override for the global flag - most users will likely use the default setup and disable auto-generation for very specific projects | +| | :heavy_check_mark: Provide project specific override for the global flag - most users will likely use the default setup and disable auto-generation for very specific projects | | **Unit tests** | :heavy_check_mark: Basic parser (known boards, match/no match)| | | :heavy_check_mark: Querying of compiler built-in includes (Note: to be changed to generic compiler such that Arduino is not necessary for unit testing) | | | :white_check_mark: Throwing arbitrary data at parser engines | @@ -43,9 +45,9 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense. (Partially done - documented in the [General Tasks](#General-Tasks) section | -| | :white_check_mark: Auto-run verify after setting a board to generate a valid `c_cpp_properties.json`, identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | -| | :white_check_mark: Document configuration settings in [README.md](README.md) | -| | :white_check_mark: Document features in [README.md](README.md) | +| | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | +| | :heavy_check_mark: Document configuration settings in [README.md](README.md) | +| | :white_check_mark: Document features in [README.md](README.md) (partially done) | | | :white_check_mark: How to handle compilation failure? Only set if more comprehensive | | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :white_check_mark: Finally: go through my code and look for TODOs | @@ -67,7 +69,7 @@ I will list every supporter here, thanks! ### Supporters 5$ -> 1 :beer: -1h coding -> 20$ -> 4 :beers: +1h coding -> 20$ -> 4 :beers: (very moderate wage though) 2020-02-04 Elektronik Workshop: 32 :beers: (8h coding) 2020-02-05 Elektronik Workshop: 40 :beers: (10h coding) 2020-02-06 Elektronik Workshop: 36 :beers: (9h coding) @@ -193,4 +195,9 @@ Remove these as they are helpless attempts to get IntelliSense working: ``` Remove this as this messes in an unpredictable and helpless way with Intellisense -[src/langService/completionProvider.ts](src/langService/completionProvider.ts) +[src/langService/completionProvider.ts](src/langService/completionProvider.ts) + +Remove this folder as this is not necessary when Intellisense works properly: +``` +syntaxes/ +``` diff --git a/README.md b/README.md index 4b7af0fa..df1f9188 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,7 @@ This extension provides several commands in the Command Palette (F1 o | `arduino.disableTestingOpen` | Enable/disable automatic sending of a test message to the serial port for checking the open status. The default value is `false` (a test message will be sent). | | `arduino.skipHeaderProvider` | Enable/disable the extension providing completion items for headers. This functionality is included in newer versions of the C++ extension. The default value is `false`.| | `arduino.defaultBaudRate` | Default baud rate for the serial port monitor. The default value is 115200. Supported values are 300, 1200, 2400, 4800, 9600, 19200, 38400, 57600, 74880, 115200, 230400 and 250000 | +| `arduino.disableIntelliSenseAutoGen` | When `true` vscode-arduino will not auto-generate an IntelliSense configuration (i.e. `.vscode/c_cpp_properties.json`) by analyzing Arduino's compiler output. | The following Visual Studio Code settings are available for the Arduino extension. These can be set in global user preferences Ctrl + , or workspace settings (`.vscode/settings.json`). The latter overrides the former. @@ -98,7 +99,8 @@ The following settings are as per sketch settings of the Arduino extension. You "board": "adafruit:samd:adafruit_feather_m0", "output": "../build", "debugger": "jlink", - "prebuild": "bash prebuild.sh" + "prebuild": "bash prebuild.sh", + "disableIntelliSenseAutoGen": "global" } ``` - `sketch` - The main sketch file name of Arduino. @@ -107,6 +109,14 @@ The following settings are as per sketch settings of the Arduino extension. You - `output` - Arduino build output path. If not set, Arduino will create a new temporary output folder each time, which means it cannot reuse the intermediate result of the previous build leading to long verify/upload time, so it is recommended to set the field. Arduino requires that the output path should not be the workspace itself or in a subfolder of the workspace, otherwise, it may not work correctly. By default, this option is not set. It's worth noting that the contents of this file could be deleted during the build process, so pick (or create) a directory that will not store files you want to keep. - `debugger` - The short name of the debugger that will be used when the board itself does not have a debugger and there is more than one debugger available. You can find the list of debuggers [here](https://github.com/Microsoft/vscode-arduino/blob/master/misc/debuggerUsbMapping.json). By default, this option is not set. - `prebuild` - External command before building the sketch file. You should only set one `prebuild` command. `command1 && command2` does not work. If you need to run multiple commands before the build, then create a script. +- `disableIntelliSenseAutoGen` - Override the global auto-generation of the IntelliSense configuration (i.e. `.vscode/c_cpp_properties.json`). Three options are available: + - `"global"`: Use the global settings (default) + - `"disable"`: Disable the auto-generation even if globally enabled + - `"enable"`: Enable the auto-generation even if globally disabled + +## IntelliSense +vscode-arduino auto-configures IntelliSense by default. vscode-arduino analyzes Arduino's compiler output during verify and generates the corresponding configuration file at `.vscode/c_cpp_properties.json` and tries as hard as possible to keep things up to date, e.g. running verify when switching the board or the sketch. +It doesn't makes sense though to run verify repeatedly. Therefore if the workspace reports problems (for instance after adding new includes to a new library) run *verify* such that IntelliSense knows of the new include directories (since the Arduino-backend performs the library resolution externally). ## Debugging Arduino Code preview Before you start to debug your Arduino code, please read [this document](https://code.visualstudio.com/docs/editor/debugging) to learn about the basic mechanisms of debugging in Visual Studio Code. Also see [debugging for C++ in VSCode](https://code.visualstudio.com/docs/languages/cpp#_debugging) for further reference. diff --git a/misc/arduinoValidator.json b/misc/arduinoValidator.json index 73baff4f..492cfdf0 100644 --- a/misc/arduinoValidator.json +++ b/misc/arduinoValidator.json @@ -32,6 +32,16 @@ "description": "Arduino Debugger Settings", "type": "string", "minLength": 1 - } + }, + "disableIntelliSenseAutoGen": { + "description": "Disable/enable the automatic generation of the IntelliSense configuration file (c_cpp_properties.json) for this project (overrides the global setting). When set to \"global\" the global extension settings will be used.", + "type": "string", + "default": "global", + "enum": [ + "global", + "disable", + "enable" + ] + } } } \ No newline at end of file diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index 34cac628..5f23b029 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -772,7 +772,12 @@ Please make sure the folder is not occupied by other procedures .`); */ private makeCompilerParserContext(dc: DeviceContext) : { callback: (s: string) => void; conclude: () => void; } { - if (!VscodeSettings.getInstance().disableIntelliSenseAutoGen) { + + const globalDisable = VscodeSettings.getInstance().disableIntelliSenseAutoGen; + const project = dc.disableIntelliSenseAutoGen; + + if (project !== "disable" && !globalDisable || + project === "enable") { // setup the parser with its engines const gccParserEngine = new ccp.ParserGcc(dc.sketch); diff --git a/src/deviceContext.ts b/src/deviceContext.ts index 2e7f18e3..ec7128db 100644 --- a/src/deviceContext.ts +++ b/src/deviceContext.ts @@ -57,6 +57,11 @@ export interface IDeviceContext { */ configuration: string; + /** + * IntelliSense configuration auto-generation project override. + */ + disableIntelliSenseAutoGen: string; + onDidChange: vscode.Event; initialize(): void; @@ -82,6 +87,8 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { private _debugger: string; + private _disableIntelliSenseAutoGen: string; + private _configuration: string; private _extensionPath: string; @@ -151,6 +158,7 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { this._configuration = deviceConfigJson.configuration; this._output = deviceConfigJson.output; this._debugger = deviceConfigJson["debugger"]; + this._disableIntelliSenseAutoGen = deviceConfigJson.disableIntelliSenseAutoGen; this._onDidChange.fire(); this._prebuild = deviceConfigJson.prebuild; this._programmer = deviceConfigJson.programmer; @@ -164,6 +172,7 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { this._configuration = null; this._output = null; this._debugger = null; + this._disableIntelliSenseAutoGen = null; this._onDidChange.fire(); this._prebuild = null; this._programmer = null; @@ -182,6 +191,7 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { this._configuration = null; this._output = null; this._debugger = null; + this._disableIntelliSenseAutoGen = null; this._onDidChange.fire(); this._prebuild = null; this._programmer = null; @@ -217,6 +227,7 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { deviceConfigJson.board = this.board; deviceConfigJson.output = this.output; deviceConfigJson["debugger"] = this.debugger_; + deviceConfigJson.disableIntelliSenseAutoGen = this.disableIntelliSenseAutoGen; deviceConfigJson.configuration = this.configuration; deviceConfigJson.programmer = this.programmer; @@ -282,6 +293,15 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { this.saveContext(); } + public get disableIntelliSenseAutoGen() { + return this._disableIntelliSenseAutoGen; + } + + public set disableIntelliSenseAutoGen(value: string) { + this._disableIntelliSenseAutoGen = value; + this.saveContext(); + } + public get configuration() { return this._configuration; } From 6f36565bf84b639ed2c6a0c7ed51b9273482f027 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 8 Feb 2020 16:29:08 +0100 Subject: [PATCH 022/100] Try to generate IntelliSense configuration even when the compilation fails --- BRANCHNOTES.md | 9 ++++++--- src/arduino/arduino.ts | 23 +++++++++++++---------- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 6f3b1ec4..9c75ae6f 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -48,16 +48,19 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | | | :heavy_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) (partially done) | -| | :white_check_mark: How to handle compilation failure? Only set if more comprehensive | +| | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet ## Motivation -I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives. The old and dysfunctional Arduino CDT extension for eclipse somehow stalled (even if it was promising), Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). +I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives: +* The old and dysfunctional Arduino CDT extension for eclipse somehow stalled (even if it was promising) +* Sloeber could be an option but the maintainer is disillusioned and the project is more or less dead. Furthermore Eclipse is pretty heavy and less accessible to beginners +* Platform IO IDE's license is very [restrictive](https://community.platformio.org/t/what-part-of-platformio-is-open-source-licenced/1447/2). -Then remains vscode-arduino. It seems that it isn't completely dead - but almost. Most of the core functionality seems to work (I used it a few days now). But the biggest show stopper is the bad IntelliSense support. +Then remains vscode-arduino. It seems that it isn't completely dead - but almost. Most of the core functionality seems to work (I used it a few days now). But the biggest show stopper is the bad IntelliSense support -- which I'll address here now. ## Beer Money :beers: You can chip in some beer money to keep me motivated - this is really appreciated. diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index 5f23b029..e021fde6 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -287,20 +287,17 @@ export class ArduinoApp { arduinoChannel.show(); - try { - const compilerParserContext = this.makeCompilerParserContext(dc); + let verifyResult: boolean; + const compilerParserContext = this.makeCompilerParserContext(dc); + try { await util.spawn(this._settings.commandPath, arduinoChannel.channel, args, {}, compilerParserContext.callback); - - if (compilerParserContext.conclude) { - compilerParserContext.conclude(); - } - arduinoChannel.end(`Finished verify sketch - ${dc.sketch}${os.EOL}`); - return true; + arduinoChannel.end(`Finished verifying sketch - ${dc.sketch}${os.EOL}`); + verifyResult = true; } catch (reason) { const msg = reason.code ? `Exit with code=${reason.code}${os.EOL}` : @@ -308,8 +305,14 @@ export class ArduinoApp { reason.message : JSON.stringify(reason); arduinoChannel.error(msg); - return false; + verifyResult = false; } + + if (compilerParserContext.conclude) { + compilerParserContext.conclude(); + } + + return verifyResult; } public tryToUpdateIncludePaths() { @@ -797,7 +800,7 @@ Please make sure the folder is not occupied by other procedures .`); conclude: _conclude, } } -// const donothing = () => void {}; + return { callback: undefined, conclude: undefined, From 28586dfa38fdb41a49c2c21c0e9f5e282a62fc9e Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sun, 9 Feb 2020 04:24:52 +0100 Subject: [PATCH 023/100] Incorporated the latest progress from cocopa development --- BRANCHNOTES.md | 2 +- package.json | 2 +- src/arduino/arduino.ts | 20 +++++++++++++++++--- 3 files changed, 19 insertions(+), 5 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 9c75ae6f..bfcc4106 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -19,7 +19,7 @@ Provide a configuration flag which allows the user to turn this feature off - th **2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) **2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. **2020-02-07** Wrote compiler command parser npm package [cocopa](https://www.npmjs.com/package/cocopa) and began writing a test framework for it. Added a global configuration switch which allows the IntelliSense configuration generation to be turned off. -**2020-02-08** Integrated `cocopa` into vscode-arduino. Added project configuration flag which can override the global flag in both ways (forced off, forced on). Made code tslint compliant. Began some documentation in [README.md](README.md) +**2020-02-08** Integrated `cocopa` into vscode-arduino. Added project configuration flag which can override the global flag in both ways (forced off, forced on). Made code tslint compliant. Began some documentation in [README.md](README.md). vscode-arduino now tries to generate an IntelliSense configuration even if compilation (verify) should fail. vscode-arduino now tries to generate a IntelliSense configuration even if Arduino's verify failed (if the main sketch compilation was invoked before anything failed) | | Tasks | |-----:|:--------| diff --git a/package.json b/package.json index e15d0306..2dd60eac 100644 --- a/package.json +++ b/package.json @@ -576,7 +576,7 @@ }, "dependencies": { "body-parser": "^1.16.1", - "cocopa": "0.0.3", + "cocopa": "0.0.4", "compare-versions": "^3.4.0", "eventemitter2": "^4.1.0", "express": "^4.14.1", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index e021fde6..affee4d0 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -782,9 +782,8 @@ Please make sure the folder is not occupied by other procedures .`); if (project !== "disable" && !globalDisable || project === "enable") { - // setup the parser with its engines - const gccParserEngine = new ccp.ParserGcc(dc.sketch); - const compilerParser = new ccp.Runner([gccParserEngine]); + const parserEngines = this.makeCompilerParserEngines(dc); + const compilerParser = new ccp.Runner(parserEngines); // set up the function to be called after parsing const _conclude = () => { @@ -807,6 +806,21 @@ Please make sure the folder is not occupied by other procedures .`); } }; + private makeCompilerParserEngines(dc: DeviceContext) { + const matchPattern = [ + // trigger parser when compiling the main sketch + ` ${path.basename(dc.sketch)}.cpp.o`, + ]; + const dontMatchPattern = [ + // make sure Arduino's not testing libraries + /-o\s\/dev\/null/, + ]; + + // setup the parser with its engines + const gccParserEngine = new ccp.ParserGcc(matchPattern, dontMatchPattern); + return [gccParserEngine]; + } + private getProgrammerString(): string { const selectProgrammer = this.programmerManager.currentProgrammer; if (!selectProgrammer) { From 31d11d18c16763509812c43a1dbb40ed1ffe742d Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sun, 9 Feb 2020 11:51:43 +0100 Subject: [PATCH 024/100] * Fixes the line splitting regex as outlined in https://github.com/microsoft/vscode-arduino/pull/771 * Removed a redundand condition --- src/arduino/board.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/arduino/board.ts b/src/arduino/board.ts index 39908022..88c1b6c8 100644 --- a/src/arduino/board.ts +++ b/src/arduino/board.ts @@ -8,7 +8,7 @@ export function parseBoardDescriptor(boardDescriptor: string, plat: IPlatform): const boardLineRegex = /([^\.]+)\.(\S+)=(.+)/; const result = new Map(); - const lines = boardDescriptor.split(/[\r|\r\n|\n]/); + const lines = boardDescriptor.split(/(?:\r|\r\n|\n)/); const menuMap = new Map(); lines.forEach((line) => { @@ -18,7 +18,7 @@ export function parseBoardDescriptor(boardDescriptor: string, plat: IPlatform): } const match = boardLineRegex.exec(line); - if (match && match.length > 3) { + if (match) { if (line.startsWith("menu.")) { menuMap.set(match[2], match[3]); return; From c6c0c50fa8e49c8a3fc424aa1aeea80e8bc9b7c0 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sun, 9 Feb 2020 13:23:48 +0100 Subject: [PATCH 025/100] * Fixed regression introduced with adaptions to latest version of cocopa * Made compile command regex match more stringent --- src/arduino/arduino.ts | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index affee4d0..c5d5da96 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -804,16 +804,25 @@ Please make sure the folder is not occupied by other procedures .`); callback: undefined, conclude: undefined, } - }; - + }; + /** + * + * @param dc + */ private makeCompilerParserEngines(dc: DeviceContext) { + + let sketch = path.basename(dc.sketch); + const dotcpp= sketch.endsWith(".ino") ? ".cpp" : ""; + sketch = `-o\\s+\\S*${ccp.regExEscape(sketch)}${dotcpp}\\.o`; + const matchPattern = [ // trigger parser when compiling the main sketch - ` ${path.basename(dc.sketch)}.cpp.o`, + RegExp(sketch), ]; + const dontMatchPattern = [ // make sure Arduino's not testing libraries - /-o\s\/dev\/null/, + /-o\s+\/dev\/null/, ]; // setup the parser with its engines From 984e2e41adc4bb0f553ac0c647f689ccb4e9f920 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sun, 9 Feb 2020 16:40:00 +0100 Subject: [PATCH 026/100] * Added more serial monitor ideas from [John Lonergan](https://github.com/microsoft/vscode-arduino/issues/463#issuecomment-583853833) * Added some ideas how to implement a better serial monitor --- BRANCHNOTES.md | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index bfcc4106..6d3063e7 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -44,15 +44,17 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: JSON input | | | :white_check_mark: JSON output | | | :white_check_mark: Configuration merging | +| | :white_check_mark: Test with cpp sketches | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense. (Partially done - documented in the [General Tasks](#General-Tasks) section | -| | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | +| | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch* c) *workbench initialized and no `c_cpp_properties.json` has been found*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | | | :heavy_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) (partially done) | | | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :white_check_mark: Finally: go through my code and look for TODOs | -`*` not committed to branch yet +`*` not committed to branch yet +`>` most of the actual parsing and configuration generation is part of [cocopa](https://github.com/elektronikworkshop/cocopa/) ([here](https://www.npmjs.com/package/cocopa)'s the npm package) ## Motivation I write a lot of code for Arduino, especially libraries. The Arduino IDE is not suited for more complex projects and I tried several alternatives: @@ -93,6 +95,25 @@ I will list every supporter here, thanks! * Logging for IntelliSense https://code.visualstudio.com/docs/cpp/enable-logging-cpp ## Future Work * Proper interactive serial terminal (this is the second major show stopper in my opinion) + * Command history option + * From https://github.com/microsoft/vscode-arduino/issues/463#issuecomment-583846263 and following: + * allow input on the serial monitor in a convenient way - ie just type and hit return, just like the Arduino IDE + * have the serial monitor window NOT keep turning off autoscroll (there is a separate ticket for this) + * have the option of the serial monitor and/or compile window auto clear each time the sketch is compiled + * Plus there is the annoying default where the compile runs in verbose mode and we have to manually edit config files to turn off the trace output + * Plus plus... Is there a way to automatically select the right serial port? + * Oh and one more. I want the serial output and perhaps compile windows to be undocked or at least I want them to sit to the right of my code window but they seem rigidly stuck at the bottom of the screen. + * And I would probably prioritize ease of use over better editing/intelligence. + * Being able to set baud rate within monitor + * Possible implementation hooks + * run node program in native terminal and connect it to extension + * https://github.com/serialport/node-serialport + * [General](https://serialport.io/docs/guide-about) + * [CLI](https://serialport.io/docs/guide-cli) + * [API](https://serialport.io/docs/guide-usage) + * write a [debugger extension](https://code.visualstudio.com/api/extension-guides/debugger-extension) with a [mock](https://github.com/Microsoft/vscode-mock-debug) which communicates with the serial + +Plus plus... Is there a way to automatically select the right serial port? * Lots of redundant code * e.g. "upload is a superset of "verify" * general lack of modularity - the above is the result From 859f298ef3187876cadb508337ae2196477a1679 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sun, 9 Feb 2020 18:25:01 +0100 Subject: [PATCH 027/100] Moved arduino specifics from cocopa to vscode-arduino --- package.json | 2 +- src/arduino/arduino.ts | 44 +++++++++++++++++++++++++++++------------- 2 files changed, 32 insertions(+), 14 deletions(-) diff --git a/package.json b/package.json index 2dd60eac..cacd0197 100644 --- a/package.json +++ b/package.json @@ -576,7 +576,7 @@ }, "dependencies": { "body-parser": "^1.16.1", - "cocopa": "0.0.4", + "cocopa": "0.0.5", "compare-versions": "^3.4.0", "eventemitter2": "^4.1.0", "express": "^4.14.1", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index c5d5da96..fd5abd7f 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -782,44 +782,62 @@ Please make sure the folder is not occupied by other procedures .`); if (project !== "disable" && !globalDisable || project === "enable") { - const parserEngines = this.makeCompilerParserEngines(dc); - const compilerParser = new ccp.Runner(parserEngines); + const engines = this.makeCompilerParserEngines(dc); + const runner = new ccp.Runner(engines); // set up the function to be called after parsing const _conclude = () => { - const cppPropsPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); - if (compilerParser.processResult(cppPropsPath)) { - arduinoChannel.info("IntelliSense configuration generated successfully."); - } else { + if (!runner.result) { arduinoChannel.warning("Failed to generate IntelliSense configuration."); + return; + } + const pPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); + // TODO: check what kind of result we've got: gcc or other architecture: + // and instantiate content accordingly (to be implemented within cocopa) + const content = new ccp.CCppPropertiesContentResult(runner.result, + "Arduino", + ccp.CCppPropertiesISMode.Gcc_X64, + ccp.CCppPropertiesCStandard.C11, + // as of 1.8.11 arduino is on C++11 + ccp.CCppPropertiesCppStandard.Cpp11); + const prop = new ccp.CCppProperties(); + prop.read(pPath); + prop.merge(content); + if (prop.write(pPath)) { + arduinoChannel.info("IntelliSense configuration updated."); + } else { + arduinoChannel.info("IntelliSense configuration already up to date."); } }; return { - callback: compilerParser.callback(), + callback: runner.callback(), conclude: _conclude, } } - return { callback: undefined, conclude: undefined, } - }; + }; /** - * - * @param dc + * + * @param dc */ private makeCompilerParserEngines(dc: DeviceContext) { let sketch = path.basename(dc.sketch); - const dotcpp= sketch.endsWith(".ino") ? ".cpp" : ""; + const dotcpp = sketch.endsWith(".ino") ? ".cpp" : ""; sketch = `-o\\s+\\S*${ccp.regExEscape(sketch)}${dotcpp}\\.o`; const matchPattern = [ + // make sure we're running g++ + /(?:^|-)g\+\+\s+/, + // make sure we're compiling + /\s+-c\s+/, // trigger parser when compiling the main sketch RegExp(sketch), ]; - + const dontMatchPattern = [ // make sure Arduino's not testing libraries /-o\s+\/dev\/null/, From 8b8808797f878c038f5ae1e230a348fc66b3f689 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Mon, 10 Feb 2020 02:26:03 +0100 Subject: [PATCH 028/100] * More unit testing within cocopa. * Implemented c_cpp_properties merging -> compiler analysis results are merged into existing configuration and will preserve configurations of different name than the vscode-studio default configuration name (currently "Arduino"). This opens up the possibility for users to write their own configurations without having to disable the autogeneration. * Implemented "write on change" - `c_cpp_properties.json` will only be written if a new configuration has been detected. --- BRANCHNOTES.md | 16 ++++++++-------- package.json | 2 +- src/arduino/arduino.ts | 2 +- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 6d3063e7..efa9bf5e 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -20,6 +20,8 @@ Provide a configuration flag which allows the user to turn this feature off - th **2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. **2020-02-07** Wrote compiler command parser npm package [cocopa](https://www.npmjs.com/package/cocopa) and began writing a test framework for it. Added a global configuration switch which allows the IntelliSense configuration generation to be turned off. **2020-02-08** Integrated `cocopa` into vscode-arduino. Added project configuration flag which can override the global flag in both ways (forced off, forced on). Made code tslint compliant. Began some documentation in [README.md](README.md). vscode-arduino now tries to generate an IntelliSense configuration even if compilation (verify) should fail. vscode-arduino now tries to generate a IntelliSense configuration even if Arduino's verify failed (if the main sketch compilation was invoked before anything failed) +**2020-02-09** Moved vscode-arduino specific from cocopa over (to keep cocopa as generic as possible). More unit testing within cocopa. Some research regarding future serial monitor implementation. Implemented c_cpp_properties merging -> compiler analysis results are merged into existing configuration and will preserve configurations of different name than the vscode-studio default configuration name (currently "Arduino"). This opens up the possibility for users to write their own configurations without having to disable the autogeneration. Implemented "write on change" - `c_cpp_properties.json` will only be written if a new configuration has been detected. Now loads of tests have to be written for cocopa. + | | Tasks | |-----:|:--------| @@ -32,18 +34,14 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :heavy_check_mark: Basic setting of parsing result | | | :heavy_check_mark: Basic file input | | | :heavy_check_mark: Basic file output | -| | :white_check_mark: Merging of parsing result and existing file content | -| | :white_check_mark: Handling inexistent files and folders | -| | :white_check_mark: Write configuration on change only | +| | :heavy_check_mark: Merging of parsing result and existing file content | +| | :heavy_check_mark: Handling inexistent files and folders | +| | :heavy_check_mark: Write configuration on change only | | | :white_check_mark: Option to backup old configurations? | | **Configuration flags** | :heavy_check_mark: Provide global disable flag for IntelliSense auto-config | | | :heavy_check_mark: Provide project specific override for the global flag - most users will likely use the default setup and disable auto-generation for very specific projects | | **Unit tests** | :heavy_check_mark: Basic parser (known boards, match/no match)| -| | :heavy_check_mark: Querying of compiler built-in includes (Note: to be changed to generic compiler such that Arduino is not necessary for unit testing) | -| | :white_check_mark: Throwing arbitrary data at parser engines | -| | :white_check_mark: JSON input | -| | :white_check_mark: JSON output | -| | :white_check_mark: Configuration merging | +| | :white_check_mark: All unit tests in cocopa | | | :white_check_mark: Test with cpp sketches | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense. (Partially done - documented in the [General Tasks](#General-Tasks) section | | | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch* c) *workbench initialized and no `c_cpp_properties.json` has been found*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | @@ -79,6 +77,8 @@ I will list every supporter here, thanks! 2020-02-05 Elektronik Workshop: 40 :beers: (10h coding) 2020-02-06 Elektronik Workshop: 36 :beers: (9h coding) 2020-02-07 Elektronik Workshop: 48 :beers: (12h coding) +2020-02-08 Elektronik Workshop: 52 :beers: (13h coding) +2020-02-09 Elektronik Workshop: 40 :beers: (10h coding) diff --git a/package.json b/package.json index cacd0197..9d735021 100644 --- a/package.json +++ b/package.json @@ -576,7 +576,7 @@ }, "dependencies": { "body-parser": "^1.16.1", - "cocopa": "0.0.5", + "cocopa": "0.0.6", "compare-versions": "^3.4.0", "eventemitter2": "^4.1.0", "express": "^4.14.1", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index fd5abd7f..be8d0e27 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -802,7 +802,7 @@ Please make sure the folder is not occupied by other procedures .`); ccp.CCppPropertiesCppStandard.Cpp11); const prop = new ccp.CCppProperties(); prop.read(pPath); - prop.merge(content); + prop.merge(content, ccp.CCppPropertiesMergeMode.ReplaceSameNames); if (prop.write(pPath)) { arduinoChannel.info("IntelliSense configuration updated."); } else { From 6df35f1cf9c45387c8a58fa97a1c006dfa712085 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Mon, 10 Feb 2020 02:33:24 +0100 Subject: [PATCH 029/100] Minor documentation rub --- BRANCHNOTES.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index efa9bf5e..e08f0681 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -97,11 +97,11 @@ I will list every supporter here, thanks! * Proper interactive serial terminal (this is the second major show stopper in my opinion) * Command history option * From https://github.com/microsoft/vscode-arduino/issues/463#issuecomment-583846263 and following: - * allow input on the serial monitor in a convenient way - ie just type and hit return, just like the Arduino IDE - * have the serial monitor window NOT keep turning off autoscroll (there is a separate ticket for this) - * have the option of the serial monitor and/or compile window auto clear each time the sketch is compiled - * Plus there is the annoying default where the compile runs in verbose mode and we have to manually edit config files to turn off the trace output - * Plus plus... Is there a way to automatically select the right serial port? + * Allow input on the serial monitor in a convenient way - ie just type and hit return, just like the Arduino IDE + * Have the serial monitor window NOT keep turning off autoscroll (there is a separate ticket for this) + * Have the option of the serial monitor and/or compile window auto clear each time the sketch is compiled + * There is the annoying default where the compile runs in verbose mode and we have to manually edit config files to turn off the trace output + * Is there a way to automatically select the right serial port? * Oh and one more. I want the serial output and perhaps compile windows to be undocked or at least I want them to sit to the right of my code window but they seem rigidly stuck at the bottom of the screen. * And I would probably prioritize ease of use over better editing/intelligence. * Being able to set baud rate within monitor @@ -112,8 +112,6 @@ I will list every supporter here, thanks! * [CLI](https://serialport.io/docs/guide-cli) * [API](https://serialport.io/docs/guide-usage) * write a [debugger extension](https://code.visualstudio.com/api/extension-guides/debugger-extension) with a [mock](https://github.com/Microsoft/vscode-mock-debug) which communicates with the serial - -Plus plus... Is there a way to automatically select the right serial port? * Lots of redundant code * e.g. "upload is a superset of "verify" * general lack of modularity - the above is the result From eb5781148134106722415de7286fa3b25c27c918 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Mon, 10 Feb 2020 05:17:19 +0100 Subject: [PATCH 030/100] Moved intellisense function to a separate file --- src/arduino/arduino.ts | 89 +--------------------------------- src/arduino/intellisense.ts | 97 +++++++++++++++++++++++++++++++++++++ 2 files changed, 99 insertions(+), 87 deletions(-) create mode 100644 src/arduino/intellisense.ts diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index be8d0e27..c371aa18 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -1,7 +1,6 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. -import * as ccp from "cocopa"; import * as fs from "fs"; import * as glob from "glob"; import * as os from "os"; @@ -24,6 +23,7 @@ import { ArduinoWorkspace } from "../common/workspace"; import { SerialMonitor } from "../serialmonitor/serialMonitor"; import { UsbDetector } from "../serialmonitor/usbDetector"; import { ProgrammerManager } from "./programmerManager"; +import { makeCompilerParserContext } from "./intellisense"; /** * Represent an Arduino application based on the official Arduino IDE. @@ -288,7 +288,7 @@ export class ArduinoApp { arduinoChannel.show(); let verifyResult: boolean; - const compilerParserContext = this.makeCompilerParserContext(dc); + const compilerParserContext = makeCompilerParserContext(dc); try { await util.spawn(this._settings.commandPath, @@ -763,91 +763,6 @@ Please make sure the folder is not occupied by other procedures .`); return true; } - /** - * Creates a context which is used for compiler command parsing - * during building (verify, upload, ...). - * - * This context makes sure that it can be used in those sections - * without having to check whether this feature is en- or disabled - * and keeps the calling context more readable. - * - * @param dc The device context of the caller. - */ - private makeCompilerParserContext(dc: DeviceContext) - : { callback: (s: string) => void; conclude: () => void; } { - - const globalDisable = VscodeSettings.getInstance().disableIntelliSenseAutoGen; - const project = dc.disableIntelliSenseAutoGen; - - if (project !== "disable" && !globalDisable || - project === "enable") { - - const engines = this.makeCompilerParserEngines(dc); - const runner = new ccp.Runner(engines); - - // set up the function to be called after parsing - const _conclude = () => { - if (!runner.result) { - arduinoChannel.warning("Failed to generate IntelliSense configuration."); - return; - } - const pPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); - // TODO: check what kind of result we've got: gcc or other architecture: - // and instantiate content accordingly (to be implemented within cocopa) - const content = new ccp.CCppPropertiesContentResult(runner.result, - "Arduino", - ccp.CCppPropertiesISMode.Gcc_X64, - ccp.CCppPropertiesCStandard.C11, - // as of 1.8.11 arduino is on C++11 - ccp.CCppPropertiesCppStandard.Cpp11); - const prop = new ccp.CCppProperties(); - prop.read(pPath); - prop.merge(content, ccp.CCppPropertiesMergeMode.ReplaceSameNames); - if (prop.write(pPath)) { - arduinoChannel.info("IntelliSense configuration updated."); - } else { - arduinoChannel.info("IntelliSense configuration already up to date."); - } - }; - return { - callback: runner.callback(), - conclude: _conclude, - } - } - return { - callback: undefined, - conclude: undefined, - } - }; - /** - * - * @param dc - */ - private makeCompilerParserEngines(dc: DeviceContext) { - - let sketch = path.basename(dc.sketch); - const dotcpp = sketch.endsWith(".ino") ? ".cpp" : ""; - sketch = `-o\\s+\\S*${ccp.regExEscape(sketch)}${dotcpp}\\.o`; - - const matchPattern = [ - // make sure we're running g++ - /(?:^|-)g\+\+\s+/, - // make sure we're compiling - /\s+-c\s+/, - // trigger parser when compiling the main sketch - RegExp(sketch), - ]; - - const dontMatchPattern = [ - // make sure Arduino's not testing libraries - /-o\s+\/dev\/null/, - ]; - - // setup the parser with its engines - const gccParserEngine = new ccp.ParserGcc(matchPattern, dontMatchPattern); - return [gccParserEngine]; - } - private getProgrammerString(): string { const selectProgrammer = this.programmerManager.currentProgrammer; if (!selectProgrammer) { diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts new file mode 100644 index 00000000..dd4059b2 --- /dev/null +++ b/src/arduino/intellisense.ts @@ -0,0 +1,97 @@ +import * as ccp from "cocopa"; +import * as path from "path"; +import * as constants from "../common/constants"; +import { DeviceContext } from "../deviceContext"; +import { VscodeSettings } from "./vscodeSettings"; +import { arduinoChannel } from "../common/outputChannel"; +import { ArduinoWorkspace } from "../common/workspace"; + +/** + * Creates a context which is used for compiler command parsing + * during building (verify, upload, ...). + * + * This context makes sure that it can be used in those sections + * without having to check whether this feature is en- or disabled + * and keeps the calling context more readable. + * + * @param dc The device context of the caller. + */ +export function makeCompilerParserContext(dc: DeviceContext) + : { callback: (s: string) => void; conclude: () => void; } { + + const globalDisable = VscodeSettings.getInstance().disableIntelliSenseAutoGen; + const project = dc.disableIntelliSenseAutoGen; + + if (project !== "disable" && !globalDisable || + project === "enable") { + + const engines = makeCompilerParserEngines(dc); + const runner = new ccp.Runner(engines); + + // set up the function to be called after parsing + const _conclude = () => { + if (!runner.result) { + arduinoChannel.warning("Failed to generate IntelliSense configuration."); + return; + } + const pPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); + // TODO: check what kind of result we've got: gcc or other architecture: + // and instantiate content accordingly (to be implemented within cocopa) + const content = new ccp.CCppPropertiesContentResult(runner.result, + "Arduino", + ccp.CCppPropertiesISMode.Gcc_X64, + ccp.CCppPropertiesCStandard.C11, + // as of 1.8.11 arduino is on C++11 + ccp.CCppPropertiesCppStandard.Cpp11); + try { + const prop = new ccp.CCppProperties(); + prop.read(pPath); + prop.merge(content, ccp.CCppPropertiesMergeMode.ReplaceSameNames); + if (prop.write(pPath)) { + arduinoChannel.info("IntelliSense configuration updated."); + } else { + arduinoChannel.info("IntelliSense configuration already up to date."); + } + } catch (e) { + console.log(e); + } + }; + return { + callback: runner.callback(), + conclude: _conclude, + } + } + return { + callback: undefined, + conclude: undefined, + } +}; + +/** + * + * @param dc + */ +function makeCompilerParserEngines(dc: DeviceContext) { + + let sketch = path.basename(dc.sketch); + const dotcpp = sketch.endsWith(".ino") ? ".cpp" : ""; + sketch = `-o\\s+\\S*${ccp.regExEscape(sketch)}${dotcpp}\\.o`; + + const matchPattern = [ + // make sure we're running g++ + /(?:^|-)g\+\+\s+/, + // make sure we're compiling + /\s+-c\s+/, + // trigger parser when compiling the main sketch + RegExp(sketch), + ]; + + const dontMatchPattern = [ + // make sure Arduino's not testing libraries + /-o\s+\/dev\/null/, + ]; + + // setup the parser with its engines + const gccParserEngine = new ccp.ParserGcc(matchPattern, dontMatchPattern); + return [gccParserEngine]; +} From d761c7776fcc49c68b7995931e91b8f905ae8723 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 12 Feb 2020 16:08:59 +0100 Subject: [PATCH 031/100] Updated packages to resolve some conflicts --- BRANCHNOTES.md | 3 +++ package.json | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index e08f0681..64d93283 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -49,6 +49,7 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: Document features in [README.md](README.md) (partially done) | | | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | +| | :white_check_mark: Parser only works when arduino is set to `verbose`, since this is the only way we get the compiler invocation command. This has to be fixed. | | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet @@ -79,6 +80,7 @@ I will list every supporter here, thanks! 2020-02-07 Elektronik Workshop: 48 :beers: (12h coding) 2020-02-08 Elektronik Workshop: 52 :beers: (13h coding) 2020-02-09 Elektronik Workshop: 40 :beers: (10h coding) +2020-02-10 Elektronik Workshop: 32 :beers: (8h coding) @@ -88,6 +90,7 @@ I will list every supporter here, thanks! * [Interactive regex debugger](https://regex101.com/) * [Git branch management](https://blog.scottlowe.org/2015/01/27/using-fork-branch-git-workflow/) * [Collapsible Markdown](https://gist.githubusercontent.com/joyrexus/16041f2426450e73f5df9391f7f7ae5f/raw/f774f242feff6bae4a5be7d6c71aa5df2e3fcb0e/README.md) +* [Arduino CLI manpage](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc) ## Issues Concerning this Project * https://github.com/Microsoft/vscode-cpptools/issues/1750 diff --git a/package.json b/package.json index 9d735021..0441c7a6 100644 --- a/package.json +++ b/package.json @@ -549,8 +549,9 @@ "devDependencies": { "@types/compare-versions": "^3.0.0", "@types/mocha": "^2.2.32", - "@types/node": "^6.0.40", + "@types/node": "^6.14.9", "@types/winreg": "^1.2.30", + "ajv": "^5.0.0", "del": "^2.2.2", "eslint": "^4.18.2", "eslint-config-standard": "^10.2.1", @@ -576,7 +577,7 @@ }, "dependencies": { "body-parser": "^1.16.1", - "cocopa": "0.0.6", + "cocopa": "^0.0.7", "compare-versions": "^3.4.0", "eventemitter2": "^4.1.0", "express": "^4.14.1", From 02cf92e1193f188f56cab531c092dce10d679be4 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 12 Feb 2020 17:00:26 +0100 Subject: [PATCH 032/100] Added instructions how to run development code --- BRANCHNOTES.md | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 64d93283..6ac99b2a 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -122,6 +122,63 @@ I will list every supporter here, thanks! * Possibility to jump to compilation errors from compiler output and highlight compiler errors ---- +## How to beta test cutting edge code from the repo + +*I wrote the follwing for @iFreilicht, if anyone has some additional findings please let me know to keep this documentation up to date* + +that's the reason I embarked on this project. Great if you'd like to help out - this comes in really handy. Getting the stuff rock solid is one of my central objectives and testing is one cornerstones here. + +Currently my work is at the following state: +* Parsing works +* Enable/disable via global or project flags +* You can have personal configurations which aren't overwritten + +But I'm not fully there yet. Things which are yet to be done are: + +* Move the parser/configuration-generator out of the "verify" code, since it requires the build to be run with `--verbose` flag enabled which most probably few like (except for me :) I always switch on all warnings and set it to verbose). + This is something I can implement pretty fast +* I have not removed the original "IntelliSense configurator" which ruins the `c_cpp_properties.json` every now and then but that's one of my next tasks as well. And you can simply delete the file as you can re-generate the perfect version with my analyzer/generator by simply verifying your sketch +* My development system runs on Ubuntu-GNU/linux and I haven't done any work for Windows yet but that's one of the next steps. The chances that it works on OSX "out of the box" are pretty good + +To run the development version clone my [repository](https://github.com/elektronikworkshop/vscode-arduino) and checkout the `intellisense-autoconfig` branch + +The following steps requires you to have `git`, `vscode`, `npm` and `nodejs` at recent versions. On my Ubuntu system I don't use the versions supplied by my package manager (`apt`) as they are usually pretty outdated - I usually install `nodejs` and `npm` from their respective/official websites. If you're on Windows you'll have to be a bit more patient since I haven't set up a virtual machine yet for testing and Windows is definitely not my domain. But I'll document the setup process as soon as I'll get to it. + +```bash +git clone https://github.com/elektronikworkshop/vscode-arduino +cd vscode-arduino +# switch to the feature branch (not necessary probably because this branch is set to default) +git checkout intellisense-autoconfig +# check if you're on the right branch +git status +# install module dependencies +npm install +# install gulp builder globally to make it available to the path (requires relaunching your shell) +npm install -g gulp +# to make sure that gulp is actually working type +gulp --tasks +# if not -> configure your $PATH and open a new terminal to make sure it's added, then +# open vscode +code . +``` +Making sure that gulp is on your `$PATH` is essential. As long this isn't the case the following steps must not be carried out. + +Then hit F5 to debug or select it from the *Debug* menu. vscode will then complain that there's *No task defined* and you let it generate the configuration for you by clicking the button *Configure Task*. After configuring the tasks debug (`F5`) or build (`Ctrl + Shift + B`) should work. + +As soon as you've got it up and running (`F5` spawns a new window), just navigate to your Arduino project. Configure in the vscode-arduino global settings the build output to `verbose` and run verify (`Ctrl + Alt + R`) as you know it. This will then generate a fresh `c_cpp_properties.json`. As long as I haven't removed the generator from the current maintainers you'll have to regenerate it as soon as you see those double asterisk-paths like `whatever/path/**` - if I get to it today, I'll give it a try and will remove/disable it for testing. You can then pull my changes in by running +```bash +git pull +``` +from within your terminal inside vscode (and your `vscode-arduino` folder). + +Different IntelliSense configurations can be selected in vscode in the lower right. The configuration my branch generates is called *Arduino* as depicted here: + +![74001156-cfce8280-496a-11ea-9b9d-7d30c83765c1](https://user-images.githubusercontent.com/21954933/74351237-2696ea80-4db7-11ea-9f7a-1bfc652ad5f5.png) + +Sometimes IntelliSense has problems within the extension host (which you're running when launching it with `F5`) and fails to resolve some header paths. This is a problem of vscode and not the extension. Then just restart. The extension host is a bit buggy if you ask me, so I hope I can provide some development snapshots in the future which will render all the steps above superfluous and let you run the latest development version with your regular vscode. + +---- + ## Implementation **Note** Check this vscode feature: ``` From b85bd94f2f213b22a48734615a9c4a91996c9298 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 12 Feb 2020 17:53:28 +0100 Subject: [PATCH 033/100] More on beta testing, added link to chat room --- BRANCHNOTES.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 6ac99b2a..9106a9c9 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -124,8 +124,17 @@ I will list every supporter here, thanks! ## How to beta test cutting edge code from the repo +For development and testing of this fork/branch I've set up a dedicated chat room: + +[![Gitter](https://badges.gitter.im/vscode-arduino-ew-fork/community.svg)](https://gitter.im/vscode-arduino-ew-fork/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) + +**Note:** the topic is strictly development and testing of this feature enhancement branch. It's not a place where you can ask for help on your Arduino problems. There are dedicated forums for that and I'll delete any question of this type. + + *I wrote the follwing for @iFreilicht, if anyone has some additional findings please let me know to keep this documentation up to date* + > I'd love to try this out and give feedback. It would be a huge quality of life improvement if this was to work. Could you point me to a resource that helps me install your version of the extension? + that's the reason I embarked on this project. Great if you'd like to help out - this comes in really handy. Getting the stuff rock solid is one of my central objectives and testing is one cornerstones here. Currently my work is at the following state: From 6bb82440edd9176cc527cd6a6f72041ae854d04f Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 12 Feb 2020 18:12:43 +0100 Subject: [PATCH 034/100] Note how to run vscode without having gulp on your user path --- BRANCHNOTES.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 9106a9c9..5d1a1922 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -163,6 +163,7 @@ git status # install module dependencies npm install # install gulp builder globally to make it available to the path (requires relaunching your shell) +# there's another option below, see ./launchcode.sh npm install -g gulp # to make sure that gulp is actually working type gulp --tasks @@ -172,7 +173,18 @@ code . ``` Making sure that gulp is on your `$PATH` is essential. As long this isn't the case the following steps must not be carried out. -Then hit F5 to debug or select it from the *Debug* menu. vscode will then complain that there's *No task defined* and you let it generate the configuration for you by clicking the button *Configure Task*. After configuring the tasks debug (`F5`) or build (`Ctrl + Shift + B`) should work. +Another option to launch code with gulp on your path is (within bash or similar) +```bash +# create launch script +echo "PATH=./node_modules/.bin:$PATH code ." > launchcode +# make it executable +chmod +x launchcode +# now you can launch vscode like this +./launchcode +``` +This way you don't have to install gulp globally anymore (no `npm install -g gulp`). The path to the vscode dependency module binary is set as temporary environment variable when launching vscode. + +When everything's fine and vscode running, hit F5 to debug or select it from the *Debug* menu. vscode will then complain that there's *No task defined* and you let it generate the configuration for you by clicking the button *Configure Task*. After configuring the tasks debug (`F5`) or build (`Ctrl + Shift + B`) should work. As soon as you've got it up and running (`F5` spawns a new window), just navigate to your Arduino project. Configure in the vscode-arduino global settings the build output to `verbose` and run verify (`Ctrl + Alt + R`) as you know it. This will then generate a fresh `c_cpp_properties.json`. As long as I haven't removed the generator from the current maintainers you'll have to regenerate it as soon as you see those double asterisk-paths like `whatever/path/**` - if I get to it today, I'll give it a try and will remove/disable it for testing. You can then pull my changes in by running ```bash From 6215d79a6b99d3cf7f255abc64abcdc22fb8edfe Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 15 Feb 2020 12:47:31 +0100 Subject: [PATCH 035/100] Updated beer money log --- BRANCHNOTES.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 5d1a1922..b89c9314 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -81,6 +81,9 @@ I will list every supporter here, thanks! 2020-02-08 Elektronik Workshop: 52 :beers: (13h coding) 2020-02-09 Elektronik Workshop: 40 :beers: (10h coding) 2020-02-10 Elektronik Workshop: 32 :beers: (8h coding) +2020-02-11 Elektronik Workshop: 16 :beers: (4h coding) +2020-02-12 Elektronik Workshop: 32 :beers: (8h coding) +2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) @@ -91,6 +94,8 @@ I will list every supporter here, thanks! * [Git branch management](https://blog.scottlowe.org/2015/01/27/using-fork-branch-git-workflow/) * [Collapsible Markdown](https://gist.githubusercontent.com/joyrexus/16041f2426450e73f5df9391f7f7ae5f/raw/f774f242feff6bae4a5be7d6c71aa5df2e3fcb0e/README.md) * [Arduino CLI manpage](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc) +* [Install extensions from file](https://vscode-docs.readthedocs.io/en/stable/extensions/install-extension/) +* [Publish extensions](https://code.visualstudio.com/api/working-with-extensions/publishing-extension) ## Issues Concerning this Project * https://github.com/Microsoft/vscode-cpptools/issues/1750 From b61f7fc458f0b61be3b9845928bac95c6313b090 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 15 Feb 2020 20:49:13 +0100 Subject: [PATCH 036/100] Updated branch notes -- documented changes not committed though. --- BRANCHNOTES.md | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index b89c9314..8b69ca76 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -14,15 +14,28 @@ The generator takes the parser's output and transforms it into a valid `c_cpp_pr Provide a configuration flag which allows the user to turn this feature off - this is useful for the cases in which this magic fails or the user has a very specific setup. Although this branch tries to eliminate most of the latter cases. ### Global Tasks in vscode-arduino +See table below. + +### Branch Log +**2020 02 05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) +**2020 02 06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. +**2020 02 07** Wrote compiler command parser npm package [cocopa](https://www.npmjs.com/package/cocopa) and began writing a test framework for it. Added a global configuration switch which allows the IntelliSense configuration generation to be turned off. +**2020 02 08** Integrated `cocopa` into vscode-arduino. Added project configuration flag which can override the global flag in both ways (forced off, forced on). Made code tslint compliant. Began some documentation in [README.md](README.md). vscode-arduino now tries to generate an IntelliSense configuration even if compilation (verify) should fail. vscode-arduino now tries to generate a IntelliSense configuration even if Arduino's verify failed (if the main sketch compilation was invoked before anything failed) +**2020 02 09** Moved vscode-arduino specific from cocopa over (to keep cocopa as generic as possible). More unit testing within cocopa. Some research regarding future serial monitor implementation. Implemented c_cpp_properties merging -> compiler analysis results are merged into existing configuration and will preserve configurations of different name than the vscode-studio default configuration name (currently "Arduino"). This opens up the possibility for users to write their own configurations without having to disable the autogeneration. Implemented "write on change" - `c_cpp_properties.json` will only be written if a new configuration has been detected. Now loads of tests have to be written for cocopa. +**2020 02 10-12** Worked primarily on cocopa and test cases, fixed some npm and build errors on vscode-arduino within my setup. +**2020 02 15** Merged `upload` `uploadUsingProgrammer` and `verify` into a single function since they shared mostly the same code +* Better readability +* Better maintainability +* Less code redundancy -> less code -> less bugs +* Keeps the calls to the Arduino build CLI at a single location + +During merging I found some bugs within those functions - mainly due to the above problem. The most notable were: +* The serial monitor state wasn't restored when something went wrong +* In one of the `upload` functions the original authors forgot to invoke the "pre build command" +* Error message formatting was fixed within `verify` only +* No consistent return values within `verify` (when it bailed out early it returned `void`) ### Status -**2020-02-05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) -**2020-02-06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. -**2020-02-07** Wrote compiler command parser npm package [cocopa](https://www.npmjs.com/package/cocopa) and began writing a test framework for it. Added a global configuration switch which allows the IntelliSense configuration generation to be turned off. -**2020-02-08** Integrated `cocopa` into vscode-arduino. Added project configuration flag which can override the global flag in both ways (forced off, forced on). Made code tslint compliant. Began some documentation in [README.md](README.md). vscode-arduino now tries to generate an IntelliSense configuration even if compilation (verify) should fail. vscode-arduino now tries to generate a IntelliSense configuration even if Arduino's verify failed (if the main sketch compilation was invoked before anything failed) -**2020-02-09** Moved vscode-arduino specific from cocopa over (to keep cocopa as generic as possible). More unit testing within cocopa. Some research regarding future serial monitor implementation. Implemented c_cpp_properties merging -> compiler analysis results are merged into existing configuration and will preserve configurations of different name than the vscode-studio default configuration name (currently "Arduino"). This opens up the possibility for users to write their own configurations without having to disable the autogeneration. Implemented "write on change" - `c_cpp_properties.json` will only be written if a new configuration has been detected. Now loads of tests have to be written for cocopa. - - | | Tasks | |-----:|:--------| | **Build output parser** | :heavy_check_mark: Basic parser working | @@ -44,12 +57,13 @@ Provide a configuration flag which allows the user to turn this feature off - th | | :white_check_mark: All unit tests in cocopa | | | :white_check_mark: Test with cpp sketches | | **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense. (Partially done - documented in the [General Tasks](#General-Tasks) section | -| | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch* c) *workbench initialized and no `c_cpp_properties.json` has been found*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *verify*? -> Good moment would be after the workbench initialization -> message in arduino channel | +| | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch* c) *workbench initialized and no `c_cpp_properties.json` has been found*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *Arduino: Rebuild IntelliSense Configuration*? -> Good moment would be after the workbench initialization -> message in arduino channel | | | :heavy_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) (partially done) | | | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | -| | :white_check_mark: Parser only works when arduino is set to `verbose`, since this is the only way we get the compiler invocation command. This has to be fixed. | +| | :heavy_check_mark: Parser only works when arduino is set to `verbose`, since this is the only way we get the compiler invocation command - this has to be fixed (done, see next item) | +| | :heavy_check_mark: Implement a *Rebuild IntelliSense Configuration* command which runs verify verbosely internally and therefore allows us to find and parse the compiler command | | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet @@ -84,6 +98,7 @@ I will list every supporter here, thanks! 2020-02-11 Elektronik Workshop: 16 :beers: (4h coding) 2020-02-12 Elektronik Workshop: 32 :beers: (8h coding) 2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) +2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) From 8e617f2c99f4fcca88b2e7190a481f5043a91eee Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Mon, 17 Feb 2020 13:08:57 +0100 Subject: [PATCH 037/100] Reworked arduino build interface by merging "upload", "upload using programmer" and "verify" into a single function and fixed some bugs I encountered en route: * verify: even if it should return true or false it returns a value only for the last returns an not if it bails out early * verify and all upload implementations: no restoration of serial monitor if they bail out on error * pre build command has been forgotten in "upload using programmer" * no proper error messaging if "upload"/"upload using programmer" spawn failed * Removed compiler parser code from "verify" to a dedicated "analyze" build since the parsing requires the build to be run verbosely * Added new IDE command to run the analysis and generate the IntelliSense configuration from it * Added command to test bench * Fixed some linting complaints --- README.md | 9 + package.json | 10 + src/arduino/arduino.ts | 285 ++++++++++++----------------- src/arduino/intellisense.ts | 108 ++++++----- src/common/util.ts | 9 +- src/debug/configurationProvider.ts | 6 +- src/extension.ts | 29 ++- test/extension.test.ts | 1 + 8 files changed, 231 insertions(+), 226 deletions(-) diff --git a/README.md b/README.md index df1f9188..1e071336 100644 --- a/README.md +++ b/README.md @@ -50,10 +50,12 @@ This extension provides several commands in the Command Palette (F1 o - **Arduino: Upload**: Build sketch and upload to Arduino board. - **Arduino: Upload Using Programmer**: Upload using an external programmer. - **Arduino: Verify**: Build sketch. +- **Arduino: Rebuild IntelliSense Configuration**: Forced/manual rebuild of the IntelliSense configuration. The extension analyzes Arduino's build output and sets the Intellisense include paths, defines, compiler arguments accordingly. ## Keybindings - **Arduino: Upload** Alt + Cmd + U *or* Alt + Ctrl + U - **Arduino: Verify** Alt + Cmd + R *or* Alt + Ctrl + R +- **Arduino: Rebuild IntelliSense Configuration** Alt + Cmd + I *or* Alt + Ctrl + I ## Options | Option | Description | @@ -115,9 +117,16 @@ The following settings are as per sketch settings of the Arduino extension. You - `"enable"`: Enable the auto-generation even if globally disabled ## IntelliSense +*TODO: Rewrite this section* vscode-arduino auto-configures IntelliSense by default. vscode-arduino analyzes Arduino's compiler output during verify and generates the corresponding configuration file at `.vscode/c_cpp_properties.json` and tries as hard as possible to keep things up to date, e.g. running verify when switching the board or the sketch. It doesn't makes sense though to run verify repeatedly. Therefore if the workspace reports problems (for instance after adding new includes to a new library) run *verify* such that IntelliSense knows of the new include directories (since the Arduino-backend performs the library resolution externally). +TODO: Note about configuration selection in lower right. + +Manual rebuild: **Ardino: Rebuild IntelliSense Configuration**, +Keybindings: **Arduino: Rebuild IntelliSense Configuration** Alt + Cmd + I *or* Alt + Ctrl + I + + ## Debugging Arduino Code preview Before you start to debug your Arduino code, please read [this document](https://code.visualstudio.com/docs/editor/debugging) to learn about the basic mechanisms of debugging in Visual Studio Code. Also see [debugging for C++ in VSCode](https://code.visualstudio.com/docs/languages/cpp#_debugging) for further reference. diff --git a/package.json b/package.json index 0441c7a6..37f5683b 100644 --- a/package.json +++ b/package.json @@ -37,6 +37,7 @@ "onCommand:arduino.verify", "onCommand:arduino.upload", "onCommand:arduino.uploadUsingProgrammer", + "onCommand:arduino.rebuildIntelliSenseConfig", "onCommand:arduino.selectProgrammer", "onCommand:arduino.selectSerialPort", "onCommand:arduino.changeBaudRate", @@ -89,6 +90,10 @@ "command": "arduino.uploadUsingProgrammer", "title": "Arduino: Upload Using Programmer" }, + { + "command": "arduino.rebuildIntelliSenseConfig", + "title": "Arduino: Rebuild IntelliSense Configuration" + }, { "command": "arduino.selectProgrammer", "title": "Arduino: Select Programmer" @@ -434,6 +439,11 @@ "command": "arduino.upload", "key": "ctrl+alt+u", "mac": "cmd+alt+u" + }, + { + "command": "arduino.rebuildIntelliSenseConfig", + "key": "ctrl+alt+i", + "mac": "cmd+alt+i" } ], "configuration": { diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index c371aa18..c46bbc5a 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -9,21 +9,28 @@ import * as vscode from "vscode"; import * as constants from "../common/constants"; import * as util from "../common/util"; -import * as Logger from "../logger/logger"; +import * as logger from "../logger/logger"; import { DeviceContext } from "../deviceContext"; import { IArduinoSettings } from "./arduinoSettings"; import { BoardManager } from "./boardManager"; import { ExampleManager } from "./exampleManager"; +import { ICoCoPaContext, isCompilerParserEnabled, makeCompilerParserContext } from "./intellisense"; import { LibraryManager } from "./libraryManager"; +import { ProgrammerManager } from "./programmerManager"; import { VscodeSettings } from "./vscodeSettings"; import { arduinoChannel } from "../common/outputChannel"; import { ArduinoWorkspace } from "../common/workspace"; import { SerialMonitor } from "../serialmonitor/serialMonitor"; import { UsbDetector } from "../serialmonitor/usbDetector"; -import { ProgrammerManager } from "./programmerManager"; -import { makeCompilerParserContext } from "./intellisense"; + +export enum BuildMode { + Verify = "Verifying", + Analyze = "Analyzing", + Upload = "Uploading", + UploadProgrammer = "Uploading (programmer)", +}; /** * Represent an Arduino application based on the official Arduino IDE. @@ -94,225 +101,167 @@ export class ArduinoApp { } } - public async upload() { + /** + * Runs the arduino builder to build/compile and - if necessary - upload + * the current sketch. + * @param mode Build mode. BuildMode.Upload: Compile and upload, + * BuildMode.UploadProgrammer: Compile and upload using the user selectable + * programmer, BuildMode.Analyze: Compile, analyze the output and generate + * IntelliSense configuration from it, BuildMode.Verify: Just compile. + * @param buildDir Override the build directory set by the project settings + * with the given directory. + */ + public async build(mode: BuildMode, buildDir?: string): Promise { + const dc = DeviceContext.getInstance(); + const args: string[] = []; + let restoreSerialMonitor: boolean = false; + let cocopa: ICoCoPaContext; + const boardDescriptor = this.getBoardBuildString(); if (!boardDescriptor) { - return; + return false; } + args.push("--board", boardDescriptor); if (!ArduinoWorkspace.rootPath) { vscode.window.showWarningMessage("Cannot find the sketch file."); - return; + return false; } if (!dc.sketch || !util.fileExistsSync(path.join(ArduinoWorkspace.rootPath, dc.sketch))) { await this.getMainSketch(dc); } - if ((!dc.configuration || !/upload_method=[^=,]*st[^,]*link/i.test(dc.configuration)) && !dc.port) { + const selectSerial = async () => { const choice = await vscode.window.showInformationMessage( "Serial port is not specified. Do you want to select a serial port for uploading?", "Yes", "No"); if (choice === "Yes") { vscode.commands.executeCommand("arduino.selectSerialPort"); } - return; } - arduinoChannel.show(); - arduinoChannel.start(`Upload sketch - ${dc.sketch}`); - - const serialMonitor = SerialMonitor.getInstance(); - - const needRestore = await serialMonitor.closeSerialMonitor(dc.port); - UsbDetector.getInstance().pauseListening(); - await vscode.workspace.saveAll(false); - - if (!this.runPreBuildCommand(dc)) { - return; - } - - const appPath = path.join(ArduinoWorkspace.rootPath, dc.sketch); - const args = ["--upload", "--board", boardDescriptor]; - if (dc.port) { - args.push("--port", dc.port); - } - args.push(appPath); - if (VscodeSettings.getInstance().logLevel === "verbose") { - args.push("--verbose"); - } - if (dc.output) { - const outputPath = path.resolve(ArduinoWorkspace.rootPath, dc.output); - const dirPath = path.dirname(outputPath); - if (!util.directoryExistsSync(dirPath)) { - Logger.notifyUserError("InvalidOutPutPath", new Error(constants.messages.INVALID_OUTPUT_PATH + outputPath)); - return; + if (mode === BuildMode.Upload) { + if ((!dc.configuration || !/upload_method=[^=,]*st[^,]*link/i.test(dc.configuration)) && !dc.port) { + await selectSerial(); + return false; } - - args.push("--pref", `build.path=${outputPath}`); - arduinoChannel.info(`Please see the build logs in Output path: ${outputPath}`); - } else { - const msg = "Output path is not specified. Unable to reuse previously compiled files. Upload could be slow. See README."; - arduinoChannel.warning(msg); - } - await util.spawn(this._settings.commandPath, arduinoChannel.channel, args).then(async () => { - UsbDetector.getInstance().resumeListening(); - if (needRestore) { - await serialMonitor.openSerialMonitor(); + args.push("--upload"); + if (dc.port) { + args.push("--port", dc.port); } - arduinoChannel.end(`Uploaded the sketch: ${dc.sketch}${os.EOL}`); - }, (reason) => { - arduinoChannel.error(`Exit with code=${reason.code}${os.EOL}`); - }); - } - - public async uploadUsingProgrammer() { - const dc = DeviceContext.getInstance(); - const boardDescriptor = this.getBoardBuildString(); - if (!boardDescriptor) { - return; - } + } else if (mode === BuildMode.UploadProgrammer) { + const programmer = this.getProgrammerString(); + if (!programmer) { + return false; + } + if (!dc.port) { + await selectSerial(); + return false; + } + args.push("--upload", + "--port", dc.port, + "--useprogrammer", + "--pref", "programmer=" + programmer); - const selectProgrammer = this.getProgrammerString(); - if (!selectProgrammer) { - return; + } else if (mode === BuildMode.Analyze) { + if (!isCompilerParserEnabled()) { + return false; + } + cocopa = makeCompilerParserContext(dc); + args.push("--verify", "--verbose"); + } else { + args.push("--verify"); } - if (!ArduinoWorkspace.rootPath) { - vscode.window.showWarningMessage("Cannot find the sketch file."); - return; + const verbose = VscodeSettings.getInstance().logLevel === "verbose"; + if (mode !== BuildMode.Analyze && verbose) { + args.push("--verbose"); } - if (!dc.sketch || !util.fileExistsSync(path.join(ArduinoWorkspace.rootPath, dc.sketch))) { - await this.getMainSketch(dc); - } - if (!dc.port) { - const choice = await vscode.window.showInformationMessage( - "Serial port is not specified. Do you want to select a serial port for uploading?", - "Yes", "No"); - if (choice === "Yes") { - vscode.commands.executeCommand("arduino.selectSerialPort"); - } - return; - } + await vscode.workspace.saveAll(false); arduinoChannel.show(); - arduinoChannel.start(`Upload sketch - ${dc.sketch}`); + arduinoChannel.start(`${mode} sketch '${dc.sketch}'`); - const serialMonitor = SerialMonitor.getInstance(); - - const needRestore = await serialMonitor.closeSerialMonitor(dc.port); - UsbDetector.getInstance().pauseListening(); - await vscode.workspace.saveAll(false); - - if (!this.runPreBuildCommand(dc)) { - return; + if (!await this.runPreBuildCommand(dc)) { + return false; } - const appPath = path.join(ArduinoWorkspace.rootPath, dc.sketch); - const args = ["--upload", "--board", boardDescriptor, "--port", dc.port, "--useprogrammer", - "--pref", "programmer=" + selectProgrammer, appPath]; - if (VscodeSettings.getInstance().logLevel === "verbose") { - args.push("--verbose"); - } - if (dc.output) { - const outputPath = path.resolve(ArduinoWorkspace.rootPath, dc.output); + if (buildDir || dc.output) { + const outputPath = path.resolve(ArduinoWorkspace.rootPath, buildDir || dc.output); const dirPath = path.dirname(outputPath); if (!util.directoryExistsSync(dirPath)) { - Logger.notifyUserError("InvalidOutPutPath", new Error(constants.messages.INVALID_OUTPUT_PATH + outputPath)); + logger.notifyUserError("InvalidOutPutPath", new Error(constants.messages.INVALID_OUTPUT_PATH + outputPath)); return; } args.push("--pref", `build.path=${outputPath}`); - arduinoChannel.info(`Please see the build logs in Output path: ${outputPath}`); + arduinoChannel.info(`Please see the build logs in output path: ${outputPath}`); } else { - const msg = "Output path is not specified. Unable to reuse previously compiled files. Upload could be slow. See README."; + const msg = "Output path is not specified. Unable to reuse previously compiled files. Build will be slower. See README."; arduinoChannel.warning(msg); } - await util.spawn(this._settings.commandPath, arduinoChannel.channel, args).then(async () => { - UsbDetector.getInstance().resumeListening(); - if (needRestore) { - await serialMonitor.openSerialMonitor(); - } - arduinoChannel.end(`Uploaded the sketch: ${dc.sketch}${os.EOL}`); - }, (reason) => { - arduinoChannel.error(`Exit with code=${reason.code}${os.EOL}`); - }); - } - - public async verify(output: string = "") { - const dc = DeviceContext.getInstance(); - const boardDescriptor = this.getBoardBuildString(); - if (!boardDescriptor) { - return; - } - - if (!ArduinoWorkspace.rootPath) { - vscode.window.showWarningMessage("Cannot find the sketch file."); - return; - } - if (!dc.sketch || !util.fileExistsSync(path.join(ArduinoWorkspace.rootPath, dc.sketch))) { - await this.getMainSketch(dc); + // stop serial monitor when everything is prepared and good + // what makes restoring of its previous state easier + if (mode === BuildMode.Upload || mode === BuildMode.UploadProgrammer) { + restoreSerialMonitor = await SerialMonitor.getInstance().closeSerialMonitor(dc.port); + UsbDetector.getInstance().pauseListening(); } - await vscode.workspace.saveAll(false); + // Push sketch as last argument + args.push(path.join(ArduinoWorkspace.rootPath, dc.sketch)); - arduinoChannel.start(`Verify sketch - ${dc.sketch}`); + let success = false; - if (!this.runPreBuildCommand(dc)) { - return; + const cleanup = async () => { + if (cocopa) { + cocopa.conclude(); + } + if (mode === BuildMode.Upload || mode === BuildMode.UploadProgrammer) { + UsbDetector.getInstance().resumeListening(); + if (restoreSerialMonitor) { + await SerialMonitor.getInstance().openSerialMonitor(); + } + } } - const appPath = path.join(ArduinoWorkspace.rootPath, dc.sketch); - const args = ["--verify", "--board", boardDescriptor, appPath]; - if (VscodeSettings.getInstance().logLevel === "verbose") { - args.push("--verbose"); - } - if (output || dc.output) { - const outputPath = path.resolve(ArduinoWorkspace.rootPath, output || dc.output); - const dirPath = path.dirname(outputPath); - if (!util.directoryExistsSync(dirPath)) { - Logger.notifyUserError("InvalidOutPutPath", new Error(constants.messages.INVALID_OUTPUT_PATH + outputPath)); - return; + // TODO: Get rid of spawn's channel parameter and just support + // stdout and stderr callbacks + const stdoutCallback = (line: string) => { + if (cocopa) { + cocopa.callback(line); + if (verbose) { + arduinoChannel.channel.append(line); + } + } else { + arduinoChannel.channel.append(line); } - - args.push("--pref", `build.path=${outputPath}`); - arduinoChannel.info(`Please see the build logs in Output path: ${outputPath}`); - } else { - const msg = "Output path is not specified. Unable to reuse previously compiled files. Verify could be slow. See README."; - arduinoChannel.warning(msg); } - arduinoChannel.show(); - - let verifyResult: boolean; - const compilerParserContext = makeCompilerParserContext(dc); - - try { - await util.spawn(this._settings.commandPath, - arduinoChannel.channel, - args, - {}, - compilerParserContext.callback); - arduinoChannel.end(`Finished verifying sketch - ${dc.sketch}${os.EOL}`); - verifyResult = true; - } catch (reason) { + await util.spawn( + this._settings.commandPath, + arduinoChannel.channel, + args, + undefined, + stdoutCallback, + ).then(async () => { + await cleanup(); + arduinoChannel.end(`${mode} sketch '${dc.sketch}${os.EOL}`); + success = true; + }, async (reason) => { + await cleanup(); const msg = reason.code ? - `Exit with code=${reason.code}${os.EOL}` : + `Exit with code=${reason.code}` : reason.message ? reason.message : JSON.stringify(reason); - arduinoChannel.error(msg); - verifyResult = false; - } - - if (compilerParserContext.conclude) { - compilerParserContext.conclude(); - } + arduinoChannel.error(`${mode} sketch '${dc.sketch}': ${msg}${os.EOL}`); + }); - return verifyResult; + return success; } public tryToUpdateIncludePaths() { @@ -400,7 +349,7 @@ export class ArduinoApp { deviceContext = util.tryParseJSON(fs.readFileSync(configFilePath, "utf8")); } if (!deviceContext) { - Logger.notifyAndThrowUserError("arduinoFileError", new Error(constants.messages.ARDUINO_FILE_ERROR)); + logger.notifyAndThrowUserError("arduinoFileError", new Error(constants.messages.ARDUINO_FILE_ERROR)); } deviceContext.configurations = deviceContext.configurations || []; @@ -766,7 +715,7 @@ Please make sure the folder is not occupied by other procedures .`); private getProgrammerString(): string { const selectProgrammer = this.programmerManager.currentProgrammer; if (!selectProgrammer) { - Logger.notifyUserError("getProgrammerString", new Error(constants.messages.NO_PROGRAMMMER_SELECTED)); + logger.notifyUserError("getProgrammerString", new Error(constants.messages.NO_PROGRAMMMER_SELECTED)); return; } return selectProgrammer; @@ -775,7 +724,7 @@ Please make sure the folder is not occupied by other procedures .`); private getBoardBuildString(): string { const selectedBoard = this.boardManager.currentBoard; if (!selectedBoard) { - Logger.notifyUserError("getBoardBuildString", new Error(constants.messages.NO_BOARD_SELECTED)); + logger.notifyUserError("getBoardBuildString", new Error(constants.messages.NO_BOARD_SELECTED)); return; } return selectedBoard.getBuildConfig(); diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts index dd4059b2..e1b73b73 100644 --- a/src/arduino/intellisense.ts +++ b/src/arduino/intellisense.ts @@ -1,10 +1,32 @@ import * as ccp from "cocopa"; import * as path from "path"; + import * as constants from "../common/constants"; -import { DeviceContext } from "../deviceContext"; -import { VscodeSettings } from "./vscodeSettings"; import { arduinoChannel } from "../common/outputChannel"; import { ArduinoWorkspace } from "../common/workspace"; +import { DeviceContext } from "../deviceContext"; + +import { VscodeSettings } from "./vscodeSettings"; + +export interface ICoCoPaContext { + callback: (s: string) => void; + conclude: () => void; +}; + +/** + * Returns true if the combination of global enable/disable and project + * specific override enable the auto-generation of the IntelliSense + * configuration. + */ +export function isCompilerParserEnabled(dc?: DeviceContext) { + if (!dc) { + dc = DeviceContext.getInstance(); + } + const globalDisable = VscodeSettings.getInstance().disableIntelliSenseAutoGen; + const projectSetting = dc.disableIntelliSenseAutoGen; + return projectSetting !== "disable" && !globalDisable || + projectSetting === "enable"; +} /** * Creates a context which is used for compiler command parsing @@ -16,60 +38,54 @@ import { ArduinoWorkspace } from "../common/workspace"; * * @param dc The device context of the caller. */ -export function makeCompilerParserContext(dc: DeviceContext) - : { callback: (s: string) => void; conclude: () => void; } { - - const globalDisable = VscodeSettings.getInstance().disableIntelliSenseAutoGen; - const project = dc.disableIntelliSenseAutoGen; +export function makeCompilerParserContext(dc: DeviceContext): ICoCoPaContext { - if (project !== "disable" && !globalDisable || - project === "enable") { + const engines = makeCompilerParserEngines(dc); + const runner = new ccp.Runner(engines); - const engines = makeCompilerParserEngines(dc); - const runner = new ccp.Runner(engines); - - // set up the function to be called after parsing - const _conclude = () => { - if (!runner.result) { - arduinoChannel.warning("Failed to generate IntelliSense configuration."); - return; - } - const pPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); - // TODO: check what kind of result we've got: gcc or other architecture: - // and instantiate content accordingly (to be implemented within cocopa) - const content = new ccp.CCppPropertiesContentResult(runner.result, - "Arduino", - ccp.CCppPropertiesISMode.Gcc_X64, - ccp.CCppPropertiesCStandard.C11, - // as of 1.8.11 arduino is on C++11 - ccp.CCppPropertiesCppStandard.Cpp11); - try { - const prop = new ccp.CCppProperties(); - prop.read(pPath); - prop.merge(content, ccp.CCppPropertiesMergeMode.ReplaceSameNames); - if (prop.write(pPath)) { - arduinoChannel.info("IntelliSense configuration updated."); - } else { - arduinoChannel.info("IntelliSense configuration already up to date."); - } - } catch (e) { - console.log(e); + // set up the function to be called after parsing + const _conclude = () => { + if (!runner.result) { + arduinoChannel.warning("Failed to generate IntelliSense configuration."); + return; + } + const pPath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); + // TODO: check what kind of result we've got: gcc or other architecture: + // and instantiate content accordingly (to be implemented within cocopa) + const content = new ccp.CCppPropertiesContentResult(runner.result, + "Arduino", + ccp.CCppPropertiesISMode.Gcc_X64, + ccp.CCppPropertiesCStandard.C11, + // as of 1.8.11 arduino is on C++11 + ccp.CCppPropertiesCppStandard.Cpp11); + try { + const prop = new ccp.CCppProperties(); + prop.read(pPath); + prop.merge(content, ccp.CCppPropertiesMergeMode.ReplaceSameNames); + if (prop.write(pPath)) { + arduinoChannel.info("IntelliSense configuration updated."); + } else { + arduinoChannel.info("IntelliSense configuration already up to date."); } - }; - return { - callback: runner.callback(), - conclude: _conclude, + } catch (e) { + // TODO: some debug trace as this shouldn't happen } - } + }; return { - callback: undefined, - conclude: undefined, + callback: runner.callback(), + conclude: _conclude, } }; /** + * Assembles compiler parser engines which then will be used to find the main + * sketch's compile command and parse the infomation from it required for + * assembling an IntelliSense configuration from it. + * + * It could return multiple engines for different compilers or - if necessary - + * return specialized engines based on the current board architecture. * - * @param dc + * @param dc Current device context used to generate the engines. */ function makeCompilerParserEngines(dc: DeviceContext) { diff --git a/src/common/util.ts b/src/common/util.ts index ada3e462..5a0d2fd4 100644 --- a/src/common/util.ts +++ b/src/common/util.ts @@ -200,13 +200,13 @@ export function isArduinoFile(filePath): boolean { return fileExistsSync(filePath) && (path.extname(filePath) === ".ino" || path.extname(filePath) === ".pde"); } -type StdOutCallback = (msg: string) => void; - +// TODO: remove output channel and just provide callback hooks for stdout and +// stderr export function spawn(command: string, outputChannel: vscode.OutputChannel, args: string[] = [], options: any = {}, - stdoutCallback?: StdOutCallback): Thenable { + stdoutCallback?: (string) => void): Thenable { return new Promise((resolve, reject) => { const stdout = ""; const stderr = ""; @@ -228,9 +228,10 @@ export function spawn(command: string, if (outputChannel) { child.stdout.on("data", (data: Buffer) => { const decoded = decodeData(data, codepage); - outputChannel.append(decoded); if (stdoutCallback) { stdoutCallback(decoded); + } else { + outputChannel.append(decoded); } }); child.stderr.on("data", (data: Buffer) => { diff --git a/src/debug/configurationProvider.ts b/src/debug/configurationProvider.ts index c08546f2..6356ad52 100644 --- a/src/debug/configurationProvider.ts +++ b/src/debug/configurationProvider.ts @@ -4,7 +4,7 @@ import * as path from "path"; import * as vscode from "vscode"; -import { ArduinoApp } from "../arduino/arduino"; +import { ArduinoApp, BuildMode } from "../arduino/arduino"; import ArduinoActivator from "../arduinoActivator"; import ArduinoContext from "../arduinoContext"; @@ -136,8 +136,8 @@ export class ArduinoDebugConfigurationProvider implements vscode.DebugConfigurat config.program = path.join(ArduinoWorkspace.rootPath, outputFolder, `${path.basename(dc.sketch)}.elf`); // always compile elf to make sure debug the right elf - if (!await ArduinoContext.arduinoApp.verify(outputFolder)) { - vscode.window.showErrorMessage("Failure to verify the program, please check output for details."); + if (!await ArduinoContext.arduinoApp.build(BuildMode.Verify, outputFolder)) { + vscode.window.showErrorMessage("Failed to verify the program, please check the output for details."); return false; } diff --git a/src/extension.ts b/src/extension.ts index 57eae256..ad7c9753 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -22,6 +22,7 @@ import { validateArduinoPath } from "./common/platform"; import * as util from "./common/util"; import { ArduinoWorkspace } from "./common/workspace"; const arduinoDebugConfigurationProviderModule = impor("./debug/configurationProvider") as typeof import ("./debug/configurationProvider"); +import { BuildMode } from "./arduino/arduino"; import { DeviceContext } from "./deviceContext"; const completionProviderModule = impor("./langService/completionProvider") as typeof import ("./langService/completionProvider"); import * as Logger from "./logger/logger"; @@ -43,8 +44,8 @@ export async function activate(context: vscode.ExtensionContext) { const workingFile = path.normalize(openEditor.document.fileName); const workspaceFolder = (vscode.workspace && ArduinoWorkspace.rootPath) || ""; if (!workspaceFolder || workingFile.indexOf(path.normalize(workspaceFolder)) < 0) { - vscode.window.showWarningMessage(`The working file "${workingFile}" is not under the workspace folder, ` + - "the arduino extension might not work appropriately."); + vscode.window.showWarningMessage(`The open file "${workingFile}" is not inside the workspace folder, ` + + "the arduino extension might not work properly."); } } const vscodeSettings = VscodeSettings.getInstance(); @@ -121,7 +122,7 @@ export async function activate(context: vscode.ExtensionContext) { location: vscode.ProgressLocation.Window, title: "Arduino: Verifying...", }, async () => { - await arduinoContextModule.default.arduinoApp.verify(); + await arduinoContextModule.default.arduinoApp.build(BuildMode.Verify); }); } catch (ex) { } @@ -142,7 +143,7 @@ export async function activate(context: vscode.ExtensionContext) { location: vscode.ProgressLocation.Window, title: "Arduino: Uploading...", }, async () => { - await arduinoContextModule.default.arduinoApp.upload(); + await arduinoContextModule.default.arduinoApp.build(BuildMode.Upload); }); } catch (ex) { } @@ -176,8 +177,11 @@ export async function activate(context: vscode.ExtensionContext) { registerArduinoCommand("arduino.uploadUsingProgrammer", async () => { if (!status.compile) { status.compile = "upload"; + // TODO: no progress indicator + // and: exceptions should be handled within build + // function try { - await arduinoContextModule.default.arduinoApp.uploadUsingProgrammer(); + await arduinoContextModule.default.arduinoApp.build(BuildMode.UploadProgrammer); } catch (ex) { } delete status.compile; @@ -186,6 +190,21 @@ export async function activate(context: vscode.ExtensionContext) { return { board: arduinoContextModule.default.boardManager.currentBoard.name }; }); + registerArduinoCommand("arduino.rebuildIntelliSenseConfig", async () => { + if (!status.compile) { + status.compile = "intellisenserebuild"; + await vscode.window.withProgress({ + location: vscode.ProgressLocation.Window, + title: "Arduino: Rebuilding IS Configuration...", + }, async () => { + await arduinoContextModule.default.arduinoApp.build(BuildMode.Analyze); + }); + delete status.compile; + } + }, () => { + return { board: arduinoContextModule.default.boardManager.currentBoard.name }; + }); + registerArduinoCommand("arduino.selectProgrammer", async () => { if (!status.compile) { status.compile = "upload"; diff --git a/test/extension.test.ts b/test/extension.test.ts index b1ff1ee2..9f0f7f10 100644 --- a/test/extension.test.ts +++ b/test/extension.test.ts @@ -35,6 +35,7 @@ suite("Arduino: Extension Tests", () => { "arduino.verify", "arduino.upload", "arduino.uploadUsingProgrammer", + "arduino.rebuildIntelliSenseConfig", "arduino.selectProgrammer", "arduino.showBoardManager", "arduino.showLibraryManager", From 7ee5f9eb9c8856002591deab2dc311157d846c01 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Mon, 17 Feb 2020 16:38:12 +0100 Subject: [PATCH 038/100] Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE` --- BRANCHNOTES.md | 20 ++++++++++++++++++++ package.json | 1 - src/arduino/arduino.ts | 22 ++++++++++++++++++---- src/arduino/arduinoContentProvider.ts | 3 ++- src/arduino/boardManager.ts | 3 ++- src/common/util.ts | 1 + src/extension.ts | 6 ++++-- src/langService/completionProvider.ts | 23 ++++++++++++++++++++++- test/boardmanager.test.ts | 3 ++- test/commands.test.ts | 3 +++ test/extension.test.ts | 4 +++- 11 files changed, 77 insertions(+), 12 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 8b69ca76..e2dcbc55 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -35,6 +35,8 @@ During merging I found some bugs within those functions - mainly due to the abov * Error message formatting was fixed within `verify` only * No consistent return values within `verify` (when it bailed out early it returned `void`) +**2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal. + ### Status | | Tasks | |-----:|:--------| @@ -99,6 +101,7 @@ I will list every supporter here, thanks! 2020-02-12 Elektronik Workshop: 32 :beers: (8h coding) 2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) 2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) +2020-02-16 Elektronik Workshop: x :beers: (xh coding) @@ -116,6 +119,7 @@ I will list every supporter here, thanks! * https://github.com/Microsoft/vscode-cpptools/issues/1750 * Problems with IntelliSense itself https://github.com/microsoft/vscode-cpptools/issues/1034 * Logging for IntelliSense https://code.visualstudio.com/docs/cpp/enable-logging-cpp + ## Future Work * Proper interactive serial terminal (this is the second major show stopper in my opinion) * Command history option @@ -140,6 +144,22 @@ I will list every supporter here, thanks! * general lack of modularity - the above is the result * It seems that this extension is pretty chaotic. Most probably some refactoring is necessary. * Possibility to jump to compilation errors from compiler output and highlight compiler errors + +## Non-categorized Notes +### Integrate upstream changes into fork +```bash +git remote add upstream https://github.com/microsoft/vscode-arduino.git +git remote -v +git fetch upstream +# make sure your working directory is clean, then +git checkout master +git merge upstream/master +git push origin master +# to pull the changes into you feature branch: +git checkout intellisense-autoconfig +git merge master +git push origin intellisense-autoconfig +``` ---- ## How to beta test cutting edge code from the repo diff --git a/package.json b/package.json index 37f5683b..0b13da94 100644 --- a/package.json +++ b/package.json @@ -41,7 +41,6 @@ "onCommand:arduino.selectProgrammer", "onCommand:arduino.selectSerialPort", "onCommand:arduino.changeBaudRate", - "onCommand:arduino.addLibPath", "onCommand:arduino.openSerialMonitor", "onCommand:arduino.sendMessageToSerialPort", "onCommand:arduino.closeSerialMonitor", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index eb8e88ba..cb691720 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -264,6 +264,8 @@ export class ArduinoApp { return success; } + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + /* public tryToUpdateIncludePaths() { const configFilePath = path.join(ArduinoWorkspace.rootPath, constants.CPP_CONFIG_FILE); if (!fs.existsSync(configFilePath)) { @@ -292,7 +294,7 @@ export class ArduinoApp { configuration.forcedInclude.push(forcedIncludePath); } } - + */ // remove all unexisting paths // concern mistake removal, comment temporary // for (let pathIndex = 0; pathIndex < configuration.includePath.length; pathIndex++) { @@ -320,12 +322,15 @@ export class ArduinoApp { // pathIndex--; // } // } - + /* if (cppConfigFileUpdated) { fs.writeFileSync(configFilePath, JSON.stringify(cppConfig, null, 4)); } } + */ + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + /* // Add selected library path to the intellisense search path. public addLibPath(libraryPath: string) { let libPaths; @@ -396,7 +401,7 @@ export class ArduinoApp { fs.writeFileSync(configFilePath, JSON.stringify(deviceContext, null, 4)); } - + */ // Include the *.h header files from selected library to the arduino sketch. public async includeLibrary(libraryPath: string) { if (!ArduinoWorkspace.rootPath) { @@ -519,6 +524,8 @@ Please make sure the folder is not occupied by other procedures .`); arduinoChannel.end(`Removed library - ${libName}${os.EOL}`); } + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + /* public getDefaultPackageLibPaths(): string[] { const result = []; const boardDescriptor = this._boardManager.currentBoard; @@ -543,8 +550,10 @@ Please make sure the folder is not occupied by other procedures .`); result.push(path.normalize(path.join(toolPath, "**"))); } return result; - } + }*/ + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + /* public getDefaultForcedIncludeFiles(): string[] { const result = []; const boardDescriptor = this._boardManager.currentBoard; @@ -557,6 +566,7 @@ Please make sure the folder is not occupied by other procedures .`); } return result; } + */ public openExample(example) { function tmpName(name) { @@ -597,6 +607,7 @@ Please make sure the folder is not occupied by other procedures .`); const dc = DeviceContext.getInstance(); const arduinoJson = { sketch: sketchFile, + // TODO: COM1 is Windows specific - what about OSX and Linux users? port: dc.port || "COM1", board: dc.board, configuration: dc.configuration, @@ -605,6 +616,8 @@ Please make sure the folder is not occupied by other procedures .`); util.mkdirRecursivelySync(path.dirname(arduinoConfigFilePath)); fs.writeFileSync(arduinoConfigFilePath, JSON.stringify(arduinoJson, null, 4)); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + /* // Generate cpptools intellisense config const cppConfigFilePath = path.join(destExample, constants.CPP_CONFIG_FILE); @@ -641,6 +654,7 @@ Please make sure the folder is not occupied by other procedures .`); }; util.mkdirRecursivelySync(path.dirname(cppConfigFilePath)); fs.writeFileSync(cppConfigFilePath, JSON.stringify(cppConfig, null, 4)); + */ } // Step 3: Open the arduino project at a new vscode window. diff --git a/src/arduino/arduinoContentProvider.ts b/src/arduino/arduinoContentProvider.ts index 6c893eb2..3623c4e9 100644 --- a/src/arduino/arduinoContentProvider.ts +++ b/src/arduino/arduinoContentProvider.ts @@ -207,7 +207,8 @@ export class ArduinoContentProvider implements vscode.TextDocumentContentProvide return res.status(400).send("BAD Request! Missing { libraryPath } parameters!"); } else { try { - await ArduinoContext.arduinoApp.addLibPath(req.body.libraryPath); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + //await ArduinoContext.arduinoApp.addLibPath(req.body.libraryPath); await ArduinoContext.arduinoApp.includeLibrary(req.body.libraryPath); return res.json({ status: "OK", diff --git a/src/arduino/boardManager.ts b/src/arduino/boardManager.ts index d8267afa..1ed3000c 100644 --- a/src/arduino/boardManager.ts +++ b/src/arduino/boardManager.ts @@ -123,7 +123,8 @@ export class BoardManager { this._currentBoard = targetBoard; dc.configuration = this._currentBoard.customConfig; this._boardConfigStatusBar.text = targetBoard.name; - this._arduinoApp.addLibPath(null); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + //this._arduinoApp.addLibPath(null); this._onBoardTypeChanged.fire(); } diff --git a/src/common/util.ts b/src/common/util.ts index 5a0d2fd4..e601dd2a 100644 --- a/src/common/util.ts +++ b/src/common/util.ts @@ -13,6 +13,7 @@ import { arduinoChannel } from "./outputChannel"; const encodingMapping: object = JSON.parse(fs.readFileSync(path.join(__dirname, "../../../misc", "codepageMapping.json"), "utf8")); +// IS-REMOVE: to be removed completely when IntelliSense implementation is merged /** * This function will return the VSCode C/C++ extesnion compatible platform literals. * @function getCppConfigPlatform diff --git a/src/extension.ts b/src/extension.ts index ad7c9753..58d0efcc 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -221,7 +221,8 @@ export async function activate(context: vscode.ExtensionContext) { }; }); - registerArduinoCommand("arduino.addLibPath", (path) => arduinoContextModule.default.arduinoApp.addLibPath(path)); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + //registerArduinoCommand("arduino.addLibPath", (path) => arduinoContextModule.default.arduinoApp.addLibPath(path)); registerArduinoCommand("arduino.openExample", (path) => arduinoContextModule.default.arduinoApp.openExample(path)); registerArduinoCommand("arduino.loadPackages", async () => await arduinoContextModule.default.boardManager.loadPackages(true)); registerArduinoCommand("arduino.installBoard", async (packageName, arch, version: string = "") => { @@ -276,7 +277,8 @@ export async function activate(context: vscode.ExtensionContext) { SerialMonitor.getInstance().initialize(); } arduinoContextModule.default.boardManager.updateStatusBar(true); - arduinoContextModule.default.arduinoApp.tryToUpdateIncludePaths(); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + //arduinoContextModule.default.arduinoApp.tryToUpdateIncludePaths(); vscode.commands.executeCommand("setContext", "vscode-arduino:showExampleExplorer", true); })(); } diff --git a/src/langService/completionProvider.ts b/src/langService/completionProvider.ts index 12782819..91653c7f 100644 --- a/src/langService/completionProvider.ts +++ b/src/langService/completionProvider.ts @@ -12,6 +12,25 @@ import ArduinoActivator from "../arduinoActivator"; import ArduinoContext from "../arduinoContext"; import { ArduinoWorkspace } from "../common/workspace"; +/** + * Provides completions for library header includes. + * + * NOTE: With the new IntelliSense auto-configuration this doesn't make + * much sense in its current state, since it tries to fetch includes + * from the wrongly guessed include paths. And it tries to fetch includes + * from the c_cpp_properties which is now automatically generated from the + * files already included -> therefore the user already included the header + * and doesn't need a completion. Furthermore IntelliSense knows the location + * as well and can complete it too. + * + * To make this useful it has to parse the actual library folders and then + * it makes only sense if it reads the library information and checks if + * the individual libraries are actually compatible with the current board + * before offering a completion. + * + * EW + * 2020-02-17 + */ export class CompletionProvider implements vscode.CompletionItemProvider { private _headerFiles = new Set(); @@ -65,10 +84,12 @@ export class CompletionProvider implements vscode.CompletionItemProvider { } this._libPaths.clear(); this._headerFiles.clear(); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + /* ArduinoContext.arduinoApp.getDefaultPackageLibPaths().forEach((defaultPath) => { this._libPaths.add(defaultPath); }); - + */ if (fs.existsSync(this._cppConfigFile)) { const deviceConfig = util.tryParseJSON(fs.readFileSync(this._cppConfigFile, "utf8")); if (deviceConfig) { diff --git a/test/boardmanager.test.ts b/test/boardmanager.test.ts index b02526df..473628d5 100644 --- a/test/boardmanager.test.ts +++ b/test/boardmanager.test.ts @@ -28,7 +28,8 @@ suite("Arduino: Board Manager.", () => { const arduinoApp = TypeMoq.Mock.ofType(ArduinoApp); arduinoApp.setup((x) => x.setPref(TypeMoq.It.isAny(), TypeMoq.It.isAny())); arduinoApp.setup((x) => x.initialize(TypeMoq.It.isAny())); - arduinoApp.setup((x) => x.addLibPath(TypeMoq.It.isAny())); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + //arduinoApp.setup((x) => x.addLibPath(TypeMoq.It.isAny())); try { boardManager = new BoardManager(arduinoSettings.object, arduinoApp.object); diff --git a/test/commands.test.ts b/test/commands.test.ts index c188dbb5..124bd403 100644 --- a/test/commands.test.ts +++ b/test/commands.test.ts @@ -38,8 +38,10 @@ suite("Arduino: Commands Tests", () => { } }); + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged // Arduino: Library Manager: Add library to include path. // tslint:disable-next-line: only-arrow-functions + /* test("should be able to run command: arduino.addLibPath", function(done) { this.timeout(60 * 1000); try { @@ -52,6 +54,7 @@ suite("Arduino: Commands Tests", () => { done(new Error(error)); } }); + */ // Arduino: Boards Manager : Manage packages for boards // tslint:disable-next-line: only-arrow-functions diff --git a/test/extension.test.ts b/test/extension.test.ts index 9f0f7f10..b7944f9d 100644 --- a/test/extension.test.ts +++ b/test/extension.test.ts @@ -43,7 +43,9 @@ suite("Arduino: Extension Tests", () => { "arduino.showExamples", "arduino.changeBoardType", "arduino.initialize", - "arduino.addLibPath", + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged + // note: it has been removed from package.json commands already + // "arduino.addLibPath", "arduino.selectSerialPort", "arduino.openSerialMonitor", "arduino.changeBaudRate", From d483735548ac0c790a4ac58e4c4ff5c228a77496 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Tue, 18 Feb 2020 01:54:11 +0100 Subject: [PATCH 039/100] Updated project log --- BRANCHNOTES.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index e2dcbc55..02dbfac9 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -35,7 +35,7 @@ During merging I found some bugs within those functions - mainly due to the abov * Error message formatting was fixed within `verify` only * No consistent return values within `verify` (when it bailed out early it returned `void`) -**2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal. +**2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. ### Status | | Tasks | @@ -66,6 +66,7 @@ During merging I found some bugs within those functions - mainly due to the abov | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :heavy_check_mark: Parser only works when arduino is set to `verbose`, since this is the only way we get the compiler invocation command - this has to be fixed (done, see next item) | | | :heavy_check_mark: Implement a *Rebuild IntelliSense Configuration* command which runs verify verbosely internally and therefore allows us to find and parse the compiler command | +| | :white_check_mark: Implement proper event generation for `DeviceContext`. a) Events should be issued only when something actually changes, b) Events should be issued for each setting separately | | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet @@ -99,9 +100,9 @@ I will list every supporter here, thanks! 2020-02-10 Elektronik Workshop: 32 :beers: (8h coding) 2020-02-11 Elektronik Workshop: 16 :beers: (4h coding) 2020-02-12 Elektronik Workshop: 32 :beers: (8h coding) -2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) +2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) 2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) -2020-02-16 Elektronik Workshop: x :beers: (xh coding) +2020-02-17 Elektronik Workshop: 52 :beers: (13h coding) @@ -114,6 +115,7 @@ I will list every supporter here, thanks! * [Arduino CLI manpage](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc) * [Install extensions from file](https://vscode-docs.readthedocs.io/en/stable/extensions/install-extension/) * [Publish extensions](https://code.visualstudio.com/api/working-with-extensions/publishing-extension) +* [Arduino Dev Tools](https://playground.arduino.cc/Main/DevelopmentTools/) ## Issues Concerning this Project * https://github.com/Microsoft/vscode-cpptools/issues/1750 @@ -144,6 +146,8 @@ I will list every supporter here, thanks! * general lack of modularity - the above is the result * It seems that this extension is pretty chaotic. Most probably some refactoring is necessary. * Possibility to jump to compilation errors from compiler output and highlight compiler errors +* Further IntelliSense enhancements/features: + * When having adding a library folder to the workspace IntelliSense should use the same configuration for it to enable library navigation and code completion. ## Non-categorized Notes ### Integrate upstream changes into fork From 558c3e307779931ed59c291bd6ea14b1c511e567 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Tue, 18 Feb 2020 13:26:09 +0100 Subject: [PATCH 040/100] Don't use hard coded paths when possible. --- src/common/workspace.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/common/workspace.ts b/src/common/workspace.ts index 55b38e98..215fa92a 100644 --- a/src/common/workspace.ts +++ b/src/common/workspace.ts @@ -4,6 +4,7 @@ import * as fs from "fs"; import * as path from "path"; import * as vscode from "vscode"; +import { ARDUINO_CONFIG_FILE } from "./constants"; export class ArduinoWorkspace { static get rootPath(): string|undefined { @@ -14,7 +15,7 @@ export class ArduinoWorkspace { for (const workspaceFolder of workspaceFolders) { const workspaceFolderPath = workspaceFolder.uri.fsPath; - const arduinoConfigPath = path.join(workspaceFolderPath, ".vscode", "arduino.json"); + const arduinoConfigPath = path.join(workspaceFolderPath, ARDUINO_CONFIG_FILE); if (fs.existsSync(arduinoConfigPath)) { return workspaceFolderPath; } From 8477dcade226eac1f2db27d7b7296dd98bb85c43 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Tue, 18 Feb 2020 18:21:21 +0100 Subject: [PATCH 041/100] New classes which will improve the device settings management and event handling. To be integrated with one of the next commits. --- src/deviceSettings.ts | 234 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 234 insertions(+) create mode 100644 src/deviceSettings.ts diff --git a/src/deviceSettings.ts b/src/deviceSettings.ts new file mode 100644 index 00000000..e9281eac --- /dev/null +++ b/src/deviceSettings.ts @@ -0,0 +1,234 @@ +import * as fs from "fs"; +import * as path from "path"; +import * as vscode from "vscode"; + +import * as constants from "./common/constants"; +import * as util from "./common/util"; +import * as logger from "./logger/logger"; + +/** + * Generic class which provides monitoring of a specific settings value. + * If the value is modified a flag is set and an event is emitted. + * + * Usually you want to specialize the setter for any given value type + * to prevent invalid or badly formatted values to enter your settings. + */ +class Setting { + /** The actual value of the setting. */ + private _value: T | undefined; + /** The setting's default value. */ + readonly default: T; + /** Indicates if the value was changed since the last call to this.commit(). */ + private _modified: boolean; + /** Event emitter which fires when the value is changed. */ + private _emitter: vscode.EventEmitter = new vscode.EventEmitter(); + + constructor(defaultValue?: T) { + this.default = defaultValue; + this._value = this.default; + } + /** + * Value-setter - sets the value. + * If modified, the modified flag is set and the modified event is + * fired. + */ + public set value(value: T | undefined) { + if (value !== this._value) { + this._value = value; + this._modified = true; + this._emitter.fire(this._value); + } + } + /** Value-getter - returns the internal value. */ + public get value() { + return this._value; + } + /** + * Returns true, if the internal value has been modified. + * To clear the modified flag call commit(). + */ + public get modified() { + return this._modified; + } + /** Returns the modified-event emitter. */ + public get emitter() { + return this._emitter; + } + /** + * Returns the internal value to its default value. + * If the default value is different from the previous value, + * it triggers the modified event and the modified flag is set. + */ + public reset() { + this.value = this.default; + } + /** Reset the modified flag (if you know what you're doing) */ + public commit() { + this._modified = false; + } +} + +/** + * String specialization of the Setting class. + */ +class StrSetting extends Setting { + /** + * When we override setter (below) we have to override getter as well + * (see JS language specs). + */ + public get value() { + return super.value; + } + /** + * Set string value. Anything else than a string will set the value to + * its default value (undefined). White spaces at the front and back are + * trimmed before setting the value. + * If the setting's value is changed during this operation, the base + * class' event emitter will fire and the modified flag will be set. + */ + public set value(value: string) { + if (typeof value !== "string") { + value = this.default; + } else { + value = value.trim(); + } + super.value = value; + } +} + +/** + * This class encapsulates all device/project specific settings and + * provides common operations on them. + */ +export class DeviceSettings { + port: StrSetting = new StrSetting(); + board: StrSetting = new StrSetting(); + sketch: StrSetting = new StrSetting(); + output: StrSetting = new StrSetting(); + debugger: StrSetting = new StrSetting(); + disableIntelliSenseAutoGen: StrSetting = new StrSetting(); + configuration: StrSetting = new StrSetting(); + prebuild: StrSetting = new StrSetting(); + programmer: StrSetting = new StrSetting(); + + /** + * @returns true if any of the settings values has its modified flag + * set. + */ + public get modified() { + return this.port.modified || + this.board.modified || + this.sketch.modified || + this.output.modified || + this.debugger.modified || + this.disableIntelliSenseAutoGen.modified || + this.configuration.modified || + this.prebuild.modified || + this.programmer.modified; + } + /** + * Clear modified flags of all settings values. + */ + public commit() { + this.port.commit(); + this.board.commit(); + this.sketch.commit(); + this.output.commit(); + this.debugger.commit(); + this.disableIntelliSenseAutoGen.commit(); + this.configuration.commit(); + this.prebuild.commit(); + this.programmer.commit(); + } + /** + * Resets all settings values to their default values. + * @param commit If true clear the modified flags after all values are + * reset. + */ + public reset(commit: boolean = true) { + this.port.reset(); + this.board.reset(); + this.sketch.reset(); + this.output.reset(); + this.debugger.reset(); + this.disableIntelliSenseAutoGen.reset(); + this.configuration.reset(); + this.prebuild.reset(); + this.programmer.reset(); + if (commit) { + this.commit(); + } + } + /** + * Load settings values from the given file. + * If a value is changed through this operation, its event emitter will + * fire. + * @param file Path to the file the settings should be loaded from. + * @param commit If true reset the modified flags after all values are read. + * @returns true if the settings are loaded successfully. + */ + public load(file: string, commit: boolean = true) { + const settings = util.tryParseJSON(fs.readFileSync(file, "utf8")); + if (settings) { + this.port.value = settings.port; + this.board.value = settings.board; + this.sketch.value = settings.sketch; + this.configuration.value = settings.configuration; + this.output.value = settings.output; + this.debugger.value = settings.debugger; + this.disableIntelliSenseAutoGen.value = settings.disableIntelliSenseAutoGen; + this.prebuild.value = settings.prebuild; + this.programmer.value = settings.programmer; + if (commit) { + this.commit(); + } + return true; + } else { + logger.notifyUserError("arduinoFileError", + new Error(constants.messages.ARDUINO_FILE_ERROR)); + return false; + } + } + /** + * Writes the settings to the given file if there are modified + * values. The modification flags are reset (commit()) on successful write. + * On write failure the modification flags are left unmodified. + * @param file Path to file the JSON representation of the settings should + * written to. If either the folder or the file does not exist they are + * created. + * @returns true on succes, false on write failure. + */ + public save(file: string) { + + if (!this.modified) { + return true; + } + + let settings: any = {}; + if (util.fileExistsSync(file)) { + settings = util.tryParseJSON(fs.readFileSync(file, "utf8")); + } + if (!settings) { + logger.notifyUserError( + "arduinoFileError", + new Error(constants.messages.ARDUINO_FILE_ERROR)); + return false; + } + + settings.sketch = this.sketch.value; + settings.port = this.port.value; + settings.board = this.board.value; + settings.output = this.output.value; + settings.debugger = this.debugger.value; + settings.disableIntelliSenseAutoGen = this.disableIntelliSenseAutoGen.value; + settings.configuration = this.configuration.value; + settings.programmer = this.programmer.value; + + util.mkdirRecursivelySync(path.dirname(file)); + fs.writeFileSync(file, JSON.stringify(settings, undefined, 4)); + + this.commit(); + + return true; + } +} From 664af5619c52feb242990ea8f008b81348969f06 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Tue, 18 Feb 2020 20:12:07 +0100 Subject: [PATCH 042/100] * Added/documented all known related issues * Updated log * Updated and extended status * General brushing of structure and text --- BRANCHNOTES.md | 120 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 80 insertions(+), 40 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 02dbfac9..d8ea45fd 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -1,22 +1,54 @@ # IntelliSense Autoconfiguration Branch ## Problem -This branch more or less addresses [these](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) issues (about seven). - -It implements a parser which parses the output from Arduino's build process to generate a very precise `c_cpp_properties.json` which in turn hopefully renders any user interaction with this file obsolete +This branch more or less addresses the following issues: +| # | Issue | Title | Comment | +|--:|:------|:------------|:--------| +| 1| [#438](https://github.com/microsoft/vscode-arduino/issues/438) | **The Extension should automagically fill the c_cpp_properties.json, so intellisense works out of the box** | This is the issue to which I usually report news concerning the progress of this project | +| 2| [#876](https://github.com/microsoft/vscode-arduino/issues/876) | **Missing #define ARDUINO 10808 logic?** | Marked as bug but it's just the same problem again | +| 3| [#839](https://github.com/microsoft/vscode-arduino/issues/829) | **Arduino.h and ESP8266 includes have squiggles** | Closed due to inactivity - not because it has been fixed | +| 4| [#969](https://github.com/microsoft/vscode-arduino/issues/969) | **INO Files defines undefined but can jump to definition** | | +| 5| [#959](https://github.com/microsoft/vscode-arduino/issues/959) | **Update board type command does not update intellisense config automatically** | | +| 6| [#892](https://github.com/microsoft/vscode-arduino/issues/892) | **Default IntelliSense config** | | +| 7| [#850](https://github.com/microsoft/vscode-arduino/issues/850) | **How to prevent modifications of c_cpp_properties.json by the extension?** | Asks if the current implementation can be turned off, because it overwrites a user's config with non working IS config -- this is sad. | +| 8| [#833](https://github.com/microsoft/vscode-arduino/issues/833) | **Allow C_Cpp.intelliSenseEngine to be set to "Default" instead of "Tag Parser" for better code completion/suggestions** | | +| 9| [#808](https://github.com/microsoft/vscode-arduino/issues/808) | **Identifier "Serial" is undefined** | | +| 10| [#474](https://github.com/microsoft/vscode-arduino/issues/474) | **Enrich device develop experience** | | + + + + + + +-- the list is probably incomplete - I didn't search exhaustively and didn't consider closed issues as long as I didn't stumble upon one. New duplicates are popping up at a rate of about one per week. + +Further related issues +* [vscode-arduino issue search for IntelliSense](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) +* [Wrongly attributed to vscode instead of vscode-arduino](https://github.com/Microsoft/vscode-cpptools/issues/1750) +* [Problems with IntelliSense itself](https://github.com/microsoft/vscode-cpptools/issues/1034) + +## Solution +Implement and add a a parser which parses the output from Arduino's build process and generate a very precise `c_cpp_properties.json` which in turn hopefully renders any user interaction with this file obsolete. + +This mechanism should try it's best to detect situations in which the setup changes (board changed, sketch changed and so forth) and re-generate this configuration. For all other situations I'll provide a command to manually re-generate it. ## Branch Goals ### Build Output Parser -The parser which parses the relevant includes, defines, compiler paths and flags from Arduino's build output +The parser which identifies the main compilation command from Arduino's build output. It then parses the relevant includes, defines, compiler paths and flags from it. + ### `c_cpp_properties.json` Generator -The generator takes the parser's output and transforms it into a valid `c_cpp_properties.json` file. +The generator takes the parser's output and transforms it into a valid `c_cpp_properties.json` file. It merges the generated configuration with the existing (e.g. user-) configurations and writes it back to the configuration file. ### Configuration Flags -Provide a configuration flag which allows the user to turn this feature off - this is useful for the cases in which this magic fails or the user has a very specific setup. Although this branch tries to eliminate most of the latter cases. +Provide a global configuration flag which allows the user to turn this feature off. A project- (sketch-) specific override will be provided which allows the user to turn it off or on - regardless of the global setting. +This is useful for the rare cases for which this magic should fail or the user has a very exotic setup. This branch tries to eliminate most of the latter cases though. Especially it will always write to the `Arduino` configuration. If the user sets up a custom configuration she/he must simply name it differently, e.g. `John's Custom Config`, and the generator won't touch it. ### Global Tasks in vscode-arduino -See table below. +* Integrate it into vscode-arduino's build mechanics +* Install event trigger generation/handling to run the analysis as soon as something changes +* Remove the current implementation +For more details see table below. -### Branch Log +## Branch Log **2020 02 05** Currently I'm able to generate error free IntelliSense setups for AVR and ESP32 using the preliminary implementation. For ESP32 I just had to add the intrinsic compiler paths manually. A solution has to be found for these ... which there is, see [here](https://stackoverflow.com/a/6666338) **2020 02 06** Got it fully working (with built-in include directories) for AVR, ESP32, ESP8266. Rewrote the backend to facilitate writing of further parser engines in the future. **2020 02 07** Wrote compiler command parser npm package [cocopa](https://www.npmjs.com/package/cocopa) and began writing a test framework for it. Added a global configuration switch which allows the IntelliSense configuration generation to be turned off. @@ -35,9 +67,10 @@ During merging I found some bugs within those functions - mainly due to the abov * Error message formatting was fixed within `verify` only * No consistent return values within `verify` (when it bailed out early it returned `void`) -**2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. +**2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. +**2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). -### Status +## Status | | Tasks | |-----:|:--------| | **Build output parser** | :heavy_check_mark: Basic parser working | @@ -58,15 +91,23 @@ During merging I found some bugs within those functions - mainly due to the abov | **Unit tests** | :heavy_check_mark: Basic parser (known boards, match/no match)| | | :white_check_mark: All unit tests in cocopa | | | :white_check_mark: Test with cpp sketches | -| **General** | :white_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense. (Partially done - documented in the [General Tasks](#General-Tasks) section | -| | :white_check_mark: Auto-run verify after a) *setting a board* b) *changing the sketch* c) *workbench initialized and no `c_cpp_properties.json` has been found*. We have to generate a valid `c_cpp_properties.json` to keep IntelliSense working in such situations. Identify other occasions where this applies (usually when adding new libraries), hint the user to run *Arduino: Rebuild IntelliSense Configuration*? -> Good moment would be after the workbench initialization -> message in arduino channel | +| **General** | :heavy_check_mark: Review and remove previous attempts messing with `c_cpp_properties.json` or IntelliSense (documented in the [General Tasks](#General-Tasks) section) `*` | +| | :white_check_mark: *Auto-run verify when* | +| |     :heavy_check_mark: a) setting a board `*` | +| |     :heavy_check_mark: b) changing the board's configuration `*` | +| |     :heavy_check_mark: c) changing the sketch `*` | +| |     :white_check_mark: d) workbench initialized and no `c_cpp_properties.json` found | +| |     :white_check_mark: e) Identify other occasions where this applies (usually when adding new libraries) | +| | :white_check_mark: Hint the user to run *Arduino: Rebuild IntelliSense Configuration*? -> Good moment would be after the workbench initialization -> message in arduino channel | +| | :white_check_mark: Better build management such that regular builds and analyze builds do not interfere | +| | :white_check_mark: Analyze task queue which fits in the latter | | | :heavy_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) (partially done) | | | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | -| | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack. Done, see [cocopa](https://www.npmjs.com/package/cocopa) | +| | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack -- done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :heavy_check_mark: Parser only works when arduino is set to `verbose`, since this is the only way we get the compiler invocation command - this has to be fixed (done, see next item) | | | :heavy_check_mark: Implement a *Rebuild IntelliSense Configuration* command which runs verify verbosely internally and therefore allows us to find and parse the compiler command | -| | :white_check_mark: Implement proper event generation for `DeviceContext`. a) Events should be issued only when something actually changes, b) Events should be issued for each setting separately | +| | :heavy_check_mark: Implement proper event generation for `DeviceContext`. a) Events should be issued only when something actually changes, b) Events should be issued for each setting separately `*`| | | :white_check_mark: Finally: go through my code and look for TODOs | `*` not committed to branch yet @@ -80,8 +121,8 @@ I write a lot of code for Arduino, especially libraries. The Arduino IDE is not Then remains vscode-arduino. It seems that it isn't completely dead - but almost. Most of the core functionality seems to work (I used it a few days now). But the biggest show stopper is the bad IntelliSense support -- which I'll address here now. -## Beer Money :beers: -You can chip in some beer money to keep me motivated - this is really appreciated. +## Beer Money :beers: -- Support +You can chip in some beer money to keep me motivated - this is *really* appreciated. [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=PVLCSRZHBJ28G&source=url) @@ -103,24 +144,20 @@ I will list every supporter here, thanks! 2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) 2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) 2020-02-17 Elektronik Workshop: 52 :beers: (13h coding) +2020-02-18 Elektronik Workshop: xx :beers: (xxh coding) ## Useful Links -* [IntelliSense issues on vscode-arduino](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) * [`c_cpp_properties.json` reference](https://code.visualstudio.com/docs/cpp/c-cpp-properties-schema-reference) * [Interactive regex debugger](https://regex101.com/) * [Git branch management](https://blog.scottlowe.org/2015/01/27/using-fork-branch-git-workflow/) * [Collapsible Markdown](https://gist.githubusercontent.com/joyrexus/16041f2426450e73f5df9391f7f7ae5f/raw/f774f242feff6bae4a5be7d6c71aa5df2e3fcb0e/README.md) * [Arduino CLI manpage](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc) * [Install extensions from file](https://vscode-docs.readthedocs.io/en/stable/extensions/install-extension/) -* [Publish extensions](https://code.visualstudio.com/api/working-with-extensions/publishing-extension) -* [Arduino Dev Tools](https://playground.arduino.cc/Main/DevelopmentTools/) - -## Issues Concerning this Project - * https://github.com/Microsoft/vscode-cpptools/issues/1750 - * Problems with IntelliSense itself https://github.com/microsoft/vscode-cpptools/issues/1034 - * Logging for IntelliSense https://code.visualstudio.com/docs/cpp/enable-logging-cpp +* [Publish vscode extensions](https://code.visualstudio.com/api/working-with-extensions/publishing-extension) +* [Debug logging for IntelliSense](https://code.visualstudio.com/docs/cpp/enable-logging-cpp) +* [Arduino Dev Tools](https://playground.arduino.cc/Main/DevelopmentTools/) (obsolete/outdated) ## Future Work * Proper interactive serial terminal (this is the second major show stopper in my opinion) @@ -244,15 +281,11 @@ Sometimes IntelliSense has problems within the extension host (which you're runn ---- -## Implementation -**Note** Check this vscode feature: -``` -Configuration provider -The ID of a VS Code extension that can provide IntelliSense configuration information for source files. For example, use the VS Code extension ID ms-vscode.cmake-tools to provide configuration information from the CMake Tools extension. -``` +# Implementation +Here are some implementation notes. Probably only of interest to me. -### Build Output Parser -#### Intrinsic Include Paths +## Build Output Parser +### Intrinsic Include Paths Some include paths are built into gcc and don't have to be specified on the command line. Just searching the compiler installation directory with something like ```bash find ~/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/ -name "include*" @@ -294,11 +327,11 @@ End of search list. ``` As one can see with the ESP32-gcc not all include directories are named `include`. Parsing of this output is pretty trivial though. -### `c_cpp_properties.json` Generator +## `c_cpp_properties.json` Generator -### Settings -#### Global Settings +## Settings +### Global Settings Under linux at `~/.config/Code/User/settings.json`, for instance: ```json { @@ -312,7 +345,7 @@ Code: [src/arduino/arduinoSettings.ts](src/arduino/arduinoSettings.ts) Code: [src/arduino/vscodeSettings.ts](src/arduino/vscodeSettings.ts) Validator: [package.json](package.json) -#### Project Settings +### Project Settings Path in project `.vscode/arduino.json` ```json { @@ -325,8 +358,8 @@ Path in project `.vscode/arduino.json` Code: [src/deviceContext.ts](src/deviceContext.ts) Validator: [misc/arduinoValidator.json](misc/arduinoValidator.json) -### General Tasks -#### Removing existing Attempts which mess with c_cpp_properties.json or Intellisense +## General Tasks +### Removing existing Attempts which mess with c_cpp_properties.json or Intellisense Remove these as they are helpless attempts to get IntelliSense working: ```ts @@ -344,7 +377,14 @@ Remove these as they are helpless attempts to get IntelliSense working: Remove this as this messes in an unpredictable and helpless way with Intellisense [src/langService/completionProvider.ts](src/langService/completionProvider.ts) -Remove this folder as this is not necessary when Intellisense works properly: +Review this folder as some of this is probably obsolete when Intellisense works properly: ``` -syntaxes/ +syntaxes/arduino.configuration.json +syntaxes/arduino.tmLanguage +# Within package.json + { + "language": "cpp", + "path": "./syntaxes/arduino.tmLanguage", + "scopeName": "source.cpp.arduino" + }, ``` From bdb0e735087940b00b03e557518c4bbf14e204a5 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 19 Feb 2020 16:18:15 +0100 Subject: [PATCH 043/100] Updated project log and status --- BRANCHNOTES.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index d8ea45fd..e8568798 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -68,7 +68,8 @@ During merging I found some bugs within those functions - mainly due to the abov * No consistent return values within `verify` (when it bailed out early it returned `void`) **2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. -**2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). +**2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). +**2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. ## Status | | Tasks | @@ -95,12 +96,12 @@ During merging I found some bugs within those functions - mainly due to the abov | | :white_check_mark: *Auto-run verify when* | | |     :heavy_check_mark: a) setting a board `*` | | |     :heavy_check_mark: b) changing the board's configuration `*` | -| |     :heavy_check_mark: c) changing the sketch `*` | +| |     :heavy_check_mark: c) selecting another sketch `*` | | |     :white_check_mark: d) workbench initialized and no `c_cpp_properties.json` found | | |     :white_check_mark: e) Identify other occasions where this applies (usually when adding new libraries) | | | :white_check_mark: Hint the user to run *Arduino: Rebuild IntelliSense Configuration*? -> Good moment would be after the workbench initialization -> message in arduino channel | -| | :white_check_mark: Better build management such that regular builds and analyze builds do not interfere | -| | :white_check_mark: Analyze task queue which fits in the latter | +| | :heavy_check_mark: Better build management such that regular builds and analyze builds do not interfere (done, 2020-02-19) `*` | +| | :heavy_check_mark: Analyze task queue which fits in the latter (done, 2020-02-19) `*` | | | :heavy_check_mark: Document configuration settings in [README.md](README.md) | | | :white_check_mark: Document features in [README.md](README.md) (partially done) | | | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | @@ -144,7 +145,8 @@ I will list every supporter here, thanks! 2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) 2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) 2020-02-17 Elektronik Workshop: 52 :beers: (13h coding) -2020-02-18 Elektronik Workshop: xx :beers: (xxh coding) +2020-02-18 Elektronik Workshop: 36 :beers: (9h coding) +2020-02-19 Elektronik Workshop: x :beers: (xh coding) @@ -185,6 +187,7 @@ I will list every supporter here, thanks! * Possibility to jump to compilation errors from compiler output and highlight compiler errors * Further IntelliSense enhancements/features: * When having adding a library folder to the workspace IntelliSense should use the same configuration for it to enable library navigation and code completion. + * Optimization: Abort analysis build as soon as compiler statement has been found ## Non-categorized Notes ### Integrate upstream changes into fork From fb77b9f0cafd99336bd00086af8b2be9cbff335c Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 19 Feb 2020 19:36:36 +0100 Subject: [PATCH 044/100] * Updated and completed documentation of the current state within README.md * Updated project log and status --- BRANCHNOTES.md | 4 ++-- README.md | 22 ++++++++++++++-------- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index e8568798..9394ea9d 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -69,7 +69,7 @@ During merging I found some bugs within those functions - mainly due to the abov **2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. **2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). -**2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. +**2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. Updated and completed the documentation for the IntelliSense usage within [README](README.md). Alpha test builds of the extension containing the latest implemented features and fixes are now available from the following [Dropbox folder](https://www.dropbox.com/sh/whmcdt26chyjgby/AAB1Ld2fzZ9Z_NfM3CRay17wa). Please note, that Windows is currently not supported yet. ## Status | | Tasks | @@ -103,7 +103,7 @@ During merging I found some bugs within those functions - mainly due to the abov | | :heavy_check_mark: Better build management such that regular builds and analyze builds do not interfere (done, 2020-02-19) `*` | | | :heavy_check_mark: Analyze task queue which fits in the latter (done, 2020-02-19) `*` | | | :heavy_check_mark: Document configuration settings in [README.md](README.md) | -| | :white_check_mark: Document features in [README.md](README.md) (partially done) | +| | :heavy_check_mark: Document features in [README.md](README.md) | | | :heavy_check_mark: Try to auto-generate even if verify (i.e. compilation) fails | | | :heavy_check_mark: Extract compiler command parser from vscode-arduino and [publish](https://itnext.io/step-by-step-building-and-publishing-an-npm-typescript-package-44fe7164964c) it as a separate package which will allow reusage and easy testing without heavy vscode-arduino rucksack -- done, see [cocopa](https://www.npmjs.com/package/cocopa) | | | :heavy_check_mark: Parser only works when arduino is set to `verbose`, since this is the only way we get the compiler invocation command - this has to be fixed (done, see next item) | diff --git a/README.md b/README.md index 1e071336..79b36d11 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ The following settings are as per sketch settings of the Arduino extension. You "output": "../build", "debugger": "jlink", "prebuild": "bash prebuild.sh", - "disableIntelliSenseAutoGen": "global" + "intelliSenseGen": "global" } ``` - `sketch` - The main sketch file name of Arduino. @@ -111,21 +111,27 @@ The following settings are as per sketch settings of the Arduino extension. You - `output` - Arduino build output path. If not set, Arduino will create a new temporary output folder each time, which means it cannot reuse the intermediate result of the previous build leading to long verify/upload time, so it is recommended to set the field. Arduino requires that the output path should not be the workspace itself or in a subfolder of the workspace, otherwise, it may not work correctly. By default, this option is not set. It's worth noting that the contents of this file could be deleted during the build process, so pick (or create) a directory that will not store files you want to keep. - `debugger` - The short name of the debugger that will be used when the board itself does not have a debugger and there is more than one debugger available. You can find the list of debuggers [here](https://github.com/Microsoft/vscode-arduino/blob/master/misc/debuggerUsbMapping.json). By default, this option is not set. - `prebuild` - External command before building the sketch file. You should only set one `prebuild` command. `command1 && command2` does not work. If you need to run multiple commands before the build, then create a script. -- `disableIntelliSenseAutoGen` - Override the global auto-generation of the IntelliSense configuration (i.e. `.vscode/c_cpp_properties.json`). Three options are available: +- `intelliSenseGen` - Override the global setting for auto-generation of the IntelliSense configuration (i.e. `.vscode/c_cpp_properties.json`). Three options are available: - `"global"`: Use the global settings (default) - `"disable"`: Disable the auto-generation even if globally enabled - `"enable"`: Enable the auto-generation even if globally disabled ## IntelliSense -*TODO: Rewrite this section* -vscode-arduino auto-configures IntelliSense by default. vscode-arduino analyzes Arduino's compiler output during verify and generates the corresponding configuration file at `.vscode/c_cpp_properties.json` and tries as hard as possible to keep things up to date, e.g. running verify when switching the board or the sketch. -It doesn't makes sense though to run verify repeatedly. Therefore if the workspace reports problems (for instance after adding new includes to a new library) run *verify* such that IntelliSense knows of the new include directories (since the Arduino-backend performs the library resolution externally). +vscode-arduino auto-configures IntelliSense by default. vscode-arduino analyzes Arduino's compiler output by running a separate build and generates the corresponding configuration file at `.vscode/c_cpp_properties.json`. vscode-arduino tries as hard as possible to keep things up to date, e.g. it runs the analysis when switching the board or the sketch. -TODO: Note about configuration selection in lower right. +It doesn't makes sense though to analyse repeatedly. Therefore if the workspace reports problems - for instance after adding new includes from a new library - run the analysis manually. -Manual rebuild: **Ardino: Rebuild IntelliSense Configuration**, -Keybindings: **Arduino: Rebuild IntelliSense Configuration** Alt + Cmd + I *or* Alt + Ctrl + I +Manual rebuild: **Arduino: Rebuild IntelliSense Configuration**, +Keybindings: Alt + Cmd + I *or* Alt + Ctrl + I +When the analysis is invoked manually it overrides global and project specific settings. + +### IntelliSense Configurations +vscode-arduino's analysis stores the result as a dedicated IntelliSense-configuration named `Arduino`. You have to select it from the far right of the status bar when you're in one of your source files as shown here: + +![74001156-cfce8280-496a-11ea-9b9d-7d30c83765c1](https://user-images.githubusercontent.com/21954933/74351237-2696ea80-4db7-11ea-9f7a-1bfc652ad5f5.png) + +This system allows you to setup and use own IntelliSense configurations in parallel to the automatically generated configurations provided through vscode-arduino. Just add your configuration to `c_cpp_properties.json` and name it differently from the default configuration (`Arduino`), e.g. `My awesome configuration` and select it from the status bar or via the command palette command **C/C++: Select a Configuration...** ## Debugging Arduino Code preview Before you start to debug your Arduino code, please read [this document](https://code.visualstudio.com/docs/editor/debugging) to learn about the basic mechanisms of debugging in Visual Studio Code. Also see [debugging for C++ in VSCode](https://code.visualstudio.com/docs/languages/cpp#_debugging) for further reference. From 6c8c11ddd5db02134dee088ddfb3eb89b32641ad Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 19 Feb 2020 19:39:59 +0100 Subject: [PATCH 045/100] Minor adjustment on README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 79b36d11..67c31d42 100644 --- a/README.md +++ b/README.md @@ -119,7 +119,7 @@ The following settings are as per sketch settings of the Arduino extension. You ## IntelliSense vscode-arduino auto-configures IntelliSense by default. vscode-arduino analyzes Arduino's compiler output by running a separate build and generates the corresponding configuration file at `.vscode/c_cpp_properties.json`. vscode-arduino tries as hard as possible to keep things up to date, e.g. it runs the analysis when switching the board or the sketch. -It doesn't makes sense though to analyse repeatedly. Therefore if the workspace reports problems - for instance after adding new includes from a new library - run the analysis manually. +It doesn't makes sense though to run the analysis repeatedly. Therefore if the workspace reports problems ("squiggles") - for instance after adding new includes from a new library - run the analysis manually: Manual rebuild: **Arduino: Rebuild IntelliSense Configuration**, Keybindings: Alt + Cmd + I *or* Alt + Ctrl + I From 8840f51914c1244b942c39ec76f60c7ca7aa715c Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 19 Feb 2020 22:45:54 +0100 Subject: [PATCH 046/100] IntelliSense auto-analysis integration * Reworked DeviceContext for fine grained event generation on settings change * Added dedicated settings classes which allows for the latter * Changed board manager and serial monitor to used the new fine grained events instead of the "something changed event" - which makes the code marginally more efficient * Implemented an analysis manager which takes care of analysis requests generated by settings change events and makes sure that analysis builds don't interfere with regular builds * Adapted the build infrastructure within ArduinoApp such that it allows for the above * Removed the global build guard and moved it to ArduinoApp (there is a corner case though for programmer selection within extension.ts which I marked with a TODO * Fixed some linting issues but did not lint most of the stuff committed here - this will be part of a later commit * I left notes here and there where I saw things which should/could be improved/changed * Removed the try/catch guards from the build invocations within extension.ts since exceptions are handled within the build function itself * Changed the project setting parameter for the IntelliSense setup to reflect its workings better --- misc/arduinoValidator.json | 2 +- src/arduino/arduino.ts | 418 +++++++++++++++----------- src/arduino/arduinoContentProvider.ts | 2 +- src/arduino/boardManager.ts | 7 +- src/arduino/intellisense.ts | 196 +++++++++++- src/arduinoContext.ts | 6 + src/common/util.ts | 2 +- src/deviceContext.ts | 221 ++++++-------- src/deviceSettings.ts | 15 +- src/extension.ts | 68 ++--- src/langService/completionProvider.ts | 9 +- src/serialmonitor/serialMonitor.ts | 24 +- src/serialmonitor/usbDetector.ts | 3 + test/boardmanager.test.ts | 2 +- 14 files changed, 598 insertions(+), 377 deletions(-) diff --git a/misc/arduinoValidator.json b/misc/arduinoValidator.json index 492cfdf0..15f3b812 100644 --- a/misc/arduinoValidator.json +++ b/misc/arduinoValidator.json @@ -33,7 +33,7 @@ "type": "string", "minLength": 1 }, - "disableIntelliSenseAutoGen": { + "intelliSenseGen": { "description": "Disable/enable the automatic generation of the IntelliSense configuration file (c_cpp_properties.json) for this project (overrides the global setting). When set to \"global\" the global extension settings will be used.", "type": "string", "default": "global", diff --git a/src/arduino/arduino.ts b/src/arduino/arduino.ts index cb691720..af35e887 100644 --- a/src/arduino/arduino.ts +++ b/src/arduino/arduino.ts @@ -15,7 +15,10 @@ import { DeviceContext } from "../deviceContext"; import { IArduinoSettings } from "./arduinoSettings"; import { BoardManager } from "./boardManager"; import { ExampleManager } from "./exampleManager"; -import { ICoCoPaContext, isCompilerParserEnabled, makeCompilerParserContext } from "./intellisense"; +import { AnalysisManager, + ICoCoPaContext, + isCompilerParserEnabled, + makeCompilerParserContext } from "./intellisense"; import { LibraryManager } from "./libraryManager"; import { ProgrammerManager } from "./programmerManager"; import { VscodeSettings } from "./vscodeSettings"; @@ -25,6 +28,11 @@ import { ArduinoWorkspace } from "../common/workspace"; import { SerialMonitor } from "../serialmonitor/serialMonitor"; import { UsbDetector } from "../serialmonitor/usbDetector"; +/** + * Supported build modes. For further explanation see the documentation + * of ArduinoApp.build(). + * The strings are used for status reporting within the above function. + */ export enum BuildMode { Verify = "Verifying", Analyze = "Analyzing", @@ -45,10 +53,30 @@ export class ArduinoApp { private _programmerManager: ProgrammerManager; + /** + * IntelliSense analysis manager. + * Makes sure that analysis builds and regular builds go along + * and that multiple subsequent analysis requests - as triggered + * by board/board-configuration changes - are bundled to a single + * analysis build run. + */ + private _analysisManager: AnalysisManager; + + /** + * Indicates if a build is currently in progress. + * If so any call to this.build() will return false immediately. + */ + private _building: boolean = false; + /** * @param {IArduinoSettings} _settings ArduinoSetting object. */ constructor(private _settings: IArduinoSettings) { + const analysisDelayMs = 1000 * 3; + this._analysisManager = new AnalysisManager( + () => { return this._building; }, + async () => { await this.build(BuildMode.Analyze); }, + analysisDelayMs); } /** @@ -71,6 +99,17 @@ export class ArduinoApp { } catch (ex) { } } + + // set up event handling for IntelliSense analysis + const requestAnalysis = async () => { + if (isCompilerParserEnabled()) { + await this._analysisManager.requestAnalysis(); + } + }; + const dc = DeviceContext.getInstance(); + dc.onChangeBoard(requestAnalysis); + dc.onChangeConfiguration(requestAnalysis); + dc.onChangeSketch(requestAnalysis); } /** @@ -101,167 +140,52 @@ export class ArduinoApp { } } + /** + * Returns true if a build is currently in progress. + */ + public get building() { + return this._building; + } + /** * Runs the arduino builder to build/compile and - if necessary - upload * the current sketch. - * @param mode Build mode. BuildMode.Upload: Compile and upload, - * BuildMode.UploadProgrammer: Compile and upload using the user selectable - * programmer, BuildMode.Analyze: Compile, analyze the output and generate - * IntelliSense configuration from it, BuildMode.Verify: Just compile. + * @param mode Build mode. + * * BuildMode.Upload: Compile and upload + * * BuildMode.UploadProgrammer: Compile and upload using the user + * selectable programmer + * * BuildMode.Analyze: Compile, analyze the output and generate + * IntelliSense configuration from it. + * * BuildMode.Verify: Just compile. + * All build modes except for BuildMode.Analyze run interactively, i.e. if + * something is missing, it tries to query the user for the missing piece + * of information (sketch, board, etc.). Analyze runs non interactively and + * just returns false. * @param buildDir Override the build directory set by the project settings * with the given directory. + * @returns true on success, false if + * * another build is currently in progress + * * board- or programmer-manager aren't initialized yet + * * or something went wrong during the build */ - public async build(mode: BuildMode, buildDir?: string): Promise { - - const dc = DeviceContext.getInstance(); - const args: string[] = []; - let restoreSerialMonitor: boolean = false; - let cocopa: ICoCoPaContext; - - const boardDescriptor = this.getBoardBuildString(); - if (!boardDescriptor) { - return false; - } - args.push("--board", boardDescriptor); + public async build(mode: BuildMode, buildDir?: string) { - if (!ArduinoWorkspace.rootPath) { - vscode.window.showWarningMessage("Cannot find the sketch file."); + if (!this._boardManager || !this._programmerManager || this._building) { return false; } - if (!dc.sketch || !util.fileExistsSync(path.join(ArduinoWorkspace.rootPath, dc.sketch))) { - await this.getMainSketch(dc); - } - - const selectSerial = async () => { - const choice = await vscode.window.showInformationMessage( - "Serial port is not specified. Do you want to select a serial port for uploading?", - "Yes", "No"); - if (choice === "Yes") { - vscode.commands.executeCommand("arduino.selectSerialPort"); - } - } - - if (mode === BuildMode.Upload) { - if ((!dc.configuration || !/upload_method=[^=,]*st[^,]*link/i.test(dc.configuration)) && !dc.port) { - await selectSerial(); - return false; - } - args.push("--upload"); - if (dc.port) { - args.push("--port", dc.port); - } - } else if (mode === BuildMode.UploadProgrammer) { - const programmer = this.getProgrammerString(); - if (!programmer) { - return false; - } - if (!dc.port) { - await selectSerial(); - return false; - } - args.push("--upload", - "--port", dc.port, - "--useprogrammer", - "--pref", "programmer=" + programmer); - - } else if (mode === BuildMode.Analyze) { - if (!isCompilerParserEnabled()) { - return false; - } - cocopa = makeCompilerParserContext(dc); - args.push("--verify", "--verbose"); - } else { - args.push("--verify"); - } - - const verbose = VscodeSettings.getInstance().logLevel === "verbose"; - if (mode !== BuildMode.Analyze && verbose) { - args.push("--verbose"); - } - - await vscode.workspace.saveAll(false); - - arduinoChannel.show(); - arduinoChannel.start(`${mode} sketch '${dc.sketch}'`); + this._building = true; - if (!await this.runPreBuildCommand(dc)) { + return await this._build(mode, buildDir) + .then((ret) => { + this._building = false; + return ret; + }) + .catch((reason) => { + this._building = false; + // TODO EW, 2020-02-19: Report unhandled error (Logger?) return false; - } - - if (buildDir || dc.output) { - const outputPath = path.resolve(ArduinoWorkspace.rootPath, buildDir || dc.output); - const dirPath = path.dirname(outputPath); - if (!util.directoryExistsSync(dirPath)) { - logger.notifyUserError("InvalidOutPutPath", new Error(constants.messages.INVALID_OUTPUT_PATH + outputPath)); - return; - } - - args.push("--pref", `build.path=${outputPath}`); - arduinoChannel.info(`Please see the build logs in output path: ${outputPath}`); - } else { - const msg = "Output path is not specified. Unable to reuse previously compiled files. Build will be slower. See README."; - arduinoChannel.warning(msg); - } - - // stop serial monitor when everything is prepared and good - // what makes restoring of its previous state easier - if (mode === BuildMode.Upload || mode === BuildMode.UploadProgrammer) { - restoreSerialMonitor = await SerialMonitor.getInstance().closeSerialMonitor(dc.port); - UsbDetector.getInstance().pauseListening(); - } - - // Push sketch as last argument - args.push(path.join(ArduinoWorkspace.rootPath, dc.sketch)); - - let success = false; - - const cleanup = async () => { - if (cocopa) { - cocopa.conclude(); - } - if (mode === BuildMode.Upload || mode === BuildMode.UploadProgrammer) { - UsbDetector.getInstance().resumeListening(); - if (restoreSerialMonitor) { - await SerialMonitor.getInstance().openSerialMonitor(); - } - } - } - - // TODO: Get rid of spawn's channel parameter and just support - // stdout and stderr callbacks - const stdoutCallback = (line: string) => { - if (cocopa) { - cocopa.callback(line); - if (verbose) { - arduinoChannel.channel.append(line); - } - } else { - arduinoChannel.channel.append(line); - } - } - - await util.spawn( - this._settings.commandPath, - arduinoChannel.channel, - args, - undefined, - stdoutCallback, - ).then(async () => { - await cleanup(); - arduinoChannel.end(`${mode} sketch '${dc.sketch}${os.EOL}`); - success = true; - }, async (reason) => { - await cleanup(); - const msg = reason.code ? - `Exit with code=${reason.code}` : - reason.message ? - reason.message : - JSON.stringify(reason); - arduinoChannel.error(`${mode} sketch '${dc.sketch}': ${msg}${os.EOL}`); }); - - return success; } // IS-REMOVE: to be removed completely when IntelliSense implementation is merged @@ -607,7 +531,7 @@ Please make sure the folder is not occupied by other procedures .`); const dc = DeviceContext.getInstance(); const arduinoJson = { sketch: sketchFile, - // TODO: COM1 is Windows specific - what about OSX and Linux users? + // TODO EW, 2020-02-18: COM1 is Windows specific - what about OSX and Linux users? port: dc.port || "COM1", board: dc.board, configuration: dc.configuration, @@ -709,14 +633,15 @@ Please make sure the folder is not occupied by other procedures .`); * @returns True if successful, false on error. */ protected async runPreBuildCommand(dc: DeviceContext): Promise { - if (dc.prebuild) { - arduinoChannel.info(`Running pre-build command: ${dc.prebuild}`); - const prebuildargs = dc.prebuild.split(" "); - const prebuildCommand = prebuildargs.shift(); + const prebuildcmdline = dc.prebuild; + if (prebuildcmdline) { + arduinoChannel.info(`Running pre-build command: ${prebuildcmdline}`); + const args = prebuildcmdline.split(/\s+/); + const cmd = args.shift(); try { - await util.spawn(prebuildCommand, + await util.spawn(cmd, arduinoChannel.channel, - prebuildargs, + args, { shell: true, cwd: ArduinoWorkspace.rootPath }); } catch (ex) { arduinoChannel.error(`Running pre-build command failed: ${os.EOL}${ex.error}`); @@ -726,29 +651,172 @@ Please make sure the folder is not occupied by other procedures .`); return true; } - private getProgrammerString(): string { - const selectProgrammer = this.programmerManager.currentProgrammer; - if (!selectProgrammer) { - logger.notifyUserError("getProgrammerString", new Error(constants.messages.NO_PROGRAMMMER_SELECTED)); - return; + /** + * Private implementation. Not to be called directly. The wrapper build() + * manages the build state. + * @param mode See build() + * @param buildDir See build() + */ + private async _build(mode: BuildMode, buildDir?: string): Promise { + const dc = DeviceContext.getInstance(); + const args: string[] = []; + let restoreSerialMonitor: boolean = false; + let cocopa: ICoCoPaContext; + + if (!this.boardManager.currentBoard) { + if (mode !== BuildMode.Analyze) { + logger.notifyUserError("getBoardBuildString", new Error(constants.messages.NO_BOARD_SELECTED)); + } + return false; } - return selectProgrammer; - } + const boardDescriptor = this.boardManager.currentBoard.getBuildConfig(); - private getBoardBuildString(): string { - const selectedBoard = this.boardManager.currentBoard; - if (!selectedBoard) { - logger.notifyUserError("getBoardBuildString", new Error(constants.messages.NO_BOARD_SELECTED)); - return; + args.push("--board", boardDescriptor); + + if (!ArduinoWorkspace.rootPath) { + vscode.window.showWarningMessage("Workspace doesn't seem to have a folder added to it yet."); + return false; + } + + if (!dc.sketch || !util.fileExistsSync(path.join(ArduinoWorkspace.rootPath, dc.sketch))) { + if (mode == BuildMode.Analyze) { + // Analyze runs non interactively + return false; + } + if (!await dc.resolveMainSketch()) { + vscode.window.showErrorMessage("No sketch file was found. Please specify the sketch in the arduino.json file"); + return false; + } + } + + const selectSerial = async () => { + const choice = await vscode.window.showInformationMessage( + "Serial port is not specified. Do you want to select a serial port for uploading?", + "Yes", "No"); + if (choice === "Yes") { + vscode.commands.executeCommand("arduino.selectSerialPort"); + } + } + + if (mode === BuildMode.Upload) { + if ((!dc.configuration || !/upload_method=[^=,]*st[^,]*link/i.test(dc.configuration)) && !dc.port) { + await selectSerial(); + return false; + } + args.push("--upload"); + if (dc.port) { + args.push("--port", dc.port); + } + } else if (mode === BuildMode.UploadProgrammer) { + const programmer = this.programmerManager.currentProgrammer; + if (!programmer) { + logger.notifyUserError("getProgrammerString", new Error(constants.messages.NO_PROGRAMMMER_SELECTED)); + return false; + } + if (!dc.port) { + await selectSerial(); + return false; + } + args.push("--upload", + "--port", dc.port, + "--useprogrammer", + "--pref", "programmer=" + programmer); + + } else if (mode === BuildMode.Analyze) { + cocopa = makeCompilerParserContext(dc); + args.push("--verify", "--verbose"); + } else { + args.push("--verify"); + } + + const verbose = VscodeSettings.getInstance().logLevel === "verbose"; + if (mode !== BuildMode.Analyze && verbose) { + args.push("--verbose"); + } + + await vscode.workspace.saveAll(false); + + // we prepare the channel here since all following code will + // or at leas can possibly output to it + arduinoChannel.show(); + arduinoChannel.start(`${mode} sketch '${dc.sketch}'`); + + if (!await this.runPreBuildCommand(dc)) { + return false; } - return selectedBoard.getBuildConfig(); - } - private async getMainSketch(dc: DeviceContext) { - await dc.resolveMainSketch(); - if (!dc.sketch) { - vscode.window.showErrorMessage("No sketch file was found. Please specify the sketch in the arduino.json file"); - throw new Error("No sketch file was found."); + if (buildDir || dc.output) { + const outputPath = path.resolve(ArduinoWorkspace.rootPath, buildDir || dc.output); + const dirPath = path.dirname(outputPath); + if (!util.directoryExistsSync(dirPath)) { + logger.notifyUserError("InvalidOutPutPath", new Error(constants.messages.INVALID_OUTPUT_PATH + outputPath)); + return false; + } + + args.push("--pref", `build.path=${outputPath}`); + arduinoChannel.info(`Please see the build logs in output path: ${outputPath}`); + } else { + const msg = "Output path is not specified. Unable to reuse previously compiled files. Build will be slower. See README."; + arduinoChannel.warning(msg); + } + + // stop serial monitor when everything is prepared and good + // what makes restoring of its previous state easier + if (mode === BuildMode.Upload || mode === BuildMode.UploadProgrammer) { + restoreSerialMonitor = await SerialMonitor.getInstance().closeSerialMonitor(dc.port); + UsbDetector.getInstance().pauseListening(); } + + // Push sketch as last argument + args.push(path.join(ArduinoWorkspace.rootPath, dc.sketch)); + + let success = false; + + const cleanup = async () => { + if (cocopa) { + cocopa.conclude(); + } + if (mode === BuildMode.Upload || mode === BuildMode.UploadProgrammer) { + UsbDetector.getInstance().resumeListening(); + if (restoreSerialMonitor) { + await SerialMonitor.getInstance().openSerialMonitor(); + } + } + } + + // TODO: Get rid of spawn's channel parameter and just support + // stdout and stderr callbacks + const stdoutCallback = (line: string) => { + if (cocopa) { + cocopa.callback(line); + if (verbose) { + arduinoChannel.channel.append(line); + } + } else { + arduinoChannel.channel.append(line); + } + } + + await util.spawn( + this._settings.commandPath, + arduinoChannel.channel, + args, + undefined, + stdoutCallback, + ).then(async () => { + await cleanup(); + arduinoChannel.end(`${mode} sketch '${dc.sketch}${os.EOL}`); + success = true; + }, async (reason) => { + await cleanup(); + const msg = reason.code ? + `Exit with code=${reason.code}` : + reason.message ? + reason.message : + JSON.stringify(reason); + arduinoChannel.error(`${mode} sketch '${dc.sketch}': ${msg}${os.EOL}`); + }); + + return success; } } diff --git a/src/arduino/arduinoContentProvider.ts b/src/arduino/arduinoContentProvider.ts index 3623c4e9..83aebe98 100644 --- a/src/arduino/arduinoContentProvider.ts +++ b/src/arduino/arduinoContentProvider.ts @@ -208,7 +208,7 @@ export class ArduinoContentProvider implements vscode.TextDocumentContentProvide } else { try { // IS-REMOVE: to be removed completely when IntelliSense implementation is merged - //await ArduinoContext.arduinoApp.addLibPath(req.body.libraryPath); + // await ArduinoContext.arduinoApp.addLibPath(req.body.libraryPath); await ArduinoContext.arduinoApp.includeLibrary(req.body.libraryPath); return res.json({ status: "OK", diff --git a/src/arduino/boardManager.ts b/src/arduino/boardManager.ts index 1ed3000c..79af7780 100644 --- a/src/arduino/boardManager.ts +++ b/src/arduino/boardManager.ts @@ -73,7 +73,10 @@ export class BoardManager { this._boardConfigStatusBar.show(); const dc = DeviceContext.getInstance(); - dc.onDidChange(() => { + dc.onChangeBoard(() => { + this.updateStatusBar(); + }); + dc.onChangeConfiguration(() => { this.updateStatusBar(); }); } @@ -124,7 +127,7 @@ export class BoardManager { dc.configuration = this._currentBoard.customConfig; this._boardConfigStatusBar.text = targetBoard.name; // IS-REMOVE: to be removed completely when IntelliSense implementation is merged - //this._arduinoApp.addLibPath(null); + // this._arduinoApp.addLibPath(null); this._onBoardTypeChanged.fire(); } diff --git a/src/arduino/intellisense.ts b/src/arduino/intellisense.ts index e1b73b73..29181c3c 100644 --- a/src/arduino/intellisense.ts +++ b/src/arduino/intellisense.ts @@ -1,3 +1,6 @@ +// Copyright (c) Elektronik Workshop. All rights reserved. +// Licensed under the MIT license. + import * as ccp from "cocopa"; import * as path from "path"; @@ -23,7 +26,7 @@ export function isCompilerParserEnabled(dc?: DeviceContext) { dc = DeviceContext.getInstance(); } const globalDisable = VscodeSettings.getInstance().disableIntelliSenseAutoGen; - const projectSetting = dc.disableIntelliSenseAutoGen; + const projectSetting = dc.intelliSenseGen; return projectSetting !== "disable" && !globalDisable || projectSetting === "enable"; } @@ -93,6 +96,8 @@ function makeCompilerParserEngines(dc: DeviceContext) { const dotcpp = sketch.endsWith(".ino") ? ".cpp" : ""; sketch = `-o\\s+\\S*${ccp.regExEscape(sketch)}${dotcpp}\\.o`; + // TODO: handle other architectures here + const matchPattern = [ // make sure we're running g++ /(?:^|-)g\+\+\s+/, @@ -111,3 +116,192 @@ function makeCompilerParserEngines(dc: DeviceContext) { const gccParserEngine = new ccp.ParserGcc(matchPattern, dontMatchPattern); return [gccParserEngine]; } + +/** + * Possible states of AnalysisManager's state machine. + */ +enum AnalysisState { + /** + * No analysis request pending. + */ + Idle = "idle", + /** + * Analysis request pending. Waiting for the time out to expire or for + * another build to complete. + */ + Waiting = "waiting", + /** + * Analysis in progress. + */ + Analyzing = "analyzing", + /** + * Analysis in progress with yet another analysis request pending. + * As soon as the current analysis completes the manager will directly + * enter the Waiting state. + */ + AnalyzingWaiting = "analyzing and waiting", +} + +/** + * Events (edges) which cause state changes within AnalysisManager. + */ +enum AnalysisEvent { + /** + * The only external event. Requests an analysis to be run. + */ + AnalysisRequest, + /** + * The internal wait timeout expired. + */ + WaitTimeout, + /** + * The current analysis build finished. + */ + AnalysisBuildDone, +} + +/** + * This class manages analysis builds for the automatic IntelliSense + * configuration synthesis. Its primary purposes are: + * + * * delaying analysis requests caused by DeviceContext setting change + * events such that multiple subsequent requests don't cause + * multiple analysis builds + * * make sure that an analysis request is postponed when another build + * is currently in progress + * + * TODO: initialization sequence: make sure events generated + * during initialization are not lost + * + * TODO: check time of c_cpp_properties.json and compare it with + * * arduino.json + * * main sketch file + * This way we can perhaps optimize this further. But be aware + * that settings events fire before their corresponding values + * are actually written to arduino.json -> time of arduino.json + * is outdated if no countermeasure is taken. + */ +export class AnalysisManager { + + /** The manager's state. */ + private _state: AnalysisState = AnalysisState.Idle; + /** A callback used by the manager to query if the build backend is busy. */ + private _isBuilding: () => boolean; + /** A callback used by the manager to initiate an analysis build. */ + private _doBuild: () => Promise; + /** Timeout for the timeouts/delays in milliseconds. */ + private _waitPeriodMs: number; + /** The internal timer used to implement the above timeouts and delays. */ + private _timer: NodeJS.Timer; + + /** + * Constructor. + * @param isBuilding Provide a callback which returns true if another build + * is currently in progress. + * @param doBuild Provide a callback which runs the analysis build. + * @param waitPeriodMs The delay the manger should wait for potential new + * analysis request. This delay is used as polling interval as well when + * checking for ongoing builds. + */ + constructor(isBuilding:() => boolean, + doBuild: () => Promise, + waitPeriodMs: number = 1000) { + this._isBuilding = isBuilding; + this._doBuild = doBuild; + this._waitPeriodMs = waitPeriodMs; + } + + /** + * File an analysis request. + * The analysis will be delayed until no further requests are filed + * within a wait period or until any build in progress has terminated. + */ + public async requestAnalysis() { + console.log("requesting analysis"); + await this.update(AnalysisEvent.AnalysisRequest); + } + + /** + * Update the manager's state machine. + * @param event The event which will cause the state transition. + * + * Implementation note: asynchronous edge actions must be called after + * setting the new state since they don't return immediately. + */ + private async update(event: AnalysisEvent) { + + switch (this._state) { + + case AnalysisState.Idle: + if (event == AnalysisEvent.AnalysisRequest) { + this._state = AnalysisState.Waiting; + this.startWaitTimeout(); + } + break; + + case AnalysisState.Waiting: + if (event == AnalysisEvent.AnalysisRequest) { + // every new request restarts timer + this.startWaitTimeout(); + } else if (event == AnalysisEvent.WaitTimeout) { + if (this._isBuilding()) { + // another build in progress, continue waiting + this.startWaitTimeout(); + } else { + // no other build in progress -> launch analysis + this._state = AnalysisState.Analyzing; + await this.startAnalysis(); + } + } + break; + + case AnalysisState.Analyzing: + if (event == AnalysisEvent.AnalysisBuildDone) { + this._state = AnalysisState.Idle; + } else if (event == AnalysisEvent.AnalysisRequest) { + this._state = AnalysisState.AnalyzingWaiting; + } + break; + + case AnalysisState.AnalyzingWaiting: + if (event == AnalysisEvent.AnalysisBuildDone) { + // emulate the transition from idle to waiting + // (we don't care if this adds an additional + // timeout - event driven analysis is not time- + // critical) + this._state = AnalysisState.Idle; + await this.update(AnalysisEvent.AnalysisRequest); + } + break; + } + } + + /** + * Starts the wait timeout timer. + * If it's already running, the current timer is stopped and restarted. + * The timeout callback will then update the state machine. + */ + private startWaitTimeout() { + if (this._timer) { + clearTimeout(this._timer); + } + this._timer = setTimeout(() => { + this.update(AnalysisEvent.WaitTimeout); + this._timer = undefined; + }, this._waitPeriodMs); + } + + /** + * Starts the analysis build. + * When done, the callback will update the state machine. + */ + private async startAnalysis() { + await this._doBuild() + .then(() => { + this.update(AnalysisEvent.AnalysisBuildDone); + }) + .catch((reason) => { + this.update(AnalysisEvent.AnalysisBuildDone); + }); + } +} diff --git a/src/arduinoContext.ts b/src/arduinoContext.ts index 58a3f8a4..321d2857 100644 --- a/src/arduinoContext.ts +++ b/src/arduinoContext.ts @@ -19,6 +19,8 @@ class ArduinoContext { this._arduinoApp = value; } + // TODO EW: This is redundant: the board manager is already part of + // the arduino app public get boardManager() { return this._boardManager; } @@ -38,6 +40,10 @@ class ArduinoContext { return this._debuggerManager; } + // TODO EW: You don't have to initialize members to null + // if they don't get a default value or aren't initialized + // within a constructor they are "undefined" by default. + // This makes comparing against null (above) superfluous. private _arduinoApp: ArduinoApp = null; private _debuggerManager: DebuggerManager = null; private _boardManager: BoardManager = null; diff --git a/src/common/util.ts b/src/common/util.ts index e601dd2a..df45253b 100644 --- a/src/common/util.ts +++ b/src/common/util.ts @@ -267,7 +267,7 @@ export function tryParseJSON(jsonString: string) { } } catch (ex) { } - return false; + return undefined; } export function isJunk(filename: string): boolean { diff --git a/src/deviceContext.ts b/src/deviceContext.ts index ec7128db..210d4cc3 100644 --- a/src/deviceContext.ts +++ b/src/deviceContext.ts @@ -6,10 +6,10 @@ import * as path from "path"; import * as vscode from "vscode"; import * as constants from "./common/constants"; import * as util from "./common/util"; -import * as Logger from "./logger/logger"; import { ARDUINO_CONFIG_FILE } from "./common/constants"; import { ArduinoWorkspace } from "./common/workspace"; +import { DeviceSettings } from "./deviceSettings" /** * Interface that represents the arduino context information. @@ -60,9 +60,7 @@ export interface IDeviceContext { /** * IntelliSense configuration auto-generation project override. */ - disableIntelliSenseAutoGen: string; - - onDidChange: vscode.Event; + intelliSenseGen: string; initialize(): void; } @@ -75,22 +73,17 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { private static _deviceContext: DeviceContext = new DeviceContext(); - private _onDidChange = new vscode.EventEmitter(); - - private _port: string; - - private _board: string; - - private _sketch: string; - - private _output: string; - - private _debugger: string; - - private _disableIntelliSenseAutoGen: string; - - private _configuration: string; - + private _settings = new DeviceSettings(); + /** + * TODO EW, 2020-02-17: + * The absolute file path of the directory containing the vscode-arduino + * extension. Not sure why this is stored here (it's a bit misplaced) and + * not in a dedicated extension object containing the extension context + * passed during activation. Another way would be a function in util.ts + * using a mechanism like + * + * path.normalize(path.join(path.dirname(__filename), "..")) + */ private _extensionPath: string; private _watcher: vscode.FileSystemWatcher; @@ -99,10 +92,6 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { private _sketchStatusBar: vscode.StatusBarItem; - private _prebuild: string; - - private _programmer: string; - /** * @constructor */ @@ -143,39 +132,25 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { * TODO: Current we use the Arduino default settings. For future release, this dependency might be removed * and the setting only depends on device.json. * @method + * + * TODO EW, 2020-02-18: + * A problem I discovered: here you try to find the config file location + * and when you're writing below, you use a hardcoded location. When + * resorting to "find", you have to store the file's location at least and + * reuse it when saving. + * But I think the intention is: load a config file from anywhere and save + * it under .vscode/arduino.json. But then the initial load has to use find + * and afterwards it must not use it anymore. */ public loadContext(): Thenable { return vscode.workspace.findFiles(ARDUINO_CONFIG_FILE, null, 1) .then((files) => { - let deviceConfigJson: any = {}; if (files && files.length > 0) { - const configFile = files[0]; - deviceConfigJson = util.tryParseJSON(fs.readFileSync(configFile.fsPath, "utf8")); - if (deviceConfigJson) { - this._port = deviceConfigJson.port; - this._board = deviceConfigJson.board; - this._sketch = deviceConfigJson.sketch; - this._configuration = deviceConfigJson.configuration; - this._output = deviceConfigJson.output; - this._debugger = deviceConfigJson["debugger"]; - this._disableIntelliSenseAutoGen = deviceConfigJson.disableIntelliSenseAutoGen; - this._onDidChange.fire(); - this._prebuild = deviceConfigJson.prebuild; - this._programmer = deviceConfigJson.programmer; - } else { - Logger.notifyUserError("arduinoFileError", new Error(constants.messages.ARDUINO_FILE_ERROR)); - } + this._settings.load(files[0].fsPath); + // on invalid configuration we continue with current settings } else { - this._port = null; - this._board = null; - this._sketch = null; - this._configuration = null; - this._output = null; - this._debugger = null; - this._disableIntelliSenseAutoGen = null; - this._onDidChange.fire(); - this._prebuild = null; - this._programmer = null; + // No configuration file found, starting over with defaults + this._settings.reset(); } return this; }, (reason) => { @@ -185,148 +160,112 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { // Logger.notifyUserError("arduinoFileUnhandleError", new Error(reason.toString())); // Workaround for change in API, populate required props for arduino.json - this._port = null; - this._board = null; - this._sketch = null; - this._configuration = null; - this._output = null; - this._debugger = null; - this._disableIntelliSenseAutoGen = null; - this._onDidChange.fire(); - this._prebuild = null; - this._programmer = null; - + this._settings.reset(); return this; }); } public showStatusBar() { - if (!this._sketch) { + if (!this._settings.sketch.value) { return false; } - - this._sketchStatusBar.text = this._sketch; + this._sketchStatusBar.text = this._settings.sketch.value; this._sketchStatusBar.show(); } - public saveContext() { - if (!ArduinoWorkspace.rootPath) { - return; - } - const deviceConfigFile = path.join(ArduinoWorkspace.rootPath, ARDUINO_CONFIG_FILE); - let deviceConfigJson: any = {}; - if (util.fileExistsSync(deviceConfigFile)) { - deviceConfigJson = util.tryParseJSON(fs.readFileSync(deviceConfigFile, "utf8")); - } - if (!deviceConfigJson) { - Logger.notifyUserError("arduinoFileError", new Error(constants.messages.ARDUINO_FILE_ERROR)); - return; - } - deviceConfigJson.sketch = this.sketch; - deviceConfigJson.port = this.port; - deviceConfigJson.board = this.board; - deviceConfigJson.output = this.output; - deviceConfigJson["debugger"] = this.debugger_; - deviceConfigJson.disableIntelliSenseAutoGen = this.disableIntelliSenseAutoGen; - deviceConfigJson.configuration = this.configuration; - deviceConfigJson.programmer = this.programmer; - - util.mkdirRecursivelySync(path.dirname(deviceConfigFile)); - fs.writeFileSync(deviceConfigFile, JSON.stringify(deviceConfigJson, (key, value) => { - if (value === null) { - return undefined; - } - return value; - }, 4)); - } - - public get onDidChange(): vscode.Event { - return this._onDidChange.event; - } + public get onChangePort() { return this._settings.port.emitter.event } + public get onChangeBoard() { return this._settings.board.emitter.event } + public get onChangeSketch() { return this._settings.sketch.emitter.event } + public get onChangeOutput() { return this._settings.output.emitter.event } + public get onChangeDebugger() { return this._settings.debugger.emitter.event } + public get onChangeISAutoGen() { return this._settings.intelliSenseGen.emitter.event } + public get onChangeConfiguration() { return this._settings.configuration.emitter.event } + public get onChangePrebuild() { return this._settings.prebuild.emitter.event } + public get onChangeProgrammer() { return this._settings.programmer.emitter.event } public get port() { - return this._port; + return this._settings.port.value; } public set port(value: string) { - this._port = value; + this._settings.port.value = value; this.saveContext(); } public get board() { - return this._board; + return this._settings.board.value; } public set board(value: string) { - this._board = value; + this._settings.board.value = value; this.saveContext(); } public get sketch() { - return this._sketch; + return this._settings.sketch.value; } public set sketch(value: string) { - this._sketch = value; + this._settings.sketch.value = value; this.saveContext(); } public get prebuild() { - return this._prebuild ? this._prebuild.trim() : ""; + return this._settings.prebuild.value; } public get output() { - return this._output; + return this._settings.output.value; } public set output(value: string) { - this._output = value; + this._settings.output.value = value; this.saveContext(); } public get debugger_() { - return this._debugger; + return this._settings.debugger.value; } public set debugger_(value: string) { - this._debugger = value; + this._settings.debugger.value = value; this.saveContext(); } - public get disableIntelliSenseAutoGen() { - return this._disableIntelliSenseAutoGen; + public get intelliSenseGen() { + return this._settings.intelliSenseGen.value; } - public set disableIntelliSenseAutoGen(value: string) { - this._disableIntelliSenseAutoGen = value; + public set intelliSenseGen(value: string) { + this._settings.intelliSenseGen.value = value; this.saveContext(); } public get configuration() { - return this._configuration; + return this._settings.configuration.value; } public set configuration(value: string) { - this._configuration = value; + this._settings.configuration.value = value; this.saveContext(); } public get programmer() { - return this._programmer; + return this._settings.programmer.value; } public set programmer(value: string) { - this._programmer = value; + this._settings.programmer.value = value; this.saveContext(); } public async initialize() { if (ArduinoWorkspace.rootPath && util.fileExistsSync(path.join(ArduinoWorkspace.rootPath, ARDUINO_CONFIG_FILE))) { - vscode.window.showInformationMessage("Arduino.json is already generated."); + vscode.window.showInformationMessage("Arduino.json already generated."); return; } else { if (!ArduinoWorkspace.rootPath) { - vscode.window.showInformationMessage("Please open an folder first."); + vscode.window.showInformationMessage("Please open a folder first."); return; } await this.resolveMainSketch(); @@ -334,20 +273,41 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { await vscode.commands.executeCommand("arduino.changeBoardType"); vscode.window.showInformationMessage("The workspace is initialized with the Arduino extension support."); } else { - vscode.window.showInformationMessage("No *.ino sketch file was found or selected, so skip initialize command."); + vscode.window.showInformationMessage("No sketch (*.ino or *.cpp) was found or selected - initialization skipped."); } } } + /** + * Note: We're using the class' setter for the sketch (i.e. this.sketch = ...) + * to make sure that any changes are synched to the configuration file. + */ public async resolveMainSketch() { - return await vscode.workspace.findFiles("**/*.ino", null) + // TODO (EW, 2020-02-18): Here you look for *.ino files but below you allow + // *.cpp/*.c files to be set as sketch + await vscode.workspace.findFiles("**/*.ino", null) .then(async (fileUris) => { if (fileUris.length === 0) { let newSketchFileName = await vscode.window.showInputBox({ - value: "app.ino", - prompt: "No .ino file was found on workspace, initialize sketch first", - placeHolder: "Input the sketch file name", + value: "my-sketch.ino", + prompt: "No sketch (*.ino) found in workspace, please provide a name", + placeHolder: "Sketch file name (*.ino or *.cpp)", validateInput: (value) => { + /* TODO (EW, 2020-02-18): + * is 'c' actually allowed? Also found on within other files. + * And the regular expression doesn't need the intenal groups. + * The outer group can be an anonymous group. + * And \w doesn't match dashes - so any sketch containing dashes + * will not be found. + * The correct expression therefore would be something like this: + * + * /^[\w\-]+\.(?:ino|cpp)$/ + * + * I'd recommend to define such regular expressions (including) + * line splitting etc.) at the global constants file. + * This is true for any hard coded paths (like the snippets below) + * as well. + */ if (value && /^\w+\.((ino)|(cpp)|c)$/.test(value.trim())) { return null; } else { @@ -364,7 +324,7 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { const textDocument = await vscode.workspace.openTextDocument(path.join(ArduinoWorkspace.rootPath, newSketchFileName)); vscode.window.showTextDocument(textDocument, vscode.ViewColumn.One, true); } else { - this._sketch = undefined; + this.sketch = undefined; } } else if (fileUris.length === 1) { this.sketch = path.relative(ArduinoWorkspace.rootPath, fileUris[0].fsPath); @@ -380,5 +340,14 @@ export class DeviceContext implements IDeviceContext, vscode.Disposable { } } }); + return this.sketch; + } + + private saveContext() { + if (!ArduinoWorkspace.rootPath) { + return; + } + const deviceConfigFile = path.join(ArduinoWorkspace.rootPath, ARDUINO_CONFIG_FILE); + this._settings.save(deviceConfigFile); } } diff --git a/src/deviceSettings.ts b/src/deviceSettings.ts index e9281eac..fbefcc41 100644 --- a/src/deviceSettings.ts +++ b/src/deviceSettings.ts @@ -1,3 +1,6 @@ +// Copyright (c) Elektronik Workshop. All rights reserved. +// Licensed under the MIT license. + import * as fs from "fs"; import * as path from "path"; import * as vscode from "vscode"; @@ -106,7 +109,7 @@ export class DeviceSettings { sketch: StrSetting = new StrSetting(); output: StrSetting = new StrSetting(); debugger: StrSetting = new StrSetting(); - disableIntelliSenseAutoGen: StrSetting = new StrSetting(); + intelliSenseGen: StrSetting = new StrSetting(); configuration: StrSetting = new StrSetting(); prebuild: StrSetting = new StrSetting(); programmer: StrSetting = new StrSetting(); @@ -121,7 +124,7 @@ export class DeviceSettings { this.sketch.modified || this.output.modified || this.debugger.modified || - this.disableIntelliSenseAutoGen.modified || + this.intelliSenseGen.modified || this.configuration.modified || this.prebuild.modified || this.programmer.modified; @@ -135,7 +138,7 @@ export class DeviceSettings { this.sketch.commit(); this.output.commit(); this.debugger.commit(); - this.disableIntelliSenseAutoGen.commit(); + this.intelliSenseGen.commit(); this.configuration.commit(); this.prebuild.commit(); this.programmer.commit(); @@ -151,7 +154,7 @@ export class DeviceSettings { this.sketch.reset(); this.output.reset(); this.debugger.reset(); - this.disableIntelliSenseAutoGen.reset(); + this.intelliSenseGen.reset(); this.configuration.reset(); this.prebuild.reset(); this.programmer.reset(); @@ -176,7 +179,7 @@ export class DeviceSettings { this.configuration.value = settings.configuration; this.output.value = settings.output; this.debugger.value = settings.debugger; - this.disableIntelliSenseAutoGen.value = settings.disableIntelliSenseAutoGen; + this.intelliSenseGen.value = settings.intelliSenseGen; this.prebuild.value = settings.prebuild; this.programmer.value = settings.programmer; if (commit) { @@ -220,7 +223,7 @@ export class DeviceSettings { settings.board = this.board.value; settings.output = this.output.value; settings.debugger = this.debugger.value; - settings.disableIntelliSenseAutoGen = this.disableIntelliSenseAutoGen.value; + settings.intelliSenseGen = this.intelliSenseGen.value; settings.configuration = this.configuration.value; settings.programmer = this.programmer.value; diff --git a/src/extension.ts b/src/extension.ts index 58d0efcc..b0a734f6 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -31,8 +31,6 @@ const nsatModule = import { SerialMonitor } from "./serialmonitor/serialMonitor"; const usbDetectorModule = impor("./serialmonitor/usbDetector") as typeof import ("./serialmonitor/usbDetector"); -const status: any = {}; - export async function activate(context: vscode.ExtensionContext) { Logger.configure(context); const activeGuid = uuidModule().replace(/-/g, ""); @@ -115,18 +113,13 @@ export async function activate(context: vscode.ExtensionContext) { registerArduinoCommand("arduino.initialize", async () => await deviceContext.initialize()); registerArduinoCommand("arduino.verify", async () => { - if (!status.compile) { - status.compile = "verify"; - try { - await vscode.window.withProgress({ - location: vscode.ProgressLocation.Window, - title: "Arduino: Verifying...", - }, async () => { - await arduinoContextModule.default.arduinoApp.build(BuildMode.Verify); - }); - } catch (ex) { - } - delete status.compile; + if (!arduinoContextModule.default.arduinoApp.building) { + await vscode.window.withProgress({ + location: vscode.ProgressLocation.Window, + title: "Arduino: Verifying...", + }, async () => { + await arduinoContextModule.default.arduinoApp.build(BuildMode.Verify); + }); } }, () => { return { @@ -136,18 +129,13 @@ export async function activate(context: vscode.ExtensionContext) { }); registerArduinoCommand("arduino.upload", async () => { - if (!status.compile) { - status.compile = "upload"; - try { - await vscode.window.withProgress({ - location: vscode.ProgressLocation.Window, - title: "Arduino: Uploading...", - }, async () => { - await arduinoContextModule.default.arduinoApp.build(BuildMode.Upload); - }); - } catch (ex) { - } - delete status.compile; + if (!arduinoContextModule.default.arduinoApp.building) { + await vscode.window.withProgress({ + location: vscode.ProgressLocation.Window, + title: "Arduino: Uploading...", + }, async () => { + await arduinoContextModule.default.arduinoApp.build(BuildMode.Upload); + }); } }, () => { return { board: arduinoContextModule.default.boardManager.currentBoard.name }; @@ -175,44 +163,38 @@ export async function activate(context: vscode.ExtensionContext) { }); registerArduinoCommand("arduino.uploadUsingProgrammer", async () => { - if (!status.compile) { - status.compile = "upload"; - // TODO: no progress indicator - // and: exceptions should be handled within build - // function - try { + if (!arduinoContextModule.default.arduinoApp.building) { + await vscode.window.withProgress({ + location: vscode.ProgressLocation.Window, + title: "Arduino: Uploading (programmer)...", + }, async () => { await arduinoContextModule.default.arduinoApp.build(BuildMode.UploadProgrammer); - } catch (ex) { - } - delete status.compile; + }); } }, () => { return { board: arduinoContextModule.default.boardManager.currentBoard.name }; }); registerArduinoCommand("arduino.rebuildIntelliSenseConfig", async () => { - if (!status.compile) { - status.compile = "intellisenserebuild"; + if (!arduinoContextModule.default.arduinoApp.building) { await vscode.window.withProgress({ location: vscode.ProgressLocation.Window, title: "Arduino: Rebuilding IS Configuration...", }, async () => { await arduinoContextModule.default.arduinoApp.build(BuildMode.Analyze); }); - delete status.compile; } }, () => { return { board: arduinoContextModule.default.boardManager.currentBoard.name }; }); registerArduinoCommand("arduino.selectProgrammer", async () => { - if (!status.compile) { - status.compile = "upload"; + // TODO EW: this guard does not prevent building when setting the programmer + if (!arduinoContextModule.default.arduinoApp.building) { try { await arduinoContextModule.default.arduinoApp.programmerManager.selectProgrammer(); } catch (ex) { } - delete status.compile; } }, () => { return { @@ -222,7 +204,7 @@ export async function activate(context: vscode.ExtensionContext) { }); // IS-REMOVE: to be removed completely when IntelliSense implementation is merged - //registerArduinoCommand("arduino.addLibPath", (path) => arduinoContextModule.default.arduinoApp.addLibPath(path)); + // registerArduinoCommand("arduino.addLibPath", (path) => arduinoContextModule.default.arduinoApp.addLibPath(path)); registerArduinoCommand("arduino.openExample", (path) => arduinoContextModule.default.arduinoApp.openExample(path)); registerArduinoCommand("arduino.loadPackages", async () => await arduinoContextModule.default.boardManager.loadPackages(true)); registerArduinoCommand("arduino.installBoard", async (packageName, arch, version: string = "") => { @@ -278,7 +260,7 @@ export async function activate(context: vscode.ExtensionContext) { } arduinoContextModule.default.boardManager.updateStatusBar(true); // IS-REMOVE: to be removed completely when IntelliSense implementation is merged - //arduinoContextModule.default.arduinoApp.tryToUpdateIncludePaths(); + // arduinoContextModule.default.arduinoApp.tryToUpdateIncludePaths(); vscode.commands.executeCommand("setContext", "vscode-arduino:showExampleExplorer", true); })(); } diff --git a/src/langService/completionProvider.ts b/src/langService/completionProvider.ts index 91653c7f..1b5d9d31 100644 --- a/src/langService/completionProvider.ts +++ b/src/langService/completionProvider.ts @@ -14,7 +14,7 @@ import { ArduinoWorkspace } from "../common/workspace"; /** * Provides completions for library header includes. - * + * * NOTE: With the new IntelliSense auto-configuration this doesn't make * much sense in its current state, since it tries to fetch includes * from the wrongly guessed include paths. And it tries to fetch includes @@ -22,14 +22,13 @@ import { ArduinoWorkspace } from "../common/workspace"; * files already included -> therefore the user already included the header * and doesn't need a completion. Furthermore IntelliSense knows the location * as well and can complete it too. - * + * * To make this useful it has to parse the actual library folders and then * it makes only sense if it reads the library information and checks if * the individual libraries are actually compatible with the current board * before offering a completion. - * - * EW - * 2020-02-17 + * + * EW, 2020-02-17 */ export class CompletionProvider implements vscode.CompletionItemProvider { diff --git a/src/serialmonitor/serialMonitor.ts b/src/serialmonitor/serialMonitor.ts index c47a30c3..93fa24ac 100644 --- a/src/serialmonitor/serialMonitor.ts +++ b/src/serialmonitor/serialMonitor.ts @@ -54,18 +54,6 @@ export class SerialMonitor implements vscode.Disposable { private _ending: SerialPortEnding; - private constructor() { - const dc = DeviceContext.getInstance(); - dc.onDidChange(() => { - if (dc.port) { - if (!this.initialized) { - this.initialize(); - } - this.updatePortListStatus(null); - } - }); - } - public initialize() { const defaultBaudRate = ArduinoContext.arduinoApp.settings.defaultBaudRate || SerialMonitor.DEFAULT_BAUD_RATE; this._outputChannel = vscode.window.createOutputChannel(SerialMonitor.SERIAL_MONITOR); @@ -92,6 +80,11 @@ export class SerialMonitor implements vscode.Disposable { this._endingStatusBar.command = "arduino.changeEnding"; this._endingStatusBar.tooltip = "Serial Port Line Ending"; this._endingStatusBar.text = `No line ending`; + + const dc = DeviceContext.getInstance(); + dc.onChangePort(() => { + this.updatePortListStatus(null); + }); } public get initialized(): boolean { return !!this._outputChannel; @@ -192,15 +185,15 @@ export class SerialMonitor implements vscode.Disposable { const rates = SerialMonitor.listBaudRates(); const chosen = await vscode.window.showQuickPick(rates.map((rate) => rate.toString())); if (!chosen) { - Logger.warn("No rate is selected, keep baud rate no changed."); + Logger.warn("No baud rate selected, keeping previous baud rate."); return; } if (!parseInt(chosen, 10)) { - Logger.warn("Invalid baud rate, keep baud rate no changed.", { value: chosen }); + Logger.warn("Invalid baud rate, keeping previous baud rate.", { value: chosen }); return; } if (!this._serialPortCtrl) { - Logger.warn("Serial Monitor have not been started."); + Logger.warn("Serial Monitor has not been started."); return; } const selectedRate: number = parseInt(chosen, 10); @@ -237,6 +230,7 @@ export class SerialMonitor implements vscode.Disposable { } } + // TODO EW: use default value for port function parameter and change all updatePortListStatus(null) calls accordingly private updatePortListStatus(port: string) { const dc = DeviceContext.getInstance(); if (port) { diff --git a/src/serialmonitor/usbDetector.ts b/src/serialmonitor/usbDetector.ts index 14d1b515..d8d96e9a 100644 --- a/src/serialmonitor/usbDetector.ts +++ b/src/serialmonitor/usbDetector.ts @@ -74,6 +74,9 @@ export class UsbDetector { if (!SerialMonitor.getInstance().initialized) { SerialMonitor.getInstance().initialize(); } + + // TODO EW: this is board manager code which should be moved into board manager + let bd = ArduinoContext.boardManager.installedBoards.get(boardKey); if (!bd) { ArduinoContext.boardManager.updatePackageIndex(deviceDescriptor.indexFile).then((shouldLoadPackageContent) => { diff --git a/test/boardmanager.test.ts b/test/boardmanager.test.ts index 473628d5..561feee2 100644 --- a/test/boardmanager.test.ts +++ b/test/boardmanager.test.ts @@ -29,7 +29,7 @@ suite("Arduino: Board Manager.", () => { arduinoApp.setup((x) => x.setPref(TypeMoq.It.isAny(), TypeMoq.It.isAny())); arduinoApp.setup((x) => x.initialize(TypeMoq.It.isAny())); // IS-REMOVE: to be removed completely when IntelliSense implementation is merged - //arduinoApp.setup((x) => x.addLibPath(TypeMoq.It.isAny())); + // arduinoApp.setup((x) => x.addLibPath(TypeMoq.It.isAny())); try { boardManager = new BoardManager(arduinoSettings.object, arduinoApp.object); From 282d6299cdca4fc01933e184c3828ec7073b3cf6 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 19 Feb 2020 23:01:36 +0100 Subject: [PATCH 047/100] Updated log and beers. --- BRANCHNOTES.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 9394ea9d..48bd6ad3 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -69,7 +69,7 @@ During merging I found some bugs within those functions - mainly due to the abov **2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. **2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). -**2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. Updated and completed the documentation for the IntelliSense usage within [README](README.md). Alpha test builds of the extension containing the latest implemented features and fixes are now available from the following [Dropbox folder](https://www.dropbox.com/sh/whmcdt26chyjgby/AAB1Ld2fzZ9Z_NfM3CRay17wa). Please note, that Windows is currently not supported yet. +**2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. Updated and completed the documentation for the IntelliSense usage within [README](README.md). Alpha test builds of the extension containing the latest implemented features and fixes are now available from the following [Dropbox folder](https://www.dropbox.com/sh/whmcdt26chyjgby/AAB1Ld2fzZ9Z_NfM3CRay17wa). Please note, that Windows is currently not supported yet. Reviewed, documented/commented all changes and committed the automatic analysis integration changes. ## Status | | Tasks | @@ -146,7 +146,7 @@ I will list every supporter here, thanks! 2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) 2020-02-17 Elektronik Workshop: 52 :beers: (13h coding) 2020-02-18 Elektronik Workshop: 36 :beers: (9h coding) -2020-02-19 Elektronik Workshop: x :beers: (xh coding) +2020-02-19 Elektronik Workshop: 48 :beers: (12h coding) From 4f6409d5a489ad9909268a7279e353c58145787a Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Wed, 19 Feb 2020 23:59:01 +0100 Subject: [PATCH 048/100] Added "some" additional issues -- currently I've found 28! --- BRANCHNOTES.md | 51 ++++++++++++++++++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 48bd6ad3..06b1b2ed 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -3,26 +3,45 @@ This branch more or less addresses the following issues: | # | Issue | Title | Comment | |--:|:------|:------------|:--------| -| 1| [#438](https://github.com/microsoft/vscode-arduino/issues/438) | **The Extension should automagically fill the c_cpp_properties.json, so intellisense works out of the box** | This is the issue to which I usually report news concerning the progress of this project | -| 2| [#876](https://github.com/microsoft/vscode-arduino/issues/876) | **Missing #define ARDUINO 10808 logic?** | Marked as bug but it's just the same problem again | -| 3| [#839](https://github.com/microsoft/vscode-arduino/issues/829) | **Arduino.h and ESP8266 includes have squiggles** | Closed due to inactivity - not because it has been fixed | -| 4| [#969](https://github.com/microsoft/vscode-arduino/issues/969) | **INO Files defines undefined but can jump to definition** | | -| 5| [#959](https://github.com/microsoft/vscode-arduino/issues/959) | **Update board type command does not update intellisense config automatically** | | -| 6| [#892](https://github.com/microsoft/vscode-arduino/issues/892) | **Default IntelliSense config** | | -| 7| [#850](https://github.com/microsoft/vscode-arduino/issues/850) | **How to prevent modifications of c_cpp_properties.json by the extension?** | Asks if the current implementation can be turned off, because it overwrites a user's config with non working IS config -- this is sad. | -| 8| [#833](https://github.com/microsoft/vscode-arduino/issues/833) | **Allow C_Cpp.intelliSenseEngine to be set to "Default" instead of "Tag Parser" for better code completion/suggestions** | | -| 9| [#808](https://github.com/microsoft/vscode-arduino/issues/808) | **Identifier "Serial" is undefined** | | -| 10| [#474](https://github.com/microsoft/vscode-arduino/issues/474) | **Enrich device develop experience** | | - - - - - +| 1| [#438](https://github.com/microsoft/vscode-arduino/issues/438) | **The Extension should automagically fill the `c_cpp_properties.json`, so intellisense works out of the box** | This is the issue to which I usually report news concerning the progress of this project | +| 2| [#969](https://github.com/microsoft/vscode-arduino/issues/969) | **INO Files defines undefined but can jump to definition** | | +| 3| [#959](https://github.com/microsoft/vscode-arduino/issues/959) | **Update board type command does not update intellisense config automatically** | | +| 4| [#892](https://github.com/microsoft/vscode-arduino/issues/892) | **Default IntelliSense config** | | +| 5| [#876](https://github.com/microsoft/vscode-arduino/issues/876) | **Missing #define ARDUINO 10808 logic?** | Marked as bug but it's just the same problem again | +| 6| [#850](https://github.com/microsoft/vscode-arduino/issues/850) | **How to prevent modifications of `c_cpp_properties.json` by the extension?** | Asks if the current implementation can be turned off, because it overwrites a user's config with non working IS config -- this is sad. | +| 7| [#833](https://github.com/microsoft/vscode-arduino/issues/833) | **Allow C_Cpp.intelliSenseEngine to be set to "Default" instead of "Tag Parser" for better code completion/suggestions** | | +| 8| [#831](https://github.com/microsoft/vscode-arduino/issues/831) | **IntelliSenseEngine doesn't work as default** | | +| 9| [#839](https://github.com/microsoft/vscode-arduino/issues/829) | **`Arduino.h` and ESP8266 includes have squiggles** | Closed due to inactivity - not because it has been fixed | +| 10| [#823](https://github.com/microsoft/vscode-arduino/issues/823) | **Intellisense is not highlighting code** | Stale issue | +| 11| [#818](https://github.com/microsoft/vscode-arduino/issues/818) | **Warning with default includePath after initialize, cannot open source file `avr/pgmspace.h`** | Stale issue | +| 12| [#808](https://github.com/microsoft/vscode-arduino/issues/808) | **Identifier `Serial` is undefined** | | +| 13| [#776](https://github.com/microsoft/vscode-arduino/issues/776) | **Can not open source file `omp.h` (dependency of `ESP8266WiFi.h`** | | +| 14| [#772](https://github.com/microsoft/vscode-arduino/issues/772) | **How to fix red squiggles under constants like D2 (upload works fine)** | | +| 15| [#761](https://github.com/microsoft/vscode-arduino/issues/761) | **When creating the `c_cpp_properties.json` it should include the libraries folder as well as all of the other folders.** | | +| 16| [#749](https://github.com/microsoft/vscode-arduino/issues/749) | **Non-fatal error on Adafruit Feather M0: cannot open source file `sam.h`** | Stale issue | +| 17| [#727](https://github.com/microsoft/vscode-arduino/issues/727) | **Intellisense for Arduino Tabs** | Stale issue | +| 18| [#684](https://github.com/microsoft/vscode-arduino/issues/684) | **Default C/C++ configuration uses MSVC for IntelliSense rather than GCC** | Stale issue | +| 19| [#678](https://github.com/microsoft/vscode-arduino/issues/678) | **Dependency error `avr32/io.h`** | Stale issue | +| 20| [#645](https://github.com/microsoft/vscode-arduino/issues/645) | **Should IntelliSense suggest only built in function?** | Stale issue | +| 21| [#613](https://github.com/microsoft/vscode-arduino/issues/613) | **Read content of `keywords.txt` for text highlights** | This will become obsolete as well since `keywords.txt` is a dirty workaround for the arduino IDE which doesn't have IntelliSense at all. | +| 22| [#563](https://github.com/microsoft/vscode-arduino/issues/563) | **Support of intellisence for library Wire(sam)** | | +| 23| [#525](https://github.com/microsoft/vscode-arduino/issues/525) | **`#include` errors detected - Tag Parser.** | Closed but still not fixed properly | +| 24| [#474](https://github.com/microsoft/vscode-arduino/issues/474) | **Enrich device develop experience** | | +| 25| [#127](https://github.com/microsoft/vscode-arduino/issues/127) | **Syntax highlighting for some of the Classes/Instances is missing** | Closed but still not fixed properly | +| 26| [#126](https://github.com/microsoft/vscode-arduino/issues/126) | **Syntax highlighting for some of the Macros is missing** | Closed but still not fixed properly | +| 27| [#125](https://github.com/microsoft/vscode-arduino/issues/125) | **Missing syntax highlighting for partial arduino constants** | Closed but still not fixed properly | +| 28| [#115](https://github.com/microsoft/vscode-arduino/issues/115) | **Dot prompting the methods doesn't work on Mac** | Closed but still not fixed properly | + + + + + -- the list is probably incomplete - I didn't search exhaustively and didn't consider closed issues as long as I didn't stumble upon one. New duplicates are popping up at a rate of about one per week. -Further related issues +Some issue searches and other related issues * [vscode-arduino issue search for IntelliSense](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=intellisense+is%3Aopen) +* [vscode-arduino issue search for intellisense label](https://github.com/microsoft/vscode-arduino/issues?utf8=%E2%9C%93&q=label%3Aintellisense) * [Wrongly attributed to vscode instead of vscode-arduino](https://github.com/Microsoft/vscode-cpptools/issues/1750) * [Problems with IntelliSense itself](https://github.com/microsoft/vscode-cpptools/issues/1034) From 1009ed36c234521b79e75a0bca1a6344ec2f9bf0 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Thu, 20 Feb 2020 00:04:05 +0100 Subject: [PATCH 049/100] Fixed wrong issue number and adjusted comment --- BRANCHNOTES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 06b1b2ed..255020a8 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -11,7 +11,7 @@ This branch more or less addresses the following issues: | 6| [#850](https://github.com/microsoft/vscode-arduino/issues/850) | **How to prevent modifications of `c_cpp_properties.json` by the extension?** | Asks if the current implementation can be turned off, because it overwrites a user's config with non working IS config -- this is sad. | | 7| [#833](https://github.com/microsoft/vscode-arduino/issues/833) | **Allow C_Cpp.intelliSenseEngine to be set to "Default" instead of "Tag Parser" for better code completion/suggestions** | | | 8| [#831](https://github.com/microsoft/vscode-arduino/issues/831) | **IntelliSenseEngine doesn't work as default** | | -| 9| [#839](https://github.com/microsoft/vscode-arduino/issues/829) | **`Arduino.h` and ESP8266 includes have squiggles** | Closed due to inactivity - not because it has been fixed | +| 9| [#829](https://github.com/microsoft/vscode-arduino/issues/829) | **`Arduino.h` and ESP8266 includes have squiggles** | Stale issue | | 10| [#823](https://github.com/microsoft/vscode-arduino/issues/823) | **Intellisense is not highlighting code** | Stale issue | | 11| [#818](https://github.com/microsoft/vscode-arduino/issues/818) | **Warning with default includePath after initialize, cannot open source file `avr/pgmspace.h`** | Stale issue | | 12| [#808](https://github.com/microsoft/vscode-arduino/issues/808) | **Identifier `Serial` is undefined** | | From fd24b3c238d3769c4d572263d691f9e9cca846d5 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 21 Feb 2020 00:40:16 +0100 Subject: [PATCH 050/100] Updated log, status and beers --- BRANCHNOTES.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 255020a8..98d68c64 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -89,6 +89,7 @@ During merging I found some bugs within those functions - mainly due to the abov **2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. **2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). **2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. Updated and completed the documentation for the IntelliSense usage within [README](README.md). Alpha test builds of the extension containing the latest implemented features and fixes are now available from the following [Dropbox folder](https://www.dropbox.com/sh/whmcdt26chyjgby/AAB1Ld2fzZ9Z_NfM3CRay17wa). Please note, that Windows is currently not supported yet. Reviewed, documented/commented all changes and committed the automatic analysis integration changes. +**2020 02 20** Windows support - what a PITA. This OS is so foobar'ed... The only positive from this experience: I found some substantial bugs in the parser which - of course (Murphy) - didn't affect me up to now. The parser should be much more resistant against strange paths and escapes: Added proper command line lexer to cocopa and worked around several ridiculous Windows shortcomings (Microsoft owes me at least 50 crates of beer). The whole mess is not cleaned up and committed yet so please don't build from the repository and use the alpha release packages as outlined above. ## Status | | Tasks | @@ -164,8 +165,9 @@ I will list every supporter here, thanks! 2020-02-15 T.D.: 4 :beers: (20$ - Thanks a lot!) 2020-02-15 Elektronik Workshop: 28 :beers: (7h coding) 2020-02-17 Elektronik Workshop: 52 :beers: (13h coding) -2020-02-18 Elektronik Workshop: 36 :beers: (9h coding) -2020-02-19 Elektronik Workshop: 48 :beers: (12h coding) +2020-02-18 Elektronik Workshop: 36 :beers: (9h coding) +2020-02-19 Elektronik Workshop: 48 :beers: (12h coding) +2020-02-20 Elektronik Workshop: 56 :beers: (14h coding) From 19d34c1add69ba8bff7fa52dafe781ffcf3e793e Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 21 Feb 2020 00:41:51 +0100 Subject: [PATCH 051/100] Forgot to stage this additional edit - it's definitely too late --- BRANCHNOTES.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/BRANCHNOTES.md b/BRANCHNOTES.md index 98d68c64..5c7f2d81 100644 --- a/BRANCHNOTES.md +++ b/BRANCHNOTES.md @@ -89,7 +89,7 @@ During merging I found some bugs within those functions - mainly due to the abov **2020 02 17** Disabled and marked all previous implementations of IntelliSense support for later removal using `IS-REMOVE`. Pulled changes from upstream and merged them into the intellisense feature branch. Began to work on event handling/generation: vscode-arduino should detect when sketch/board/configuration and so on has changed, then re-analyze the current setup and set the IntelliSense configuration accordingly. This works more or less but there's a lot to fix in the current implementation which kept me busy till late today (I need some sleep now). Cleanup and commits follow tomorrow. Approaching alpha version for curious testers. OSX and Linux comes first, Windows will follow later. **2020 02 18** Finished basic event triggering. Rewrote `DeviceContext` for proper settings modification detection (trigger events only on actual change) and generation of setting specific events (e.g. board changed) instead of one global event (aka. "something in the settings changed"). **2020 02 19** Implemented proper build scheduling for analysis build by writing an `AnalysisManager` class. This class collects multiple changes (e.g. board and configuration, which often are changed shortly after another) before running an analysis. In case another build or analysis is in progress it postpones newly filed analysis requests until the other build has completed. Updated and completed the documentation for the IntelliSense usage within [README](README.md). Alpha test builds of the extension containing the latest implemented features and fixes are now available from the following [Dropbox folder](https://www.dropbox.com/sh/whmcdt26chyjgby/AAB1Ld2fzZ9Z_NfM3CRay17wa). Please note, that Windows is currently not supported yet. Reviewed, documented/commented all changes and committed the automatic analysis integration changes. -**2020 02 20** Windows support - what a PITA. This OS is so foobar'ed... The only positive from this experience: I found some substantial bugs in the parser which - of course (Murphy) - didn't affect me up to now. The parser should be much more resistant against strange paths and escapes: Added proper command line lexer to cocopa and worked around several ridiculous Windows shortcomings (Microsoft owes me at least 50 crates of beer). The whole mess is not cleaned up and committed yet so please don't build from the repository and use the alpha release packages as outlined above. +**2020 02 20** Windows support - what a PITA. This OS is so foobar'ed... The only positive outcome from this experience: I found some substantial bugs in the parser which - of course (Murphy) - didn't affect me up to now. The parser should be much more resistant against strange paths and escapes now: Added proper command line lexer to cocopa and worked around several ridiculous Windows shortcomings (Microsoft owes me at least 50 crates of beer). The whole mess is not cleaned up and committed yet so please don't build from the repository and use the alpha release packages as outlined above. ## Status | | Tasks | @@ -98,7 +98,7 @@ During merging I found some bugs within those functions - mainly due to the abov | | :heavy_check_mark: Support for different boards (done for AVR, ESP32, ESP8266) -- The code has been designed such that it is easy to write/add new parser engines (for non gcc compilers for instance) | | | :heavy_check_mark: Getting intrinsic gcc include paths | | | :heavy_check_mark: Handling quoted arguments | -| | :white_check_mark: X-platform support | +| | :heavy_check_mark: X-platform support | | **`c_cpp_properties.json` generator** | :heavy_check_mark: Basic objects | | | :heavy_check_mark: Basic setting of parsing result | | | :heavy_check_mark: Basic file input | From bb344cb65078178019a5e02106401aa3fb18b017 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Fri, 21 Feb 2020 21:45:52 +0100 Subject: [PATCH 052/100] Removed obsolete doc folder as all this is now part of cocopa --- .../compilerinfo/avr-gcc_built_in_specs.txt | 33 ------------------- .../xtensa-esp32-elf-gcc_built_in_specs.txt | 32 ------------------ 2 files changed, 65 deletions(-) delete mode 100644 doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt delete mode 100644 doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt diff --git a/doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt b/doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt deleted file mode 100644 index 950f0c06..00000000 --- a/doc/intellisense/compilerinfo/avr-gcc_built_in_specs.txt +++ /dev/null @@ -1,33 +0,0 @@ -Using built-in specs. -Reading specs from /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/device-specs/specs-avr2 -COLLECT_GCC=/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/avr-gcc -Target: avr -Configured with: ../gcc/configure --enable-fixed-point --enable-languages=c,c++ --prefix=/home/jenkins/workspace/avr-gcc-staging/label/debian7-x86_64/objdir --disable-nls --disable-libssp --disable-libada --disable-shared --with-avrlibc=yes --with-dwarf2 --disable-doc --target=avr -Thread model: single -gcc version 7.3.0 (GCC) -COLLECT_GCC_OPTIONS='-E' '-v' '-specs=device-specs/specs-avr2' - /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../libexec/gcc/avr/7.3.0/cc1plus -E -quiet -v -iprefix /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/ - -mn-flash=6 -mskip-bug -mn-flash=6 -mskip-bug -fno-rtti -fno-enforce-eh-specs -fno-exceptions -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/avr" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/backward" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/sys-include" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/avr" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include/c++/7.3.0/backward" -ignoring duplicate directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/include" -ignoring duplicate directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/include-fixed" -ignoring nonexistent directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/sys-include" -ignoring duplicate directory "/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/../../lib/gcc/avr/7.3.0/../../../../avr/include" -#include "..." search starts here: -#include <...> search starts here: - /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/include - /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/include-fixed - /home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/include -End of search list. -# 1 "" -# 1 "" -# 1 "" -# 1 "" -COMPILER_PATH=/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../libexec/gcc/avr/7.3.0/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../libexec/gcc/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/bin/ -LIBRARY_PATH=/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/:/home/uli/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino5/bin/../lib/gcc/avr/7.3.0/../../../../avr/lib/ -COLLECT_GCC_OPTIONS='-E' '-v' '-specs=device-specs/specs-avr2' diff --git a/doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt b/doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt deleted file mode 100644 index 7e39fe2e..00000000 --- a/doc/intellisense/compilerinfo/xtensa-esp32-elf-gcc_built_in_specs.txt +++ /dev/null @@ -1,32 +0,0 @@ -Using built-in specs. -COLLECT_GCC=/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc -Target: xtensa-esp32-elf -Configured with: /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/configure --build=x86_64-build_pc-linux-gnu --host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf --prefix=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf --with-local-prefix=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/sysroot --with-sysroot=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/sysroot --with-newlib --enable-threads=no --disable-shared --with-pkgversion='crosstool-NG crosstool-ng-1.22.0-80-g6c4433a' --disable-__cxa_atexit --enable-cxx-flags='-fno-rtti -ffunction-sections' --with-gmp=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-mpfr=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-mpc=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-isl=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-cloog=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --with-libelf=/builds/idf/crosstool-NG/.build/xtensa-esp32-elf/buildtools --enable-lto --enable-target-optspace --without-long-double-128 --disable-libgomp --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libquadmath-support --disable-nls --enable-languages=c,c++ --disable-libstdcxx-verbose --enable-threads=posix --enable-gcov-custom-rtio -Thread model: posix -gcc version 5.2.0 (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) -COLLECT_GCC_OPTIONS='-E' '-v' - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../libexec/gcc/xtensa-esp32-elf/5.2.0/cc1plus -E -quiet -v -iprefix /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/ -isysroot /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot - -ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0" -ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/xtensa-esp32-elf" -ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/backward" -ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/include" -ignoring nonexistent directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/sysroot/include" -ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/include-fixed" -ignoring duplicate directory "/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/../../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include" -#include "..." search starts here: -#include <...> search starts here: - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0 - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/xtensa-esp32-elf - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include/c++/5.2.0/backward - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/include - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/include-fixed - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/include - /home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/usr/include -End of search list. -# 1 "" -# 1 "" -# 1 "" -# 1 "" -COMPILER_PATH=/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../libexec/gcc/xtensa-esp32-elf/5.2.0/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../libexec/gcc/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ -LIBRARY_PATH=/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/lib/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/lib/:/home/uli/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../xtensa-esp32-elf/sysroot/usr/lib/ -COLLECT_GCC_OPTIONS='-E' '-v' From c7ae8497bf1b4978a83ba9fe5f4572f5560a74d4 Mon Sep 17 00:00:00 2001 From: Uli Franke Date: Sat, 22 Feb 2020 14:25:22 +0100 Subject: [PATCH 053/100] Fixed handling of invalid manual configurations. Improved code locality for board configuration manipulation and fixed a race condition with fine grained event handling. Details: ## Handling of invalid manual configurations Added missing checks when board configurations are loaded from the configuration file: * Up to now vscode-arduino blindly loaded any board configuration from `arduino.json` even if this would result in invalid board configurations which in turn lead to compilation (verify, upload) failure. * Up to now this state couldn't be recovered by simply removing the offending configuration from the configuration file. Even worse: it stored the wrong configuration in between board changes. To reproduce the bug in 0.2.29 1. Select Arduino Nano with the *Arduino Board Configuration* 2. Set configuration in `arduino.json` to `cpu=cray2` and save 3. Verify -> fails 4. Switch board to Arduino Uno 5. Switch back to Arduino Nano: The wrong configuration is back and now the user can't even select another (correct) configuration from the *Arduino Board Configuration* window 6. Delete the wrong configuration and save -> verify still fails `vscode-arduino` does not fall back to a default configuration. The user has now two options: find the correct configuration by himself and set it within arduino.json. Very experienced users could probably accomplish that. Everone else can just restart vscode. I corrected that by enhancing IBoard.loadConfig and IBoard.updateConfig member functions to * check for proper formatting of the config string loaded from `arduino.json` * check if the configuration IDs and the option IDs are valid If any of the above fails, the functions bail out and return the error. The board manager then loads a default configuration and issues a warning, that the configuration is invalid. This way the user gets the chance to fix her/his configuration but gets informed at the same time, that a different configuration than the intended is loaded (prevents surprises). This situation is only relevant, when users start fiddling with the configuration values in `arduino.json`. As long as they just set the board and the configurations from within the *Arduino Board Configuration Window* nothing bad can happen. But now custom configurations are handled in a cleaner way. ## Code locality for configuration manipulation The DeviceContext's board configuration was set in board.ts and boardManager.ts in different places - even when it was loaded after a DeviceContext's configuration update event which is prone to infinite loops. This has been resolved and it's not re-written/re-set during loading a configuration on change. This is valid for board manager's updateStatusBar function which fiddled with the board and the configuration. Now updateStatusBar really just updates the status bar. And it isn't necessary to call it from outside the board manager anymore due to proper event handling which identifies the situations during which the status bar has to be updated. Therefore this member is now private. In board manager itself operations that affect device context and current board now happen only within doChangeBoardType and the event handlers of DeviceContext callbacks onDeviceContextConfigurationChange and onDeviceContextBoardChange. This prevents the accidental creation of infinite event loops, makes the code more understandable, maintainable and therefore resilient against future bugs. --- src/arduino/arduinoContentProvider.ts | 3 + src/arduino/board.ts | 92 ++++++++++++++----- src/arduino/boardManager.ts | 125 ++++++++++++++++++++------ src/extension.ts | 2 - 4 files changed, 172 insertions(+), 50 deletions(-) diff --git a/src/arduino/arduinoContentProvider.ts b/src/arduino/arduinoContentProvider.ts index 83aebe98..b67fc689 100644 --- a/src/arduino/arduinoContentProvider.ts +++ b/src/arduino/arduinoContentProvider.ts @@ -11,6 +11,7 @@ import * as JSONHelper from "../common/cycle"; import * as Logger from "../logger/logger"; import LocalWebServer from "./localWebServer"; import { VscodeSettings } from "./vscodeSettings"; +import { DeviceContext } from "../deviceContext"; export class ArduinoContentProvider implements vscode.TextDocumentContentProvider { private _webserver: LocalWebServer; @@ -263,6 +264,8 @@ export class ArduinoContentProvider implements vscode.TextDocumentContentProvide } else { try { ArduinoContext.boardManager.currentBoard.updateConfig(req.body.configId, req.body.optionId); + const dc = DeviceContext.getInstance(); + dc.configuration = ArduinoContext.boardManager.currentBoard.customConfig; return res.json({ status: "OK", }); diff --git a/src/arduino/board.ts b/src/arduino/board.ts index 88c1b6c8..44baab54 100644 --- a/src/arduino/board.ts +++ b/src/arduino/board.ts @@ -1,8 +1,7 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT license. -import { DeviceContext } from "../deviceContext"; -import { IBoard, IBoardConfigItem, IPlatform } from "./package"; +import { IBoard, IBoardConfigItem, IPlatform, BoardConfigResult } from "./package"; export function parseBoardDescriptor(boardDescriptor: string, plat: IPlatform): Map { const boardLineRegex = /([^\.]+)\.(\S+)=(.+)/; @@ -52,15 +51,15 @@ export class Board implements IBoard { this._configItems = []; } - public get board(): string { + public get board() { return this._board; } - public get platform(): IPlatform { + public get platform() { return this._platform; } - public addParameter(key: string, value: string): void { + public addParameter(key: string, value: string) { const match = key.match(MENU_REGEX); if (match) { const existingItem = this._configItems.find((item) => item.id === match[1]); @@ -82,8 +81,7 @@ export class Board implements IBoard { } } } - - public getBuildConfig(): string { + public getBuildConfig() { return `${this.getPackageName()}:${this.platform.architecture}:${this.board}${this.customConfig ? ":" + this.customConfig : ""}`; } @@ -100,36 +98,86 @@ export class Board implements IBoard { } } - public get configItems(): IBoardConfigItem[] { + public get configItems() { return this._configItems; } - public loadConfig(configString: string): void { + public loadConfig(configString: string) { + // An empty or undefined config string resets the configuration + if (!configString) { + this.resetConfig(); + return BoardConfigResult.Success; + } const configSections = configString.split(","); const keyValueRegex = /(\S+)=(\S+)/; - configSections.forEach((configSection) => { - const match = configSection.match(keyValueRegex); - if (match && match.length >= 2) { - this.updateConfig(match[1], match[2]); + let result = BoardConfigResult.Success; + for (const section of configSections) { + const match = section.match(keyValueRegex); + if (!match) { + return BoardConfigResult.InvalidFormat; } - }); + const r = this.updateConfig(match[1], match[2]); + switch (r) { + case BoardConfigResult.SuccessNoChange: + result = r; + break; + case BoardConfigResult.Success: + break; + default: + return r; + } + }; + return result; } - public updateConfig(configId: string, optionId: string): boolean { + /** + * For documentation see the documentation on IBoard.updateConfig(). + */ + public updateConfig(configId: string, optionId: string) { const targetConfig = this._configItems.find((config) => config.id === configId); if (!targetConfig) { - return false; + return BoardConfigResult.InvalidConfigID; + } + // Iterate through all options and ... + for (const o of targetConfig.options) { + // Make sure that we only set valid options, e.g. when loading + // from config files. + if (o.id == optionId) { + if (targetConfig.selectedOption !== optionId) { + targetConfig.selectedOption = optionId; + return BoardConfigResult.Success; + } + return BoardConfigResult.SuccessNoChange; + } else { + return BoardConfigResult.InvalidOptionID; + } } - if (targetConfig.selectedOption !== optionId) { - targetConfig.selectedOption = optionId; - const dc = DeviceContext.getInstance(); - dc.configuration = this.customConfig; - return true; + return BoardConfigResult.InvalidOptionID; + } + + public resetConfig() { + for (const c of this._configItems) { + c.selectedOption = c.options[0].id; } - return false; } public getPackageName() { return this.platform.packageName ? this.platform.packageName : this.platform.package.name; } } + +/** + * Test if two boards are of the same type, i.e. have the same key. + * @param {IBoard | undefined} a A board. + * @param {IBoard | undefined} b And another board. + * @returns {boolean} true if two boards are of the same type, else false. + */ +export function boardEqual(a: IBoard | undefined, + b: IBoard | undefined) { + if (a && b) { + return a.key === b.key; + } else if (a || b) { + return false; + } + return true; +} diff --git a/src/arduino/boardManager.ts b/src/arduino/boardManager.ts index 79af7780..083ad523 100644 --- a/src/arduino/boardManager.ts +++ b/src/arduino/boardManager.ts @@ -12,8 +12,8 @@ import { arduinoChannel } from "../common/outputChannel"; import { DeviceContext } from "../deviceContext"; import { ArduinoApp } from "./arduino"; import { IArduinoSettings } from "./arduinoSettings"; -import { parseBoardDescriptor } from "./board"; -import { IBoard, IPackage, IPlatform } from "./package"; +import { boardEqual, parseBoardDescriptor } from "./board"; +import { BoardConfigResult, IBoard, IPackage, IPlatform } from "./package"; import { VscodeSettings } from "./vscodeSettings"; export class BoardManager { @@ -67,18 +67,17 @@ export class BoardManager { // Load default platforms from arduino installation directory and user manually installed platforms. this.loadInstalledPlatforms(); - // Load all supported boards type. + // Load all supported board types this.loadInstalledBoards(); - this.updateStatusBar(); - this._boardConfigStatusBar.show(); const dc = DeviceContext.getInstance(); - dc.onChangeBoard(() => { - this.updateStatusBar(); - }); - dc.onChangeConfiguration(() => { - this.updateStatusBar(); - }); + dc.onChangeBoard(() => this.onDeviceContextBoardChange()); + dc.onChangeConfiguration(() => this.onDeviceContextConfigurationChange()); + + // load initial board from DeviceContext by emulating + // a board change event. + this.onDeviceContextBoardChange(); + this.updateStatusBar(true); } public async changeBoardType() { @@ -122,14 +121,24 @@ export class BoardManager { public doChangeBoardType(targetBoard: IBoard) { const dc = DeviceContext.getInstance(); + + if (dc.board === targetBoard.key) { + return; + } + + // Resetting the board first that we don't overwrite the configuration + // of the previous board. + this._currentBoard = null; + // This will cause a configuration changed event which will have no + // effect because no current board is set. + dc.configuration = targetBoard.customConfig; + // This will generate a device context board event which will set the + // correct board and configuration. We know that it will trigger - we + // made sure above that the boards actually differ dc.board = targetBoard.key; - this._currentBoard = targetBoard; - dc.configuration = this._currentBoard.customConfig; - this._boardConfigStatusBar.text = targetBoard.name; + // IS-REMOVE: to be removed completely when IntelliSense implementation is merged // this._arduinoApp.addLibPath(null); - - this._onBoardTypeChanged.fire(); } public get packages(): IPackage[] { @@ -251,19 +260,12 @@ export class BoardManager { } } - public updateStatusBar(show: boolean = true): void { + private updateStatusBar(show: boolean = true): void { if (show) { this._boardConfigStatusBar.show(); - const dc = DeviceContext.getInstance(); - const selectedBoard = this._boards.get(dc.board); - if (selectedBoard) { - this._currentBoard = selectedBoard; - this._boardConfigStatusBar.text = selectedBoard.name; - if (dc.configuration) { - this._currentBoard.loadConfig(dc.configuration); - } + if (this._currentBoard) { + this._boardConfigStatusBar.text = this._currentBoard.name; } else { - this._currentBoard = null; this._boardConfigStatusBar.text = "