Skip to content

Daemon mode (GRPC) + pluggable discovery #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 169 commits into from
Jun 3, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
169 commits
Select commit Hold shift + click to select a range
6eb7f2d
Some minor changes to readme
cmaglie Jan 29, 2019
2772438
InitLibraryManager now requires a Configuration
cmaglie Feb 8, 2019
57e0cda
go-Formatted some imports
cmaglie Feb 8, 2019
46d175e
Made commands.lib.ListLibraries public
cmaglie Feb 8, 2019
dfb3720
Made configs.Configuration.ArduinoDownloadsDir field public
cmaglie Feb 8, 2019
94e5765
Added daemon mode (WIP)
cmaglie Feb 8, 2019
015f0d7
Big refactoring for daemon mode
cmaglie Feb 11, 2019
12868a5
Deamon implementation and refactoring
cmaglie Feb 22, 2019
c6ad703
Set sketchbookdir config during init
cmaglie Feb 27, 2019
0f18110
cli.Init* funcitons now have a more meaningful name
cmaglie Feb 27, 2019
ec4ff4e
library_index.json is automatically updated on Init (again)
cmaglie Feb 27, 2019
c226f62
Use config.NewConfiguration to setup a new configuration
cmaglie Feb 27, 2019
76198da
Better client example
cmaglie Mar 1, 2019
875ec10
Moved compile module under cli
cmaglie Mar 23, 2019
fb8da3c
Added Compile rpc command
cmaglie Mar 25, 2019
3277596
Rebuilt grpc bindings
cmaglie Mar 25, 2019
08c0a66
Added daemon callback for Compile rpc call
cmaglie Mar 25, 2019
51dd95f
Moved "Compile" implementation from cli to commands (WIP)
cmaglie Mar 25, 2019
4408123
Use rpc fields to run Compile command
cmaglie Mar 25, 2019
b4f7ea4
Minor fix added return value to the compile function in command/compile
Rocketct Mar 25, 2019
9dcf531
Implemented run function in cli/compile
Rocketct Mar 25, 2019
6555b10
Implemented Client "Compile" call on Daemon/Client
Rocketct Mar 27, 2019
7ba912f
Removed 'Result' message from grpc .proto
cmaglie Mar 27, 2019
196e956
output: fixed divide by 0 in some cases
cmaglie Mar 5, 2019
c13d9fd
Using grpc status errors instead of embedding errors in messages
cmaglie Mar 28, 2019
034a193
Use normal errors from 'command'
cmaglie Mar 28, 2019
3ba5f72
Stream builder output in Compile command
cmaglie Apr 1, 2019
79aa5a5
Set correct output io.Writer during Compile
cmaglie Apr 1, 2019
63969a4
Moved core module under cli
Rocketct Apr 1, 2019
5bb021f
Added Install rpc command
Rocketct Apr 1, 2019
1801430
Rebuilt grpc bindings
Rocketct Apr 1, 2019
74998fe
Added daemon callback for Install rpc call
Rocketct Apr 1, 2019
8e7db44
Moved "PlatformInstall" implementation from cli to commands (WIP)
Rocketct Apr 2, 2019
75b755a
Porting of command/core/download
Rocketct Apr 2, 2019
ec10ca1
Porting of command/core/uninstall
Rocketct Apr 2, 2019
13a5c97
Modified proto file and removed unused file in command/core
Rocketct Apr 2, 2019
344cb7c
Fixed commands return value and moved update index in pakageindex
Rocketct Apr 3, 2019
3e34dbd
Error generation from UpdateIndex
Rocketct Apr 3, 2019
fd5334c
modified return value for install and uninstall
Rocketct Apr 4, 2019
22113fe
Added return to uninstall command
Rocketct Apr 4, 2019
c7d3999
WIP: get feedback for download progress
cmaglie Apr 4, 2019
100f309
Fixed empty version handling
cmaglie Apr 4, 2019
ea2038a
another WIP
cmaglie Apr 4, 2019
b7fb8ce
Merge branch 'daemon' of github.com:Rocketct/arduino-cli into HEAD
Rocketct Apr 4, 2019
3796d73
Implemented core commands return value management
Rocketct Apr 4, 2019
a7e256f
Rebuilt grpc bindings
Rocketct Apr 4, 2019
51e89fd
Moved PlatformUpgrade implementation from cli to commands
Rocketct Apr 4, 2019
4079544
Changed return management for download, install and upgrade command
Rocketct Apr 4, 2019
83eddd1
Fixed progress bar in CLI mode
cmaglie Apr 4, 2019
ac2c924
Removed unused file
cmaglie Apr 5, 2019
7f809ce
Core Install and progress notification on GRPC daemon
cmaglie Apr 5, 2019
4b248bc
added Rescan function
Rocketct Apr 9, 2019
0f4666e
Implemented createInstance function
Rocketct Apr 9, 2019
9ac0936
Rescan is called automatically after state-changing operations
cmaglie Apr 10, 2019
d4395bc
makeup: changed visibility of methods and local var caps
cmaglie Apr 10, 2019
824c82a
Fixed a bunch of lint warnings
cmaglie Apr 10, 2019
684a37b
Removing dependency commands->formatter
cmaglie Apr 10, 2019
693036c
Implemented TaskProgress, replacing formatter module
cmaglie Apr 11, 2019
95ab26e
Added GRPC feedback on TaskProgress
cmaglie Apr 11, 2019
e78cd8e
Correctly get PackageManager in commands
cmaglie Apr 11, 2019
11483db
Added some more TaskProgress feedback
cmaglie Apr 11, 2019
01cef30
Ported Update command in cli-daemon
Rocketct Apr 11, 2019
d445531
Added task progress streaming to 'core uninstall' commands
cmaglie Apr 11, 2019
b1b837d
changed errors messages
Rocketct Apr 11, 2019
58cde92
Added DownloadProgress in Compile
cmaglie Apr 11, 2019
103a7c3
fixed two small error messages
cmaglie Apr 11, 2019
49d678d
Renamed ProgressCB -> DownloadProgressCB; other small makeups
cmaglie Apr 11, 2019
93819d1
Simplified pipe-streamer creation in GRPC daemon
cmaglie Apr 11, 2019
f895ae5
Fixed names in .proto
cmaglie Apr 11, 2019
c591d4d
Fully redirecting stdout/err from Compile command
cmaglie Apr 12, 2019
3024f43
Fixed 'core download' call from cli
cmaglie Apr 12, 2019
ec68c6d
Porting update_index command
Rocketct Apr 12, 2019
43f8906
Merge branch 'daemon' of github.com:cmaglie/arduino-cli into HEAD
Rocketct Apr 12, 2019
b320404
Minor fix and comment removed
Rocketct Apr 12, 2019
890546e
porting of list and search core command
Rocketct Apr 15, 2019
554160c
Added output to new 'core list' command
cmaglie Apr 16, 2019
494a924
Added output to new 'core search' command
Rocketct Apr 16, 2019
26c6ef4
Removed unused core_Structure file
Rocketct Apr 16, 2019
c11c8bd
Removed streaming response from 'core saerch/list' commands
cmaglie Apr 16, 2019
f6b75c2
Cleaned up all 'core' commands/cli modules
cmaglie Apr 16, 2019
e01d5b5
Added license text
cmaglie Apr 16, 2019
8bc318e
Added license to protoc files
cmaglie Apr 16, 2019
afe0a4f
Small fix to daemon/client
cmaglie Apr 16, 2019
1711a26
Download progress now prints 'already downloaded' when appropriate
cmaglie Apr 16, 2019
89d1780
Porting of command lib/download
Rocketct Apr 16, 2019
518a332
Fixed some error messages
cmaglie Apr 16, 2019
d473179
instance Init now handle missing indexes
cmaglie Apr 17, 2019
695f23c
Improved client.go GRPC example
cmaglie Apr 17, 2019
5df31b9
Merge branch 'daemon' of github.com:cmaglie/arduino-cli into HEAD
Rocketct Apr 18, 2019
da8abe6
Porting of install and unistall lib command
Rocketct Apr 18, 2019
0460549
Removed duplicates leftovers
cmaglie Apr 18, 2019
d32d6c0
Fixed progress bar in library index download
cmaglie Apr 19, 2019
f200965
Updated some comments
cmaglie Apr 19, 2019
c374f93
Output progress status only in text mode
cmaglie Apr 19, 2019
d3f36e4
Fixed regression
cmaglie Apr 19, 2019
f59e20e
Fixed output (that made the tests fail)
cmaglie Apr 19, 2019
7399a45
fixed another test
cmaglie Apr 19, 2019
3b877f8
fixed regression: accept empty version in 'core install'
cmaglie Apr 19, 2019
2d5f68a
output: Added some helpers and missing license
cmaglie Apr 19, 2019
3da06d7
fix regressions: use correct exit code when error occurs
cmaglie Apr 19, 2019
b5cd987
Implemented RPC call LibraryInstall
cmaglie Apr 24, 2019
77f4efe
Initialize librarymanager with an empty Index
cmaglie Apr 24, 2019
d53d3c2
Implemented RPC call LibraryUninstall
cmaglie Apr 24, 2019
d880e28
Added helper function to extract parsed version from RPC requests
cmaglie Apr 24, 2019
a3ddf57
factored out some commands/lib helpers
cmaglie Apr 26, 2019
c375d33
Fixed nil GRPC response
cmaglie Apr 29, 2019
dc9e5e0
Implemented GRPC LibraryUpgradeAll command
cmaglie Apr 29, 2019
e192f39
Fix test
cmaglie Apr 30, 2019
1cb56ac
more error handling, fixed more tests
cmaglie Apr 30, 2019
bd5e787
Removed 'version' argument from PlatformUpgrade
cmaglie Apr 30, 2019
472bfb5
Put all rpc go-generation in a single file
cmaglie May 3, 2019
f7128ff
porting command lib/search (WIP)
Rocketct Apr 30, 2019
e8a1a28
fix lib search output
Rocketct May 3, 2019
9910ea7
GRPC: preparing for java bindings
cmaglie May 3, 2019
af3eae7
porting of lib/list command to daemon
Rocketct May 7, 2019
7a51b99
Fixed sketchPath and *Sketch creation
cmaglie May 7, 2019
274e217
Small makeup to GRPC client example
cmaglie May 7, 2019
1a1f21f
Correct typos in Client.go error messages
per1234 May 6, 2019
eab7ad5
Added GRPC UpdateLibrariesIndex command
cmaglie May 10, 2019
bdf3772
Do not bail out for missing 'core' index on 'lib' commands
cmaglie May 12, 2019
eb65649
Fixing test: solving regression in lib search output
cmaglie May 13, 2019
fb93127
Fixing test: solving regression in lib search output (part 2)
cmaglie May 13, 2019
df44d15
Fixing test: solving regression in lib search output (part 3)
cmaglie May 13, 2019
584d657
Moved commands/root module on more appropriate cli/root
cmaglie May 13, 2019
9fc2db6
Fix tests: additional core urls
cmaglie May 13, 2019
47e974c
Fixed cli initialization subroutines, maybe this is the right one
cmaglie May 13, 2019
37cc5ed
small fix
cmaglie May 13, 2019
f11373b
Merge tag '0.3.6-alpha.preview' into daemon
cmaglie May 15, 2019
6f90002
Merge branch 'master' into daemon
cmaglie May 15, 2019
6bb9425
Porting attach command(WIP)
Rocketct May 16, 2019
ad9cee9
Merge branch 'master' of https://github.com/arduino/arduino-cli into …
Rocketct May 16, 2019
e94a344
add Attach command stream support
Rocketct May 16, 2019
74042f1
Fixed UpdateIndex regression
cmaglie May 17, 2019
1806457
Use PlatformRelease.IsInstalled() method when appropriate
cmaglie Jan 7, 2019
a03e360
Added methods to cycle on all installed platforms or boards
cmaglie Jan 7, 2019
30e6d2c
board list: reduced timeout to 1s
cmaglie Jan 7, 2019
5a4a13a
discovery: first implementation (WIP)
cmaglie Jan 2, 2019
a91c8a5
discovery: Do not start process on Discovery creation
cmaglie Jan 2, 2019
8905926
discovery: added timeout on LIST command
cmaglie Jan 2, 2019
ef00542
discovery: Timeout is now configurable
cmaglie Jan 7, 2019
cf1d8d0
discovery: Added method to retrieve all installed discoveries
cmaglie Jan 7, 2019
ace6552
discovery: Added method to identify boards from BoardPort identificat…
cmaglie Jan 7, 2019
5d00ed2
board list command now uses pluggable-discoveries
cmaglie Jan 7, 2019
42f576d
discovery: serial-discovery is now downloaded automatically
cmaglie Jan 7, 2019
bab5455
Removed direct dependency (to make dep check happy)
cmaglie Jan 7, 2019
b8617f0
Merge branch 'discovery' into daemon
cmaglie May 21, 2019
b8cc951
pluggable discovery: Fixed timeout for LIST to 2 seconds
cmaglie May 22, 2019
e97f574
Implemented GRPC call BoardList (WIP)
cmaglie May 22, 2019
f06a599
pluggable-discovery: use Discovery cmd-line as ID
cmaglie May 23, 2019
6ec261e
Use rpc getters instead of direct access to fields
cmaglie May 23, 2019
8874d91
Moved tool Install/Download helpers from commands/core to commands
cmaglie May 23, 2019
02a8800
Added missing license
cmaglie May 23, 2019
2c1b10d
Added status streaming to Init grpc call
cmaglie May 23, 2019
65f1043
pluggable-discovery handling in grpc Instances (WIP)
cmaglie May 23, 2019
14186ed
Reimplemented serial-discovery discovery
cmaglie May 23, 2019
7373b09
Restored timeout flag in 'board list'
cmaglie May 23, 2019
f027d13
grpc client test now works again
cmaglie May 23, 2019
b6a5eb4
Download ctags builtin tool during Init
cmaglie May 24, 2019
3b98cef
Implemented BoardListAll grpc method
cmaglie May 24, 2019
b951097
Removed no more used formatter/output/... structs
cmaglie May 24, 2019
11d9e6e
Implemented Version grpc command
cmaglie May 24, 2019
c57b075
Fixed client go regression
cmaglie May 24, 2019
e8e359b
Removed no more used formatter/output module
cmaglie May 24, 2019
f98cd2b
pluggable-discovery: better closing procedure
cmaglie May 24, 2019
ecffc26
Added BoardList and BoardListAll grpc client examples
cmaglie May 24, 2019
ab73120
Fix tests
cmaglie May 28, 2019
19be7cf
Fixed linter warnings
cmaglie May 28, 2019
b95af45
update tools to v1.0
masci Jun 3, 2019
59a4830
use https urls
masci Jun 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
87 changes: 84 additions & 3 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 0 additions & 4 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,6 @@
branch = "master"
name = "github.com/bgentry/go-netrc"

[[constraint]]
name = "github.com/codeclysm/cc"
version = "1.2.1"

[[constraint]]
branch = "master"
name = "github.com/mitchellh/go-homedir"
Expand Down
31 changes: 19 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ Once downloaded, place the executable into a directory which is in your `PATH` e

### Build the latest "bleeding-edge" from source

* You should have a recent Go compiler installed.
* Run `go get -u github.com/arduino/arduino-cli`
* The `arduino-cli` executable will be produced in `$GOPATH/bin/arduino-cli`
- You should have a recent Go compiler installed.
- Run `go get -u github.com/arduino/arduino-cli`
- The `arduino-cli` executable will be produced in `$GOPATH/bin/arduino-cli`

You may want to copy the executable into a directory which is in your `PATH` environment variable
(such as `/usr/local/bin/`).
Expand All @@ -45,7 +45,8 @@ The Arduino CLI aims to replace the majority of features the Arduino IDE has wit
## Getting Started

### Step 1. Create a new sketch
The command will create a new empty sketch named MyFirstSketch in the default directory under $HOME/Arduino/

The command will create a new empty sketch named MyFirstSketch in the default directory under \$HOME/Arduino/

$ arduino-cli sketch new MyFirstSketch
Sketch created in: /home/luca/Arduino/MyFirstSketch
Expand All @@ -58,6 +59,7 @@ The command will create a new empty sketch named MyFirstSketch in the default di
}

### Step 2. Modify your sketch

Use your favourite file editor or IDE to modify the .ino file under: `$HOME/Arduino/MyFirstSketch/MyFirstSketch.ino`
and change the file to look like this one:

Expand All @@ -73,6 +75,7 @@ and change the file to look like this one:
}

### Step 3. Connect the board to your PC

If you are running a fresh install of the arduino-cli you probably need to update the platform indexes by running:

$ arduino-cli core update-index
Expand Down Expand Up @@ -139,7 +142,6 @@ Now verify we have installed the core properly by running
ID Installed Latest Name
arduino:samd 1.6.19 1.6.19 Arduino SAMD Boards (32-bits ARM Cortex-M0+)


We can finally check if the board is now recognized as a MKR1000

$ arduino-cli board list
Expand All @@ -162,25 +164,29 @@ Great! Now we have the Board FQBN (Fully Qualified Board Name) `arduino:samd:mkr
and the Board Name look good, we are ready to compile and upload the sketch

#### Adding 3rd party cores
To add 3rd party core packages add a link of the additional package to the file `arduino-cli.yaml`

To add 3rd party core packages add a link of the additional package to the file `arduino-cli.yaml`

If you want to add the ESP8266 core, for example:

board_manager:
additional_urls:
- http://arduino.esp8266.com/stable/package_esp8266com_index.json

And then run:

arduino-cli core update-index
arduino-cli core install esp8266:esp8266

### Step 5. Compile the sketch

To compile the sketch we have to run the `compile` command with the proper FQBN we just got in the previous command.

$ arduino-cli compile --fqbn arduino:samd:mkr1000 Arduino/MyFirstSketch
Sketch uses 9600 bytes (3%) of program storage space. Maximum is 262144 bytes.

### Step 6. Upload your sketch

We can finally upload the sketch and see our board blinking, we now have to specify the serial port used by our board other than the FQBN:

$ arduino-cli upload -p /dev/ttyACM0 --fqbn arduino:samd:mkr1000 Arduino/MyFirstSketch
Expand Down Expand Up @@ -216,6 +222,7 @@ We can finally upload the sketch and see our board blinking, we now have to spec
CPU reset.

### Step 7. Add libraries

Now we can try to add a useful library to our sketch. We can at first look at the name of a library, our favourite one is the wifi101, here the command to get more info

$ arduino-cli lib search wifi101
Expand Down Expand Up @@ -250,6 +257,7 @@ We are now ready to install it! Please be sure to use the full name of the lib a
## Inline Help

`arduino-cli` is a container of commands, to see the full list just run:

```bash
$ arduino-cli
Arduino Command Line Interface (arduino-cli).
Expand Down Expand Up @@ -295,7 +303,7 @@ Flags:
-h, --help help for core

Global Flags:
--config-file string The custom config file (if not specified the default one will be used). (example "/home/user/.config/arduino/arduino-cli/arduino-cli.yaml")
--config-file string The custom config file (if not specified the default one will be used).
--debug Enables debug output (super verbose, used to debug the CLI).
--format string The output format, can be [text|json]. (default "text")

Expand All @@ -318,11 +326,10 @@ Because:

#### What is the FQBN for ...?

* Arduino UNO: `arduino:avr:uno`
* Arduino Mega: `arduino:avr:mega`
* Arduino Nano: `arduino:avr:nano` or `arduino:avr:nano:cpu=atmega328old` if you have the old bootloader
- Arduino UNO: `arduino:avr:uno`
- Arduino Mega: `arduino:avr:mega`
- Arduino Nano: `arduino:avr:nano` or `arduino:avr:nano:cpu=atmega328old` if you have the old bootloader

#### How can I find the core/FQBN for a board?

See: https://github.com/arduino/arduino-cli#step-4-find-and-install-the-right-core

8 changes: 4 additions & 4 deletions arduino/cores/cores.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ package cores
import (
"strings"

"github.com/arduino/go-paths-helper"
paths "github.com/arduino/go-paths-helper"

"github.com/arduino/arduino-cli/arduino/resources"
"github.com/arduino/go-properties-orderedmap"
"go.bug.st/relaxed-semver"
properties "github.com/arduino/go-properties-orderedmap"
semver "go.bug.st/relaxed-semver"
)

// Platform represents a platform package.
Expand Down Expand Up @@ -158,7 +158,7 @@ func (platform *Platform) latestReleaseVersion() *semver.Version {
func (platform *Platform) GetAllInstalled() []*PlatformRelease {
res := []*PlatformRelease{}
for _, release := range platform.Releases {
if release.InstallDir != nil {
if release.IsInstalled() {
res = append(res, release)
}
}
Expand Down
65 changes: 65 additions & 0 deletions arduino/cores/packagemanager/identify.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* This file is part of arduino-cli.
*
* Copyright 2018 ARDUINO SA (http://www.arduino.cc/)
*
* This software is released under the GNU General Public License version 3,
* which covers the main part of arduino-cli.
* The terms of this license can be found at:
* https://www.gnu.org/licenses/gpl-3.0.en.html
*
* You can be released from the requirements of the above licenses by purchasing
* a commercial license. Buying such a license is mandatory if you want to modify or
* otherwise use the software for commercial activities involving the Arduino
* software without disclosing the source code of your own applications. To purchase
* a commercial license, send an email to [email protected].
*/

package packagemanager

import (
"fmt"

"github.com/arduino/arduino-cli/arduino/cores"
properties "github.com/arduino/go-properties-orderedmap"
)

// IdentifyBoard returns a list of baords matching the provided identification properties.
func (pm *PackageManager) IdentifyBoard(idProps *properties.Map) []*cores.Board {
if idProps.Size() == 0 {
return []*cores.Board{}
}

checkSuffix := func(props *properties.Map, s string) (checked bool, found bool) {
for k, v1 := range idProps.AsMap() {
v2, ok := props.GetOk(k + s)
if !ok {
return false, false
}
if v1 != v2 {
return true, false
}
}
return false, true
}

foundBoards := []*cores.Board{}
for _, board := range pm.InstalledBoards() {
if _, found := checkSuffix(board.Properties, ""); found {
foundBoards = append(foundBoards, board)
continue
}
id := 0
for {
again, found := checkSuffix(board.Properties, fmt.Sprintf(".%d", id))
if found {
foundBoards = append(foundBoards, board)
}
if !again {
break
}
id++
}
}
return foundBoards
}
Loading