Skip to content

Daemon mode (GRPC) + pluggable discovery #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 169 commits into from
Jun 3, 2019
Merged

Daemon mode (GRPC) + pluggable discovery #217

merged 169 commits into from
Jun 3, 2019

Conversation

cmaglie
Copy link
Member

@cmaglie cmaglie commented May 28, 2019

This (quite big) PR adds a "daemonized" mode to arduino-cli. When the cli is launched with the command arduino-cli daemon it starts to listen to the TCP port :50051 waiting for gRPC commands.

  • First of all we encoded all the exising arduino-cli commands into a protobuffer gRPC API, the definition can be found inside the rpc/*.proto files.
  • All the existing arduino-cli commands (inside commands folder) have been ported to use the gRPC request/response structures (generated from the .proto files) as input/output.
  • The command line parsing has been extracted from commands and isolated into cli folder.
  • The RPC server implementation has been done in the daemon folder. The gRPC server can now just directly forward the incoming rpc calls toward the cli commands implementation in the commands folder.
  • There is an RPC client example inside daemon/client folder.
  • The "classic" JSON output still works, it's simply implemented by JSON-serializing the gRPC responses.
  • The pluggable discovery has been merged here and it's used for the serial discovery on linux64 only (MDNS discovery and all the OS variants will be added later)
  • The gRPC API is 95% done, but it may be subject to last minute change.

cmaglie and others added 30 commits January 30, 2019 00:22
@cmaglie
Some minor changes to readme
6eb7f2d
@cmaglie
InitLibraryManager now requires a Configuration
2772438
@cmaglie
go-Formatted some imports
57e0cda
@cmaglie
Made commands.lib.ListLibraries public
46d175e
@cmaglie
Made configs.Configuration.ArduinoDownloadsDir field public
dfb3720
@cmaglie
Added daemon mode (WIP)
94e5765
@cmaglie
Big refactoring for daemon mode
015f0d7 
All the arduino-cli commands implementation has been moved from package
`command` to package `cli`.

The `board details` command has been choosen as a 'tracing bullet' to
reimplement all the cli command using grpc/protoc to generate
Request/Response (input/output) structures.

We should probably abandon the idea to use a generic `formatter` module
since it seems that it doesn't help to reduce boilerplate code (for
example in the `board details` command there is no need for the
`formatter` module nor it's clear how it can improve things).

The `output` module now has only functions for pretty-printing data.
@cmaglie
Deamon implementation and refactoring
12868a5
@cmaglie
Set sketchbookdir config during init
c6ad703
@cmaglie
cli.Init* funcitons now have a more meaningful name
0f18110
@cmaglie
library_index.json is automatically updated on Init (again)
ec4ff4e
@cmaglie
Use config.NewConfiguration to setup a new configuration
c226f62
@cmaglie
Better client example
76198da
@cmaglie
Moved compile module under cli
875ec10
@cmaglie
Added Compile rpc command
fb8da3c
@cmaglie
Rebuilt grpc bindings
3277596
@cmaglie
Added daemon callback for Compile rpc call
08c0a66
@cmaglie
Moved "Compile" implementation from cli to commands (WIP)
51dd95f
@cmaglie
Use rpc fields to run Compile command
4408123
@Rocketct
Minor fix added return value to the compile function in command/compile
b4f7ea4
@Rocketct
Implemented run function in cli/compile
9dcf531
@Rocketct
Implemented Client "Compile" call on Daemon/Client
6555b10
@cmaglie
Removed 'Result' message from grpc .proto
7ba912f
@cmaglie
output: fixed divide by 0 in some cases
196e956
@cmaglie
Using grpc status errors instead of embedding errors in messages
c13d9fd
@cmaglie
Use normal errors from 'command'
034a193 
Normal errors are automatically wrapped into GRPC errors with code
equals to "Unknown".
@cmaglie
Stream builder output in Compile command
3ba5f72
@cmaglie
Set correct output io.Writer during Compile
79aa5a5 
With this patch we correctly forward output messages (to the CLI
terminal or to the GRPC client).
@Rocketct
Moved core module under cli
63969a4
@Rocketct
Added Install rpc command
5bb021f
@cmaglie cmaglie added the type: enhancement Proposed improvement label May 28, 2019
@cmaglie cmaglie added this to the next alpha milestone May 28, 2019
@CLAassistant
Copy link

CLAassistant commented May 28, 2019 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@arduino arduino deleted a comment from ArduinoBot May 28, 2019
@per1234
Copy link
Contributor

per1234 commented Jun 1, 2019

@cmaglie all the test build URLs are giving 404 errors. That's no problem for me because it's easy enough for me to build from source, but if this indicates a problem with the build system it would be nice to get it fixed to make it easy for anyone to do beta testing.

@facchinm
Copy link
Member

facchinm commented Jun 3, 2019

Hi @per1234 ,
the links should be ok now, there was a misconfiguration of the URL we were uploading to.

@cmaglie cmaglie merged commit 15bf593 into arduino:master Jun 3, 2019
@cmaglie cmaglie deleted the daemon branch June 3, 2019 22:45
@ArduinoBot
Copy link
Contributor

✅ Build completed.

⬇️ Linux 64: https://downloads.arduino.cc/PR/arduino-cli/arduino-cli-51-PR217-linux64.tar.bz2

⬇️ Linux 32: https://downloads.arduino.cc/PR/arduino-cli/arduino-cli-51-PR217-linux32.tar.bz2

⬇️ Linux ARM: https://downloads.arduino.cc/PR/arduino-cli/arduino-cli-51-PR217-linuxarm.tar.bz2

⬇️ Windows: https://downloads.arduino.cc/PR/arduino-cli/arduino-cli-51-PR217-windows.zip

⬇️ OSX: https://downloads.arduino.cc/PR/arduino-cli/arduino-cli-51-PR217-osx.zip

ℹ️ To test this build:

  1. Just replace arduino-cli binary with the provided one

@arduino arduino deleted a comment from ArduinoBot Jun 4, 2019
mastrolinux pushed a commit that referenced this pull request Jul 16, 2019
@cmaglie
Merge pull request #217 from cmaglie/daemon
d8f3c42 
Daemon mode (GRPC) + pluggable discovery
per1234 added a commit that referenced this pull request Aug 9, 2021
@per1234
Merge pull request #217 from per1234/install-script
29e3108 
Sync installation assets from template
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@cmaglie cmaglie

@Rocketct Rocketct

Labels
type: enhancement Proposed improvement
Projects
None yet
Milestone
next alpha
Development

Successfully merging this pull request may close these issues.
7 participants
@cmaglie @CLAassistant @per1234 @facchinm @ArduinoBot @Rocketct @masci