Skip to content

Implement support for the CoderVPN Control Protocol #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
spikecurtis opened this issue Sep 19, 2024 · 0 comments
Closed

Implement support for the CoderVPN Control Protocol #1

spikecurtis opened this issue Sep 19, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@spikecurtis
Copy link
Collaborator

Our Go code that creates the tailnet connection and services the VPN packets needs to communicate with OS-native code that can do things like

  • Log to the native system log
  • Update the status tray with peer status
  • Configure the network settings (IP and DNS)

Since Go and native code will have different memory layouts and synchronization primitives, we will use a bidirectional stream to communicate between Go and the native code, even if Go and native code coexist in the same process.

We define a new internal protocol to facilitate this communication. This protocol uses Protobufs for message serialization, but does not use dRPC like our other Protobuf-based APIs since dRPC is not available for the languages we’d like to use for native OS development (e.g. C#, Swift).

The protocol runs over a single bidirectional stream between the native code (called the Manager) and the Go code (called the Tunnel). We keep everything simple and send messages serially on the stream (rather than rely on a multiplexing library like yamux, which is only implemented in Go). This means that RPC responses might get behind other, unrelated messages, but we expect this control protocol to be very light in traffic, and response times are generally not crucial.

On opening the stream, each side sends a single plaintext header of the form codervpn <major.minor> <role>\n where <major.minor> is the protocol version and <role> is “manager” or “tunnel”. After reading and verifying the other side’s header, they each proceed to send a series of messages encoded as:

  • 4-byte unsigned integer, big-endian: length
  • <length> bytes of Protobuf encoded message, either TunnelMessage or ManagerMessage according to their role, as follows:
// RPC allows a very simple unary request/response RPC mechanism.  The requester
// generates a unique msg_id which it sets on the request, the responder sets
// response_to that msg_id on the response message
message RPC {
    uint64 msg_id = 1;
    uint64 response_to = 2;
}

// ManagerMessage is a message from the manager (to the tunnel).
message ManagerMessage {
    RPC rpc = 1;
    oneof msg {
        GetPeerUpdate get_peer_update = 2;
        NetworkSettingsResponse network_settings = 3;
        StartRequest start = 4;
        StopRequest stop = 5;
    }
}

// TunnelMessage is a message from the tunnel (to the manager).
message TunnelMessage {
    RPC rpc = 1;
    oneof msg {
        Log log = 2;
        PeerUpdate peer_update = 3;
        NetworkSettingsRequest network_settings = 4;
        StartResponse start = 5;
        StopResponse stop = 6;
    }
}

// Log is a log message generated by the tunnel.  The manager should log it to
// the system log. It is one-way tunnel -> manager with no response.
message Log {
    enum Level {
        UNSPECIFIED = 0;
        INFO = 1;
        WARN = 2;
        ERROR = 3;
        CRITICAL = 4;
        FATAL = 5;
    }
    Level level = 1;

    string message = 2;
    repeated string logger_names = 3;

    message Field {
        string name = 1;
        string value = 2;
    }
    repeated Field fields = 4;
}

// GetPeerUpdate asks for a PeerUpdate with a full set of data.
message GetPeerUpdate {}

// PeerUpdate is an update about workspaces and agents connected via the tunnel.
// It is generated in response to GetPeerUpdate (which dumps the full set). It is
// also generated on any changes (not in response to any request).
message PeerUpdate {
    repeated Workspace upserted_workspaces = 1;
    repeated Agent upserted_agents = 2;
    repeated Workspace deleted_workspaces = 3;
    repeated Agent deleted_agents = 4;
}

message Workspace {
    bytes id = 1; // UUID
    string name = 2;
    
  enum Status {
		UNKNOWN = 0;
		PENDING = 1;
		STARTING = 2;
		RUNNING = 3;
		STOPPING = 4;
		STOPPED = 5;
		FAILED = 6;
		CANCELING = 7;
		CANCELED = 8;
		DELETING = 9;
		DELETED = 10;
	}
	Status status = 3;
}

message Agent {
    bytes id = 1; // UUID
    string name = 2;
    bytes workspace_id = 3; // UUID
    string fqdn = 4;
    repeated string ip_addrs = 5;
    // last_handshake is the primary indicator of whether we are connected to a
    // peer. Zero value or anything longer than 5 minutes ago means there is a
    // problem.
    google.protobuf.Timestamp last_handshake = 6;
}

// NetworkSettingsRequest is based on
// https://developer.apple.com/documentation/networkextension/nepackettunnelnetworksettings
// for macOS.  It is a request/response message with response
// NetworkSettingsResponse
message NetworkSettingsRequest {
    uint32 tunnel_overhead_bytes = 1;
    uint32 mtu = 2;

    message DNSSettings {
        repeated string servers = 1;
        repeated string search_domains = 2;
        // domain_name is the primary domain name of the tunnel
        string domain_name = 3;
        repeated string match_domains = 4;
        // match_domains_no_search specifies if the domains in the matchDomains
        // list should not be appended to the resolver’s list of search domains.
        bool match_domains_no_search = 5;
    }
    DNSSettings dns_settings = 3;

    string tunnel_remote_address = 4;

    message IPv4Settings {
        repeated string addrs = 1;
        repeated string subnet_masks = 2;
        // router is the next-hop router in dotted-decimal format
        string router = 3;

        message IPv4Route {
            string destination = 1;
            string mask = 2;
            // router is the next-hop router in dotted-decimal format
            string router = 3;
        }
        repeated IPv4Route included_routes = 4;
        repeated IPv4Route excluded_routes = 5;
    }
    IPv4Settings ipv4_settings = 5;

    message IPv6Settings {
        repeated string addrs = 1;
        repeated uint32 prefix_lengths = 2;

        message IPv6Route {
            string destination = 1;
            uint32 prefix_length = 2;
            // router is the address of the next-hop
            string router = 3;
        }
        repeated IPv6Route included_routes = 3;
        repeated IPv6Route excluded_routes = 4;
    }
    IPv6Settings ipv6_settings = 6;
}

// NetworkSettingsResponse is the response from the manager to the tunnel for a
// NetworkSettingsRequest
message NetworkSettingsResponse {
    bool success = 1;
    string error_message = 2;
}

// StartRequest is a request from the manager to start the tunnel.  The tunnel
// replies with a StartResponse.
message StartRequest {
    int32 tunnel_file_descriptor = 1;
    string coder_url = 2;
    string api_token = 3;
}

message StartResponse {
    bool success = 1;
    string error_message = 2;
}

// StopRequest is a request from the manager to stop the tunnel. The tunnel
// replies with a StopResponse.
message StopRequest {}

// StopResponse is a response to stopping the tunnel. After sending this response,
// the tunnel closes its side of the bidirectional stream for writing.
message StopResponse {
    bool success = 1;
    string error_message = 2;
}
@coder-labeler coder-labeler bot added the enhancement New feature or request label Sep 19, 2024
@spikecurtis spikecurtis self-assigned this Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant