Skip to content

Commit 8b8ca37

Browse files
evaneliasevanelias
authored
host matching: handle wildcards with non-standard port (#10)
In OpenSSH, wildcard host pattern entries in a known_hosts file can match hosts regardless of their port number. However, x/crypto/ssh/knownhosts does not follow this behavior, instead requiring strict port equality; see bug golang/go#52056 for background. This commit implements a workaround in skeema/knownhosts, which is enabled when using the NewDB constructor. Conceptually, the workaround works like this: * At constructor time, when re-reading the known_hosts file (originally to look for @cert-authority lines), also look for lines that have wildcards in the host pattern and no port number specified. Track these lines in a new field of the HostKeyDB struct for later use. * When a host key callback returns no matches (KeyError with empty Want slice) and the host had a nonstandard (non-22) port number, try the callback again, this time manipulating the host arg to be on port 22. * If this second call returned nil error, that means the host key now matched a known_hosts entry on port 22, so consider the host as known. * If this second call returned a KeyError with non-empty Want slice, filter down the resulting keys to only correspond to lines with known wildcards, using the preprocessed information from the first step. This ensures we aren't incorrectly returning non-wildcard entries among the Want slice. The implementation for the latter 3 bullets gets embedded directly in the host key callback returned by HostKeyDB.HostKeyCallback, by way of some nested callback wrapping. This only happens if the first bullet actually found at least one wildcard in the file.
1 parent 7c797a4 commit 8b8ca37

File tree

4 files changed

+167
-31
lines changed

4 files changed

+167
-31
lines changed

‎README.md

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,18 @@
77

88
> This repo is brought to you by [Skeema](https://github.com/skeema/skeema), a
99
> declarative pure-SQL schema management system for MySQL and MariaDB. Our
10-
> premium products include extensive [SSH tunnel](https://www.skeema.io/docs/options/#ssh)
10+
> premium products include extensive [SSH tunnel](https://www.skeema.io/docs/features/ssh/)
1111
> functionality, which internally makes use of this package.
1212
1313
Go provides excellent functionality for OpenSSH known_hosts files in its
1414
external package [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts).
15-
However, that package is somewhat low-level, making it difficult to implement full known_hosts management similar to command-line `ssh`'s behavior for `StrictHostKeyChecking=no` configuration.
15+
However, that package is somewhat low-level, making it difficult to implement full known_hosts management similar to command-line `ssh`'s behavior for `StrictHostKeyChecking=no` configuration. Additionally, it has several known issues which have been open for multiple years.
1616

17-
This repo ([github.com/skeema/knownhosts](https://github.com/skeema/knownhosts)) is a thin wrapper package around [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts), adding the following functionality:
17+
Package [github.com/skeema/knownhosts](https://github.com/skeema/knownhosts) provides a thin wrapper around [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts), adding the following functionality:
1818

1919
* Look up known_hosts public keys for any given host
20-
* Auto-populate ssh.ClientConfig.HostKeyAlgorithms easily based on known_hosts, providing a solution for [golang/go#29286](https://github.com/golang/go/issues/29286)
20+
* Auto-populate ssh.ClientConfig.HostKeyAlgorithms easily based on known_hosts, providing a solution for [golang/go#29286](https://github.com/golang/go/issues/29286). This also properly handles cert algorithms for hosts using CA keys when [using the NewDB constructor](#enhancements-requiring-extra-parsing) added in v1.3.0.
21+
* Properly match wildcard hostname known_hosts entries regardless of port number, providing a solution for [golang/go#52056](https://github.com/golang/go/issues/52056). (Added in v1.3.0; requires [using the NewDB constructor](#enhancements-requiring-extra-parsing))
2122
* Write new known_hosts entries to an io.Writer
2223
* Properly format/normalize new known_hosts entries containing ipv6 addresses, providing a solution for [golang/go#53463](https://github.com/golang/go/issues/53463)
2324
* Determine if an ssh.HostKeyCallback's error corresponds to a host whose key has changed (indicating potential MitM attack) vs a host that just isn't known yet
@@ -57,6 +58,19 @@ func sshConfigForHost(hostWithPort string) (*ssh.ClientConfig, error) {
5758
}
5859
```
5960

61+
## Enhancements requiring extra parsing
62+
63+
Originally, this package did not re-read/re-parse the known_hosts files at all, relying entirely on [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts) for all known_hosts file reading and processing. This package only offered a constructor called `New`, returning a host key callback, identical to the call pattern of [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts) but with extra methods available on the callback type.
64+
65+
However, a couple bugs in [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts) cannot possibly be solved without re-reading the known_hosts file. Therefore, as of v1.3.0 of this package, we now offer an alternative constructor `NewDB`, which does an additional read of the known_hosts file (after the one from [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts)), in order to detect:
66+
67+
* @cert-authority lines, so that we can correctly reeturn cert key algorithms instead of normal host key algorithms when appropriate
68+
* host pattern wildcards, so that we can match OpenSSH's behavior for non-standard port numbers, unlike how [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts) normally treats them
69+
70+
Aside from *detecting* these special cases, this package otherwise still directly uses [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts) for host lookups and all other known_hosts file processing. We do not fork or re-implement core behaviors of [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts).
71+
72+
The performance impact of this extra read should be minimal, as the file should typically be in the filesystem cache already from the original read by [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts). That said, users who wish to avoid the extra read can stay with the `New` constructor, which intentionally retains its pre-v1.3.0 behavior as-is. However, the extra fixes for @cert-authority and host pattern wildcards will not be enabled in that case.
73+
6074
## Writing new known_hosts entries
6175

6276
If you wish to mimic the behavior of OpenSSH's `StrictHostKeyChecking=no` or `StrictHostKeyChecking=ask`, this package provides a few functions to simplify this task. For example:

‎example_test.go

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,45 @@ func ExampleNewDB() {
4848
defer client.Close()
4949
}
5050

51+
func ExampleHostKeyCallback_ToDB() {
52+
khFile := "/home/myuser/.ssh/known_hosts"
53+
var kh *knownhosts.HostKeyDB
54+
var err error
55+
56+
// Example of using conditional logic to determine whether or not to perform
57+
// extra parsing pass on the known_hosts file in order to enable enhanced
58+
// behaviors
59+
if os.Getenv("SKIP_KNOWNHOSTS_ENHANCEMENTS") != "" {
60+
// Create a HostKeyDB using New + ToDB: this will skip the extra known_hosts
61+
// processing
62+
var cb knownhosts.HostKeyCallback
63+
if cb, err = knownhosts.New(khFile); err == nil {
64+
kh = cb.ToDB()
65+
}
66+
} else {
67+
// Create a HostKeyDB using NewDB: this will perform extra known_hosts
68+
// processing, allowing proper support for CAs, as well as OpenSSH-like
69+
// wildcard matching on non-standard ports
70+
kh, err = knownhosts.NewDB(khFile)
71+
}
72+
if err != nil {
73+
log.Fatal("Failed to read known_hosts: ", err)
74+
}
75+
76+
sshHost := "yourserver.com:22"
77+
config := &ssh.ClientConfig{
78+
User: "myuser",
79+
Auth: []ssh.AuthMethod{ /* ... */ },
80+
HostKeyCallback: kh.HostKeyCallback(),
81+
HostKeyAlgorithms: kh.HostKeyAlgorithms(sshHost),
82+
}
83+
client, err := ssh.Dial("tcp", sshHost, config)
84+
if err != nil {
85+
log.Fatal("Failed to dial: ", err)
86+
}
87+
defer client.Close()
88+
}
89+
5190
func ExampleWriteKnownHost() {
5291
sshHost := "yourserver.com:22"
5392
khPath := "/home/myuser/.ssh/known_hosts"

‎knownhosts.go

Lines changed: 97 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -22,22 +22,31 @@ import (
2222
// behaviors, such as the ability to perform host key/algorithm lookups from
2323
// known_hosts entries.
2424
type HostKeyDB struct {
25-
callback ssh.HostKeyCallback
26-
isCert map[string]bool // keyed by "filename:line"
25+
callback ssh.HostKeyCallback
26+
isCert map[string]bool // keyed by "filename:line"
27+
isWildcard map[string]bool // keyed by "filename:line"
2728
}
2829

2930
// NewDB creates a HostKeyDB from the given OpenSSH known_hosts file(s). It
3031
// reads and parses the provided files one additional time (beyond logic in
31-
// golang.org/x/crypto/ssh/knownhosts) in order to handle CA lines properly.
32+
// golang.org/x/crypto/ssh/knownhosts) in order to:
33+
//
34+
// - Handle CA lines properly and return ssh.CertAlgo* values when calling the
35+
// HostKeyAlgorithms method, for use in ssh.ClientConfig.HostKeyAlgorithms
36+
// - Allow * wildcards in hostnames to match on non-standard ports, providing
37+
// a workaround for https://github.com/golang/go/issues/52056 in order to
38+
// align with OpenSSH's wildcard behavior
39+
//
3240
// When supplying multiple files, their order does not matter.
3341
func NewDB(files ...string) (*HostKeyDB, error) {
3442
cb, err := xknownhosts.New(files...)
3543
if err != nil {
3644
return nil, err
3745
}
3846
hkdb := &HostKeyDB{
39-
callback: cb,
40-
isCert: make(map[string]bool),
47+
callback: cb,
48+
isCert: make(map[string]bool),
49+
isWildcard: make(map[string]bool),
4150
}
4251

4352
// Re-read each file a single time, looking for @cert-authority lines. The
@@ -59,6 +68,16 @@ func NewDB(files ...string) (*HostKeyDB, error) {
5968
if len(line) > 15 && bytes.HasPrefix(line, []byte("@cert-authority")) && (line[15] == ' ' || line[15] == '\t') {
6069
mapKey := fmt.Sprintf("%s:%d", filename, lineNum)
6170
hkdb.isCert[mapKey] = true
71+
line = bytes.TrimSpace(line[16:])
72+
}
73+
// truncate line to just the host pattern field
74+
if i := bytes.IndexAny(line, "\t "); i >= 0 {
75+
line = line[:i]
76+
}
77+
// Does the host pattern contain a * wildcard and no specific port?
78+
if i := bytes.IndexRune(line, '*'); i >= 0 && !bytes.Contains(line[i:], []byte("]:")) {
79+
mapKey := fmt.Sprintf("%s:%d", filename, lineNum)
80+
hkdb.isWildcard[mapKey] = true
6281
}
6382
}
6483
if err := scanner.Err(); err != nil {
@@ -73,11 +92,56 @@ func NewDB(files ...string) (*HostKeyDB, error) {
7392
// Alternatively, you can wrap it with an outer callback to potentially handle
7493
// appending a new entry to the known_hosts file; see example in WriteKnownHost.
7594
func (hkdb *HostKeyDB) HostKeyCallback() ssh.HostKeyCallback {
76-
return hkdb.callback
95+
// Either NewDB found no wildcard host patterns, or hkdb was created from
96+
// HostKeyCallback.ToDB in which case we didn't scan known_hosts for them:
97+
// return the callback (which came from x/crypto/ssh/knownhosts) as-is
98+
if len(hkdb.isWildcard) == 0 {
99+
return hkdb.callback
100+
}
101+
102+
// If we scanned for wildcards and found at least one, return a wrapped
103+
// callback with extra behavior: if the host lookup found no matches, and the
104+
// host arg had a non-standard port, re-do the lookup on standard port 22. If
105+
// that second call returns a *xknownhosts.KeyError, filter down any resulting
106+
// Want keys to known wildcard entries.
107+
f := func(hostname string, remote net.Addr, key ssh.PublicKey) error {
108+
callbackErr := hkdb.callback(hostname, remote, key)
109+
if callbackErr == nil || IsHostKeyChanged(callbackErr) { // hostname has known_host entries as-is
110+
return callbackErr
111+
}
112+
justHost, port, splitErr := net.SplitHostPort(hostname)
113+
if splitErr != nil || port == "" || port == "22" { // hostname already using standard port
114+
return callbackErr
115+
}
116+
// If we reach here, the port was non-standard and no known_host entries
117+
// were found for the non-standard port. Try again with standard port.
118+
if tcpAddr, ok := remote.(*net.TCPAddr); ok && tcpAddr.Port != 22 {
119+
remote = &net.TCPAddr{
120+
IP: tcpAddr.IP,
121+
Port: 22,
122+
Zone: tcpAddr.Zone,
123+
}
124+
}
125+
callbackErr = hkdb.callback(justHost+":22", remote, key)
126+
var keyErr *xknownhosts.KeyError
127+
if errors.As(callbackErr, &keyErr) && len(keyErr.Want) > 0 {
128+
wildcardKeys := make([]xknownhosts.KnownKey, 0, len(keyErr.Want))
129+
for _, wantKey := range keyErr.Want {
130+
if hkdb.isWildcard[fmt.Sprintf("%s:%d", wantKey.Filename, wantKey.Line)] {
131+
wildcardKeys = append(wildcardKeys, wantKey)
132+
}
133+
}
134+
callbackErr = &xknownhosts.KeyError{
135+
Want: wildcardKeys,
136+
}
137+
}
138+
return callbackErr
139+
}
140+
return ssh.HostKeyCallback(f)
77141
}
78142

79143
// PublicKey wraps ssh.PublicKey with an additional field, to identify
80-
// whether they key corresponds to a certificate authority.
144+
// whether the key corresponds to a certificate authority.
81145
type PublicKey struct {
82146
ssh.PublicKey
83147
Cert bool
@@ -96,7 +160,8 @@ func (hkdb *HostKeyDB) HostKeys(hostWithPort string) (keys []PublicKey) {
96160
placeholderAddr := &net.TCPAddr{IP: []byte{0, 0, 0, 0}}
97161
placeholderPubKey := &fakePublicKey{}
98162
var kkeys []xknownhosts.KnownKey
99-
if hkcbErr := hkdb.callback(hostWithPort, placeholderAddr, placeholderPubKey); errors.As(hkcbErr, &keyErr) {
163+
callback := hkdb.HostKeyCallback()
164+
if hkcbErr := callback(hostWithPort, placeholderAddr, placeholderPubKey); errors.As(hkcbErr, &keyErr) {
100165
kkeys = append(kkeys, keyErr.Want...)
101166
knownKeyLess := func(i, j int) bool {
102167
if kkeys[i].Filename < kkeys[j].Filename {
@@ -190,14 +255,16 @@ func keyTypeToCertAlgo(keyType string) string {
190255
// otherwise identical to ssh.HostKeyCallback, and does not introduce any file-
191256
// parsing behavior beyond what is in golang.org/x/crypto/ssh/knownhosts.
192257
//
258+
// In most situations, use HostKeyDB and its constructor NewDB instead of using
259+
// the HostKeyCallback type. The HostKeyCallback type is only provided for
260+
// backwards compatibility with older versions of this package, as well as for
261+
// very strict situations where any extra known_hosts file-parsing is
262+
// undesirable.
263+
//
193264
// Methods of HostKeyCallback do not provide any special treatment for
194265
// @cert-authority lines, which will (incorrectly) look like normal non-CA host
195-
// keys. HostKeyCallback should generally only be used in situations in which
196-
// @cert-authority lines won't appear, and/or in very strict situations where
197-
// any extra known_hosts file-parsing is undesirable.
198-
//
199-
// In most situations, use HostKeyDB and its constructor NewDB instead of using
200-
// the HostKeyCallback type.
266+
// keys. Additionally, HostKeyCallback lacks the fix for applying * wildcard
267+
// known_host entries to all ports, like OpenSSH's behavior.
201268
type HostKeyCallback ssh.HostKeyCallback
202269

203270
// New creates a HostKeyCallback from the given OpenSSH known_hosts file(s). The
@@ -207,9 +274,9 @@ type HostKeyCallback ssh.HostKeyCallback
207274
// When supplying multiple files, their order does not matter.
208275
//
209276
// In most situations, you should avoid this function, as the returned value
210-
// does not handle @cert-authority lines correctly. See doc comment for
211-
// HostKeyCallback for more information. Instead, use NewDB to create a
212-
// HostKeyDB with proper CA support.
277+
// lacks several enhanced behaviors. See doc comment for HostKeyCallback for
278+
// more information. Instead, most callers should use NewDB to create a
279+
// HostKeyDB, which includes these enhancements.
213280
func New(files ...string) (HostKeyCallback, error) {
214281
cb, err := xknownhosts.New(files...)
215282
return HostKeyCallback(cb), err
@@ -222,16 +289,20 @@ func (hkcb HostKeyCallback) HostKeyCallback() ssh.HostKeyCallback {
222289
}
223290

224291
// ToDB converts the receiver into a HostKeyDB. However, the returned HostKeyDB
225-
// lacks proper CA support. It is usually preferable to create a CA-supporting
226-
// HostKeyDB instead, by using NewDB.
227-
// This method is provided for situations in which the calling code needs to
228-
// make CA support optional / user-configurable. This way, calling code can
229-
// conditionally create a non-CA-supporting HostKeyDB by calling New(...).ToDB()
230-
// or a CA-supporting HostKeyDB by calling NewDB(...).
292+
// lacks the enhanced behaviors described in the doc comment for NewDB: proper
293+
// CA support, and wildcard matching on nonstandard ports.
294+
//
295+
// It is generally preferable to create a HostKeyDB by using NewDB. The ToDB
296+
// method is only provided for situations in which the calling code needs to
297+
// make the extra NewDB behaviors optional / user-configurable, perhaps for
298+
// reasons of performance or code trust (since NewDB reads the known_host file
299+
// an extra time, which may be undesirable in some strict situations). This way,
300+
// callers can conditionally create a non-enhanced HostKeyDB by using New and
301+
// ToDB. See code example.
231302
func (hkcb HostKeyCallback) ToDB() *HostKeyDB {
232-
// This intentionally leaves the isCert map field as nil, as there is no way
233-
// to retroactively populate it from just a HostKeyCallback. Methods of
234-
// HostKeyDB will skip any CA-related behaviors accordingly.
303+
// This intentionally leaves the isCert and isWildcard map fields as nil, as
304+
// there is no way to retroactively populate them from just a HostKeyCallback.
305+
// Methods of HostKeyDB will skip any related enhanced behaviors accordingly.
235306
return &HostKeyDB{callback: ssh.HostKeyCallback(hkcb)}
236307
}
237308

‎knownhosts_test.go

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -191,7 +191,13 @@ func TestWithCertLines(t *testing.T) {
191191
expectedAlgos: []string{ssh.KeyAlgoRSASHA512, ssh.KeyAlgoRSASHA256, ssh.KeyAlgoRSA, ssh.CertAlgoECDSA256v01},
192192
},
193193
{
194-
host: "whatever.lol.test:22", // only matches the * entry
194+
host: "whatever.test:22", // only matches the * entry
195+
expectedKeyTypes: []string{ssh.KeyAlgoECDSA256},
196+
expectedIsCert: []bool{true},
197+
expectedAlgos: []string{ssh.CertAlgoECDSA256v01},
198+
},
199+
{
200+
host: "whatever.test:22022", // only matches the * entry
195201
expectedKeyTypes: []string{ssh.KeyAlgoECDSA256},
196202
expectedIsCert: []bool{true},
197203
expectedAlgos: []string{ssh.CertAlgoECDSA256v01},
@@ -202,6 +208,12 @@ func TestWithCertLines(t *testing.T) {
202208
expectedIsCert: []bool{true, true, true},
203209
expectedAlgos: []string{ssh.CertAlgoRSASHA512v01, ssh.CertAlgoRSASHA256v01, ssh.CertAlgoRSAv01, ssh.CertAlgoECDSA256v01, ssh.CertAlgoED25519v01},
204210
},
211+
{
212+
host: "oddport.certy.test:2345",
213+
expectedKeyTypes: []string{ssh.KeyAlgoRSA, ssh.KeyAlgoECDSA256, ssh.KeyAlgoED25519},
214+
expectedIsCert: []bool{true, true, true},
215+
expectedAlgos: []string{ssh.CertAlgoRSASHA512v01, ssh.CertAlgoRSASHA256v01, ssh.CertAlgoRSAv01, ssh.CertAlgoECDSA256v01, ssh.CertAlgoED25519v01},
216+
},
205217
}
206218
for _, tc := range testCases {
207219
annotatedKeys := kh.HostKeys(tc.host)

0 commit comments

Comments
 (0)