Sending a payload of (2^24−1) bytes

[BohuTANG]

Issue description

By reference:
https://dev.mysql.com/doc/internals/en/sending-more-than-16mbyte.html
Sending a payload of 16 777 215 (2^24−1) bytes looks like:

ff ff ff 00 ...
00 00 00 01

In this case, readPacket will get an unexpected error in:
https://github.com/go-sql-driver/mysql/blob/master/packets.go#L41

		if pktLen < 1 {
			errLog.Print(ErrMalformPkt)
			mc.Close()
			return nil, driver.ErrBadConn
		}

[arnehormann]

Did you get hit by this or did you find it while reading the code?
I'm asking because we have a test for that and I'd like to know how/if fails: https://github.com/go-sql-driver/mysql/blob/master/driver_test.go#L955

[BohuTANG]

This test has no effect on this issue.
It happened only when the payload length is a multiple of maxPacketSize.
So our last chunk of the packet as follows(https://dev.mysql.com/doc/internals/en/sending-more-than-16mbyte.html):

00 00 00 01

In readPacket:

data, err := mc.buf.readNext(4)
...
pktLen := int(uint32(data[0]) | uint32(data[1])<<8 | uint32(data[2])<<16)

here, the pktLen is zero, but it's the last chunk length of the packet.

[arnehormann]

Thanks for the report!
I'll have a look at it over the weekend.

[BohuTANG]

We should add packet_test.go, so could easily inject a boundary case.

[dgryski]

Would it make sense to fuzz the protocol implementation?

[arnehormann]

@dgryski that might make sense - and it is definitely desirable. I lack practical experience with fuzzing - but I really enjoy your articles and wanted to get my feet wet (or warm and fuzzy) for quite some time, so I'm game if you want to collaborate - I'd love to learn.

Right now I don't have a clue how to even start fuzzing the driver - and we lack the hardware. But I have some conceptual questions first:

Can it handle packet types and the transitions between them? Does it have to?
Does protocol fuzzing start from tcpdumps (building on e.g. https://github.com/google/netboot/tree/master/pcap)?
If so, where do we get those - and how many / which types?
Does it require an oracle (a MySQL server or the C-based connector) to calculate the returned score? How do we get it?

[dgryski]

There are a few different kinds of fuzzing. The first one is just searching for things like crashes, hangs, etc. That's very easy to do with go-fuzz, and is basically documented here: https://medium.com/@dgryski/go-fuzz-jiasu.xzqcsaa.nyc.mn-arolek-ase-3c74d5a3150c . For the SQL driver, we'd probably want to fuzz the various read* methods in packets.go. This is probably where I'd start and would just protect the client from buggy (or perhaps malicious) servers. I think "buggy" is the important bit here since there are many proxies and other mysql tools which speak the mysql protocol to varying degrees, and won't be as heavily tested as the standard implementation. The Cockroachdb team has fuzzed their postgres wire-protocol, for example: https://github.com/cockroachdb/go-fuzz/blob/master/examples/pgwire/main.go

Other kinds of randomized tests certainly could be constructed using a "real" mysql-server and the official, C-based driver to match behaviour but I think that's a much larger project.

[BohuTANG]

We need a packet_test.go(white-box) file before fuzz(black-box), and it's easier for the rest to test the 'doubt point'.

[julienschmidt]

See Pull Request #516

[julienschmidt]

PR #516 just added unit tests for mc.readPacket().
If someone is interested in working on more unit tests for other packet functions, or someone even wants to attempt to write a fuzzer, please go ahead and send pull requests! Your effort is highly appreciated!