Skip to content

Commit 2d9153b

Browse files
committed
msgpack: add optional msgpack.v5 support
The msgpack.v5 code located in msgpack_v5.go and msgpack_v5_helper_test.go for tests. It is the same logic for submodules. An user can use msgpack.v5 with a build tag: go_tarantool_msgpack_v5 Part of #124
1 parent 19da011 commit 2d9153b

16 files changed

+304
-17
lines changed

CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ Versioning](http://semver.org/spec/v2.0.0.html) except to the first release.
1818
- queue-utube handling (#85)
1919
- Master discovery (#113)
2020
- SQL support (#62)
21+
- Optional msgpack.v5 usage (#124)
2122

2223
### Fixed
2324

README.md

+37
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,11 @@ faster than other packages according to public benchmarks.
2222
## Table of contents
2323

2424
* [Installation](#installation)
25+
* [Build tags](#build-tags)
2526
* [Documentation](#documentation)
2627
* [API reference](#api-reference)
2728
* [Walking\-through example](#walking-through-example)
29+
* [msgpack.v5 migration](#msgpackv5-migration)
2830
* [Contributing](#contributing)
2931
* [Alternative connectors](#alternative-connectors)
3032

@@ -51,6 +53,17 @@ This should put the source and binary files in subdirectories of
5153
`github.com/tarantool/go-tarantool` to the `import {...}` section at the start
5254
of any Go program.
5355

56+
### Build tags
57+
58+
To replace usage of `msgpack.v2` with `msgpack.v5` you need to use the build
59+
tag:
60+
```
61+
go_tarantool_msgpack_v5
62+
```
63+
**Note:** In future releases, `msgpack.v5` may be used by default. We recommend
64+
to read [msgpack.v5 migration notes](#msgpackv5-migration) and try to
65+
use msgpack.v5 before the changes.
66+
5467
## Documentation
5568

5669
Read the [Tarantool documentation][tarantool-doc-data-model-url]
@@ -121,6 +134,30 @@ There are two parameters:
121134
* a space number (it could just as easily have been a space name), and
122135
* a tuple.
123136

137+
### msgpack.v5 migration
138+
139+
Most function names and argument types in `msgpack.v5` and `msgpack.v2`
140+
have not changed (except at least `EncodeInt`, `EncodeUint` and `RegisterExt`).
141+
But there are a lot of changes in a logic of encoding and deconding. On the plus
142+
side the migration seems easy, but on the minus side you need to be very
143+
careful.
144+
145+
First of all, `EncodeInt8`, `EncodeInt16`, `EncodeInt32`, `EncodeInt64`
146+
and `EncodeUint*` analogues at `msgpack.v5` encode numbers as is without loss of
147+
type. In `msgpack.v2` the type of a number is reduced to a value.
148+
149+
Secondly, a base decoding function does not convert numbers to `int64` or
150+
`uint64`. The change makes manual type conversions much more difficult and can
151+
lead to runtime erros with an old code. We recommend not use type conversions
152+
and give preference to `*Typed` functions (besides, it's faster).
153+
154+
There are also changes in the logic that can lead to errors in the old code,
155+
[as example](https://github.com/vmihailenco/msgpack/issues/327). Although in
156+
`msgpack.v5` has added some functions for the logic tuning see
157+
[UseLooseInterfaceDecoding](https://pkg.go.dev/github.com/vmihailenco/msgpack/v5#Decoder.UseLooseInterfaceDecoding), [UseCompactInts](https://pkg.go.dev/github.com/vmihailenco/msgpack/v5#Encoder.UseCompactInts) etc, but it is still impossible
158+
to achieve full compliance of behavior between `msgpack.v5` and `msgpack.v2`. So
159+
we don't go this way. We use standart settings if it possible.
160+
124161
## Contributing
125162

126163
See [the contributing guide](CONTRIBUTING.md) for detailed instructions on how

go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,5 @@ require (
99
google.golang.org/appengine v1.6.7 // indirect
1010
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
1111
gopkg.in/vmihailenco/msgpack.v2 v2.9.2
12+
github.com/vmihailenco/msgpack/v5 v5.3.5
1213
)

go.sum

+5
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,13 @@ github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLA
1212
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
1313
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
1414
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
15+
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
1516
github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMTY=
1617
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
18+
github.com/vmihailenco/msgpack/v5 v5.3.5 h1:5gO0H1iULLWGhs2H5tbAHIZTV8/cYafcFOr9znI5mJU=
19+
github.com/vmihailenco/msgpack/v5 v5.3.5/go.mod h1:7xyJ9e+0+9SaZT0Wt1RGleJXzli6Q/V5KbhBonMG9jc=
20+
github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
21+
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
1722
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
1823
golang.org/x/net v0.0.0-20190603091049-60506f45cf65 h1:+rhAzEzT3f4JtomfC371qB+0Ola2caSKcY69NUBZrRQ=
1924
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=

msgpack.go

+16-13
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package tarantool
25

36
import (
@@ -54,53 +57,53 @@ func msgpackIsString(code byte) bool {
5457
}
5558

5659
func (s *single) DecodeMsgpack(d *msgpack.Decoder) error {
57-
return s.decodeMsgpackImpl(&Decoder{d})
60+
return s.decodeMsgpackImpl(&Decoder{Decoder: d})
5861
}
5962

6063
func (space *Space) DecodeMsgpack(d *msgpack.Decoder) error {
61-
return space.decodeMsgpackImpl(&Decoder{d})
64+
return space.decodeMsgpackImpl(&Decoder{Decoder: d})
6265
}
6366

6467
func (field *Field) DecodeMsgpack(d *msgpack.Decoder) error {
65-
return field.decodeMsgpackImpl(&Decoder{d})
68+
return field.decodeMsgpackImpl(&Decoder{Decoder: d})
6669
}
6770

6871
func (index *Index) DecodeMsgpack(d *msgpack.Decoder) error {
69-
return index.decodeMsgpackImpl(&Decoder{d})
72+
return index.decodeMsgpackImpl(&Decoder{Decoder: d})
7073
}
7174

7275
func (indexField *IndexField) DecodeMsgpack(d *msgpack.Decoder) error {
73-
return indexField.decodeMsgpackImpl(&Decoder{d})
76+
return indexField.decodeMsgpackImpl(&Decoder{Decoder: d})
7477
}
7578

7679
func (meta *ColumnMetaData) DecodeMsgpack(d *msgpack.Decoder) error {
77-
return meta.decodeMsgpackImpl(&Decoder{d})
80+
return meta.decodeMsgpackImpl(&Decoder{Decoder: d})
7881
}
7982

8083
func (info *SQLInfo) DecodeMsgpack(d *msgpack.Decoder) error {
81-
return info.decodeMsgpackImpl(&Decoder{d})
84+
return info.decodeMsgpackImpl(&Decoder{Decoder: d})
8285
}
8386

8487
func (k IntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
85-
return k.encodeMsgpackImpl(&Encoder{enc})
88+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
8689
}
8790

8891
func (k UintKey) EncodeMsgpack(enc *msgpack.Encoder) error {
89-
return k.encodeMsgpackImpl(&Encoder{enc})
92+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
9093
}
9194

9295
func (k StringKey) EncodeMsgpack(enc *msgpack.Encoder) error {
93-
return k.encodeMsgpackImpl(&Encoder{enc})
96+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
9497
}
9598

9699
func (k IntIntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
97-
return k.encodeMsgpackImpl(&Encoder{enc})
100+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
98101
}
99102

100103
func (o Op) EncodeMsgpack(enc *msgpack.Encoder) error {
101-
return o.encodeMsgpackImpl(&Encoder{enc})
104+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
102105
}
103106

104107
func (o OpSplice) EncodeMsgpack(enc *msgpack.Encoder) error {
105-
return o.encodeMsgpackImpl(&Encoder{enc})
108+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
106109
}

msgpack_helper_test.go

+7-4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package tarantool_test
25

36
import (
@@ -6,17 +9,17 @@ import (
69
)
710

811
func (m *Member) EncodeMsgpack(e *msgpack.Encoder) error {
9-
return m.encodeMsgpackImpl(&Encoder{e})
12+
return m.encodeMsgpackImpl(&Encoder{Encoder: e})
1013
}
1114

1215
func (m *Member) DecodeMsgpack(d *msgpack.Decoder) error {
13-
return m.decodeMsgpackImpl(&Decoder{d})
16+
return m.decodeMsgpackImpl(&Decoder{Decoder: d})
1417
}
1518

1619
func (c *Tuple2) EncodeMsgpack(e *msgpack.Encoder) error {
17-
return c.encodeMsgpackImpl(&Encoder{e})
20+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
1821
}
1922

2023
func (c *Tuple2) DecodeMsgpack(d *msgpack.Decoder) error {
21-
return c.decodeMsgpackImpl(&Decoder{d})
24+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
2225
}

msgpack_v5.go

+112
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package tarantool
5+
6+
import (
7+
"io"
8+
9+
"github.com/vmihailenco/msgpack/v5"
10+
"github.com/vmihailenco/msgpack/v5/msgpcode"
11+
)
12+
13+
type Encoder struct {
14+
*msgpack.Encoder
15+
}
16+
17+
type Decoder struct {
18+
*msgpack.Decoder
19+
}
20+
21+
func newEncoder(w io.Writer) *Encoder {
22+
enc := msgpack.NewEncoder(w)
23+
return &Encoder{Encoder: enc}
24+
}
25+
26+
func newDecoder(r io.Reader) *Decoder {
27+
dec := msgpack.NewDecoder(r)
28+
dec.SetMapDecoder(func(dec *msgpack.Decoder) (interface{}, error) {
29+
return dec.DecodeUntypedMap()
30+
})
31+
return &Decoder{Decoder: dec}
32+
}
33+
34+
func (e *Encoder) encodeUintImpl(v uint64) error {
35+
return e.EncodeUint(v)
36+
}
37+
38+
func (e *Encoder) encodeIntImpl(v int64) error {
39+
return e.EncodeInt(v)
40+
}
41+
42+
func msgpackIsUint(code byte) bool {
43+
return code == msgpcode.Uint8 || code == msgpcode.Uint16 ||
44+
code == msgpcode.Uint32 || code == msgpcode.Uint64 ||
45+
msgpcode.IsFixedNum(code)
46+
}
47+
48+
func msgpackIsMap(code byte) bool {
49+
return code == msgpcode.Map16 || code == msgpcode.Map32 || msgpcode.IsFixedMap(code)
50+
}
51+
52+
func msgpackIsArray(code byte) bool {
53+
return code == msgpcode.Array16 || code == msgpcode.Array32 ||
54+
msgpcode.IsFixedArray(code)
55+
}
56+
57+
func msgpackIsString(code byte) bool {
58+
return msgpcode.IsFixedString(code) || code == msgpcode.Str8 ||
59+
code == msgpcode.Str16 || code == msgpcode.Str32
60+
}
61+
62+
func (s *single) DecodeMsgpack(d *msgpack.Decoder) error {
63+
return s.decodeMsgpackImpl(&Decoder{Decoder: d})
64+
}
65+
66+
func (space *Space) DecodeMsgpack(d *msgpack.Decoder) error {
67+
return space.decodeMsgpackImpl(&Decoder{Decoder: d})
68+
}
69+
70+
func (field *Field) DecodeMsgpack(d *msgpack.Decoder) error {
71+
return field.decodeMsgpackImpl(&Decoder{Decoder: d})
72+
}
73+
74+
func (index *Index) DecodeMsgpack(d *msgpack.Decoder) error {
75+
return index.decodeMsgpackImpl(&Decoder{Decoder: d})
76+
}
77+
78+
func (indexField *IndexField) DecodeMsgpack(d *msgpack.Decoder) error {
79+
return indexField.decodeMsgpackImpl(&Decoder{Decoder: d})
80+
}
81+
82+
func (meta *ColumnMetaData) DecodeMsgpack(d *msgpack.Decoder) error {
83+
return meta.decodeMsgpackImpl(&Decoder{Decoder: d})
84+
}
85+
86+
func (info *SQLInfo) DecodeMsgpack(d *msgpack.Decoder) error {
87+
return info.decodeMsgpackImpl(&Decoder{Decoder: d})
88+
}
89+
90+
func (k IntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
91+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
92+
}
93+
94+
func (k UintKey) EncodeMsgpack(enc *msgpack.Encoder) error {
95+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
96+
}
97+
98+
func (k StringKey) EncodeMsgpack(enc *msgpack.Encoder) error {
99+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
100+
}
101+
102+
func (k IntIntKey) EncodeMsgpack(enc *msgpack.Encoder) error {
103+
return k.encodeMsgpackImpl(&Encoder{Encoder: enc})
104+
}
105+
106+
func (o Op) EncodeMsgpack(enc *msgpack.Encoder) error {
107+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
108+
}
109+
110+
func (o OpSplice) EncodeMsgpack(enc *msgpack.Encoder) error {
111+
return o.encodeMsgpackImpl(&Encoder{Encoder: enc})
112+
}

msgpack_v5_helper_test.go

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package tarantool_test
5+
6+
import (
7+
. "github.com/tarantool/go-tarantool"
8+
"github.com/vmihailenco/msgpack/v5"
9+
)
10+
11+
func (m *Member) EncodeMsgpack(e *msgpack.Encoder) error {
12+
return m.encodeMsgpackImpl(&Encoder{Encoder: e})
13+
}
14+
15+
func (m *Member) DecodeMsgpack(d *msgpack.Decoder) error {
16+
return m.decodeMsgpackImpl(&Decoder{Decoder: d})
17+
}
18+
19+
func (c *Tuple2) EncodeMsgpack(e *msgpack.Encoder) error {
20+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
21+
}
22+
23+
func (c *Tuple2) DecodeMsgpack(d *msgpack.Decoder) error {
24+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
25+
}

queue/msgpack.go

+3
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package queue
25

36
import (

queue/msgpack_helper_test.go

+3
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package queue_test
25

36
import (

queue/msgpack_v5.go

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package queue
5+
6+
import (
7+
"github.com/tarantool/go-tarantool"
8+
"github.com/vmihailenco/msgpack/v5"
9+
)
10+
11+
func (qd *queueData) DecodeMsgpack(d *msgpack.Decoder) error {
12+
return qd.decodeMsgpackImpl(&tarantool.Decoder{Decoder: d})
13+
}
14+
15+
func (t *Task) DecodeMsgpack(d *msgpack.Decoder) error {
16+
return t.decodeMsgpackImpl(&tarantool.Decoder{Decoder: d})
17+
}

queue/msgpack_v5_helper_test.go

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
//go:build go_tarantool_msgpack_v5
2+
// +build go_tarantool_msgpack_v5
3+
4+
package queue_test
5+
6+
import (
7+
. "github.com/tarantool/go-tarantool"
8+
"github.com/vmihailenco/msgpack/v5"
9+
)
10+
11+
func (c *customData) DecodeMsgpack(d *msgpack.Decoder) error {
12+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
13+
}
14+
15+
func (c *customData) EncodeMsgpack(e *msgpack.Encoder) error {
16+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
17+
}
18+
19+
func (c *dummyData) DecodeMsgpack(d *msgpack.Decoder) error {
20+
return c.decodeMsgpackImpl(&Decoder{Decoder: d})
21+
}
22+
23+
func (c *dummyData) EncodeMsgpack(e *msgpack.Encoder) error {
24+
return c.encodeMsgpackImpl(&Encoder{Encoder: e})
25+
}

uuid/msgpack.go

+3
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
//go:build !go_tarantool_msgpack_v5
2+
// +build !go_tarantool_msgpack_v5
3+
14
package uuid
25

36
import (

0 commit comments

Comments
 (0)