Skip to content

Use mc.buf as scratchpad for intarpolation #318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 18, 2015

Conversation

methane
Copy link
Member

@methane methane commented Feb 18, 2015

benchmark                  old ns/op     new ns/op     delta
BenchmarkInterpolation     1900          1363          -28.26%

benchmark                  old allocs     new allocs     delta
BenchmarkInterpolation     2              1              -50.00%

benchmark                  old bytes     new bytes     delta
BenchmarkInterpolation     448           160           -64.29%

@methane
Copy link
Member Author

methane commented Feb 18, 2015

I can remove last one allocation: Stop using mc.exec() and build packet directly.
But it may be dirty a little. May I do it?

@methane methane force-pushed the interpolate-reduce-alloc branch from 9b7ffba to 545d7e7 Compare February 18, 2015 05:33
@arnehormann
Copy link
Member

I'm opposed to it, I like having the packet building in one place.
But I do like the part where you use mc.buf.takeSmallBuffer (or takeCompleteBuffer) as a scratch space... the packet construction code doesn't need it yet.
By the way - why did you change buffer.go?

return "", driver.ErrSkip
}
estimated += len(query)
buffSize := max
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bufSize

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@julienschmidt
Copy link
Member

What allocation exactly are you trying to eliminate?


buf := make([]byte, 0, estimated)
buf := mc.buf.takeBuffer(buffSize)[:0]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allocation is removed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean by stop using mc.exec()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think he means the conversion of the query from []byte to string to []byte

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mean it.
Future Go may be able to optimize it, but I think Go 1.5 will not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Draft it. If it doesn't get too dirty, I would be fine with it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if that is fine, it can be applied in other areas, too.
We could e.g. have a variable-cached version of a full COM_QUIT and COM_STMT_CLOSE packet. The former can be sent as is, the letter only has to have the sequence number changed...
But that scatters the network Write calls to multiple places and makes maintaining the driver a little harder.

@methane methane force-pushed the interpolate-reduce-alloc branch 2 times, most recently from b6519ad to cc1ec6b Compare February 18, 2015 09:39
@methane
Copy link
Member Author

methane commented Feb 18, 2015

@arnehormann

But I do like the part where you use mc.buf.takeSmallBuffer (or takeCompleteBuffer) as a scratch space... the packet construction code doesn't need it yet.

packet size is len(query) + 4. This size is used just after interpolate.
So I think reserving same size is good idea.

By the way - why did you change buffer.go?

fixed.

@methane methane force-pushed the interpolate-reduce-alloc branch from cc1ec6b to 4ea43cb Compare February 18, 2015 09:53
benchmark                  old ns/op     new ns/op     delta
BenchmarkInterpolation     1900          1363          -28.26%

benchmark                  old allocs     new allocs     delta
BenchmarkInterpolation     2              1              -50.00%

benchmark                  old bytes     new bytes     delta
BenchmarkInterpolation     448           160           -64.29%
@methane methane force-pushed the interpolate-reduce-alloc branch from 4ea43cb to e3e2d32 Compare February 18, 2015 09:55
@methane
Copy link
Member Author

methane commented Feb 18, 2015

Now interpolateParams() uses mc.buf.takeCompleteBuffer().
And I've eliminated estimateParamLength().

@methane methane changed the title Use mc.buf while interpolating Use mc.buf while as scratchpad for intarpolation Feb 18, 2015
@methane methane changed the title Use mc.buf while as scratchpad for intarpolation Use mc.buf as scratchpad for intarpolation Feb 18, 2015
@arnehormann
Copy link
Member

LGTM!

arnehormann added a commit that referenced this pull request Feb 18, 2015
Use mc.buf as scratchpad for intarpolation
@arnehormann arnehormann merged commit 60fe63a into go-sql-driver:master Feb 18, 2015
@julienschmidt julienschmidt added this to the v1.3 milestone Feb 18, 2015
@methane methane deleted the interpolate-reduce-alloc branch February 18, 2015 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants