Skip to content

(*smallWBuf).Write holds memory #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
SergeiHaidukov opened this issue Nov 30, 2020 · 3 comments
Closed

(*smallWBuf).Write holds memory #95

SergeiHaidukov opened this issue Nov 30, 2020 · 3 comments

Comments

@SergeiHaidukov
Copy link

SergeiHaidukov commented Nov 30, 2020

Hello !
I call a function in tarantool and pass ~10MB []map[string]interface{}. After that, memory is allocated on the heap, but it is not released after the procedure is executed. What can be done about it ?

Showing top 10 nodes out of 62

  flat  flat%   sum%        cum   cum%
0.78GB 43.04% 43.04%     0.78GB 43.04%  github.com/tarantool/go-tarantool.(*smallWBuf).Write
0.63GB 34.80% 77.84%     0.63GB 34.80%  github.com/tarantool/go-tarantool.(*smallWBuf).WriteString
0.11GB  5.78% 83.62%     0.19GB 10.43%  gopkg.in/vmihailenco/msgpack%2ev2.decodeMap
0.10GB  5.68% 89.30%     0.10GB  5.68%  github.com/tarantool/go-tarantool.(*smallWBuf).WriteByte
0.05GB  2.87% 92.16%     0.05GB  2.87%  github.com/json-iterator/go.(*Stream).Write
@funny-falcon
Copy link

Yes. I reuse write buffer, and didn't bother with eventual dropping it out.
In other connectors I drop buffer once in 1000 calls. Should I do it here as well?
You can decrease impact by setting Concurrency to lower value, I suppose. Try to set it to 1.

(btw, 10MB data in a single call is not too good for single core tarantool. But probably it is not issue for you)

@SergeiHaidukov
Copy link
Author

Thanks for the answer!
With Concurrency:1 consumes less memory. I suppose it makes sense to drop the buffer.
Can make this behavior configurable. Specify the number of calls.

@funny-falcon
Copy link

Try latest version, please

oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

    Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

    Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

	Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#112).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

	Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 2, 2022
Overview

This release adds a number of features. Also it significantly improves
testing, CI and documentation.

Breaking changes

    There are no breaking changes in the release.

New features

    Support UUID type in msgpack (#90).

    queue-utube handling (#85).

    Master discovery (#113).

    SQL support (#62).

Bugfixes

    Reset buffer if its average use size smaller than quater of
    capacity (#95).

Testing

    Coveralls support (#149).

    Reusable testing workflow (integration testing with latest
    Tarantool) (#112).

    Simple CI based on GitHub actions (#114).

    Handle everything with `go test` (#115).

    Fix queue tests (#107).

    Make test case consistent with comments (#105).

Other

    Go modules support (#91).

    Update API documentation: comments and examples (#123).
oleg-jukovec added a commit that referenced this issue Jun 2, 2022
Overview

This release adds a number of features. Also it significantly improves
testing, CI and documentation.

Breaking changes

    There are no breaking changes in the release.

New features

    Support UUID type in msgpack (#90).

    queue-utube handling (#85).

    Master discovery (#113).

    SQL support (#62).

Bugfixes

    Reset buffer if its average use size smaller than quater of
    capacity (#95).

Testing

    Coveralls support (#149).

    Reusable testing workflow (integration testing with latest
    Tarantool) (#112).

    Simple CI based on GitHub actions (#114).

    Handle everything with `go test` (#115).

    Fix queue tests (#107).

    Make test case consistent with comments (#105).

Other

    Go modules support (#91).

    Update API documentation: comments and examples (#123).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants