Skip to content

[breaking] Fix gRPC streaming synchronization on close #1787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 1, 2022

Conversation

cmaglie
Copy link
Member

@cmaglie cmaglie commented Jun 30, 2022

Please check if the PR fulfills these requirements

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • UPGRADING.md has been updated with a migration guide (for breaking changes)

What kind of change does this PR introduce?
Adds some synchronization to properly handle streaming close.

What is the current behavior?
Sometimes the last part of the streaming call is lost (it could be the last piece of the compile output for example).
Moreover, the gRPC thread is leaked, even if the amount of resources leakied is negligible, this may add up if the daemon is left running for a long time with a huge number of compiles.

This issue has been there for a long time, the gRPC rate-limiting implemented here #1759 just made it more visible.

What is the new behavior?
The streaming synchronization is now properly handled.

Does this PR introduce a breaking change, and is titled accordingly?
The function that we use internally FeedStreamTo has been changed.
Anyway since this is a public-facing API the change has been documented.

@cmaglie cmaglie self-assigned this Jun 30, 2022
@cmaglie cmaglie added criticality: highest Of highest impact type: imperfection Perceived defect in any part of project and removed criticality: highest Of highest impact labels Jun 30, 2022
@per1234 per1234 added topic: code Related to content of the project itself topic: gRPC Related to the gRPC interface labels Jun 30, 2022
@cmaglie cmaglie requested a review from a team June 30, 2022 09:39
@cmaglie cmaglie force-pushed the fix_grpc_streaming_sync_on_close branch from 9a0f393 to d9b973a Compare June 30, 2022 09:46
@cmaglie
Copy link
Member Author

cmaglie commented Jun 30, 2022

I've also found that io.Pipe is non-buffering, this means that the write side has to wait until the reading side has completely read the data. So I'll revert #1759 too, until I found a better solution.

@cmaglie
Copy link
Member Author

cmaglie commented Jun 30, 2022

I've also found that io.Pipe is non-buffering, this means that the write side has to wait until the reading side has completely read the data. So I'll revert #1759 too, until I found a better solution.

Well, actually I found an implementation of buffered pipes under MIT license, that seems to fix the issue. I've updated the PR, now it should be production-ready.

@cmaglie cmaglie force-pushed the fix_grpc_streaming_sync_on_close branch from 94ad507 to 08f22db Compare June 30, 2022 10:45
@davegarthsimpson
Copy link

linked to #1073, this change resolves the issue of missing compile output in the IDE.

Copy link
Contributor

@per1234 per1234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes arduino/arduino-ide#1073 for me.

Thanks Cristian!

@cmaglie cmaglie merged commit 6ac5f7a into arduino:master Jul 1, 2022
@cmaglie cmaglie deleted the fix_grpc_streaming_sync_on_close branch July 1, 2022 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
criticality: highest Of highest impact topic: code Related to content of the project itself topic: gRPC Related to the gRPC interface type: imperfection Perceived defect in any part of project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants