Skip to content

Prevent leaking of parallel decomposition into Job state #585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
elizarov opened this issue Sep 19, 2018 · 0 comments
Closed

Prevent leaking of parallel decomposition into Job state #585

elizarov opened this issue Sep 19, 2018 · 0 comments
Labels
bug for 1.0 release

Comments

@elizarov
Copy link
Contributor

When Job uses parallel decomposition of its work by launching child coroutines it accidentally leaks its internal implementation details. When a child coroutine crashes with exception it cancels its parent, thereby obviously revealing this failure with cancelled state of the parent job. However, when the parent itself crashes, it still technically in completed state (not cancelled), so by observing the state of the one can find out implementation details of a job.

For example this code:

val job = launch { 
    doSomething()
}

would produce a completed (failed) state of the job if doSomething() crashed, while this code:

val job = launch {
    launch {  doSomething() } // delegate to child
}

would produce a cancelled state of the job if doSomething() crashes.

We want to eliminate this leak. As a solution, when child fails it does not cancel the parent anymore, but uses an internal mechanism to fail the parent.

This issues supersedes #220
@elizarov elizarov mentioned this issue Sep 19, 2018
Closed
elizarov added a commit that referenced this issue Sep 19, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
adcbd5e 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 19, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
20258bd 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 20, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
444823f 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 20, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
1c9afc9 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 20, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
0aa06dd 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 20, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
b0a3c84 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 20, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
b56e194 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
@elizarov elizarov mentioned this issue Sep 20, 2018
Merged
elizarov added a commit that referenced this issue Sep 20, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
2999129 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
elizarov added a commit that referenced this issue Sep 21, 2018
@elizarov
Fix exception aggregation to ensure atomic handling of exceptions
3777779 
* Removed legacy onFinishing handler support from JobSupport.
  - It is no longer needed, because handleJobException covers #208
* Fixed bugs that were masked by cancelling parent from onFinishing.
* Consistent "Finishing" state was introduced in internal machinery:
  - Job enters finishing state when it is failing or it is completing
    and has children
  - Finishing state cleanly aggregates all failures, tracks cancellation
    and completion status
* Job.isFailed is introduced as a consistent way to query the "failing"
  state of the job that was previously only implicitly available via
  invokeOnCompletion handler (cause != null means a failed job) and
  the documentation for both Job & Deferred is updated to reflect that.
* Source-incompatible change: Job.invokeOnCompletion boolean parameter is
  change to onFailing. Such handlers are now invoked as soon as the job
  starts failing and the root cause exception of the failure is consistently
  passed to all the handlers.
* The following internal methods were introduced to facilitate this:
  - Job.failParent(exception) is used by child to signal failure to parent
  - Job.cancelChild(parentJob) is used by parent to cancel child.
* Child never aggregates exception received from it parent, but uses
  it as it root cause if there is no other exception.
* JobSupport.handleJobException() logic for launch/actor is split into:
  - failParent - can be invoked multiple times on race;
  - handleJobException which is invoked exactly once.
* Exception materiazization is much lazier now, which should
  significantly improve performance when cancelling large hierarchies.
* Other minor perf improvements in JobSupport code.

Fixes #585
@qwwdfsad qwwdfsad mentioned this issue Jun 14, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug for 1.0 release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@elizarov