Skip to content

Commit 603b498

Browse files
committed
Add RFC for foreach-parallel feature
1 parent 249e8d8 commit 603b498

File tree

1 file changed

+110
-0
lines changed

1 file changed

+110
-0
lines changed
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
---
2+
RFC: RFCnnnn
3+
Author: Paul Higinbotham
4+
Status: Draft
5+
SupercededBy: N/A
6+
Version: 1.0
7+
Area: Engine
8+
Comments Due: July 1, 2019
9+
Plan to implement: Yes
10+
---
11+
12+
# Implement PowerShell language foreach -parallel
13+
14+
Windows PowerShell currently supports the foreach language keyword with the -parallel switch flag, but only for workflow scripts.
15+
16+
```powershell
17+
18+
workflow wf1 {
19+
$list = 1..5
20+
foreach -parallel -throttlelimit 5 ($item in $list) {
21+
Start-Sleep -Seconds 1
22+
Write-Output "Output $item"
23+
}
24+
}
25+
26+
```
27+
28+
This will run the script block with each value in the `$list` array, in parallel using workflow jobs.
29+
However, workflow is not supported in PowerShell Core 6, partly because it is a Windows only solution but also because it is cumbersome to use.
30+
In addition the workflow implementation is very heavy weight, using lots of system resources.
31+
32+
This is a proposal to re-implement `foreach -parallel` in PowerShell Core, using PowerShell's support for concurrency via Runspaces.
33+
It is similar to the [ThreadJob module](https://www.powershellgallery.com/packages/ThreadJob/1.1.2) except that it becomes part of the PowerShell language via `foreach -parallel`.
34+
35+
## Motivation
36+
37+
As a PowerShell User,
38+
I can do simple fan-out concurrency from within the language, without having to obtain and load a separate module or deal with PowerShell jobs.
39+
40+
## Specification
41+
42+
The PowerShell `foreach -parallel` language keyword will be re-implemented to perform invoke script blocks in parallel, similar to how it works for workflow functions except that script blocks will be invoked on threads within the same process rather than in workflow jobs running in separate processes.
43+
The default behavior is to fan-out script block execution to multiple threads, and then wait for all threads to finish.
44+
However, a `-asjob` switch will also be supported that returns a PowerShell job object for asynchronous use.
45+
If the number of foreach iterations exceed the throttle limit value, then only the throttle limit number of threads are created at a time and the rest are queued until a running thread becomes available.
46+
47+
### Supported foreach parameters
48+
49+
- `-parallel`
50+
- `-throttlelimit`
51+
- `-timeout`
52+
- `-asjob`
53+
54+
### P0 Features
55+
56+
- `foreach -parallel` fans out script block execution to threads, along with a bound single foreach iteration value
57+
58+
- `-throttlelimit` parameter value specifies the maximum number of threads that can run at one time
59+
60+
- `-timeout` parameter value specifies a maximum time to wait for all iterations to complete, after which 'stop' will be called on all running script blocks to terminate execution
61+
62+
- `-asjob` switch causes foreach to return a PowerShell job object that is used to asynchronously monitor execution
63+
64+
- When a job object is returned, it will be compatible with all relevant job cmdlets
65+
66+
- All script blocks running in parallel will run isolated from each other.
67+
Only foreach iteration objects will be passed to the parallel script block.
68+
69+
### Data stream handling
70+
71+
`foreach -parallel` will use normal PowerShell pipes to return various data streams.
72+
Data will be returned in order received.
73+
Except when `-asjob` switch is used, in which case a single job object is returned.
74+
The returned job object will contain an array of child jobs that represent each iteration of the foreach.
75+
76+
### Examples
77+
78+
```powershell
79+
$computerNames = 'computer1','computer2','computer3','computer4','computer5'
80+
$logs = foreach -parallel -throttle 10 -timeout 300 ($computer in $computerNames)
81+
{
82+
Get-Logs -ComputerName $computer
83+
}
84+
```
85+
86+
```powershell
87+
$computerNames = 'computer1','computer2','computer3','computer4','computer5'
88+
$job = foreach -parallel -asjob ($computer in $computerNames)
89+
{
90+
Get-Logs -ComputerName $computer
91+
}
92+
$logs = $job | Wait-Job | Receive-Job
93+
```
94+
95+
```powershell
96+
$params += @{
97+
$argTitle = "Title1"
98+
$argValue = 102
99+
}
100+
foreach -parallel ($param in $params)
101+
{
102+
c:\scripts\ToRun.ps1 @param
103+
}
104+
```
105+
106+
## Alternate Proposals and Considerations
107+
108+
One alternative is to create a `ForEach-Parallel` cmdlet instead of re-implementing the `foreach -parallel` keyword.
109+
This would work well but would not be as useful as making it part of the PowerShell language.
110+
But if re-implementing the foreach keyword becomes problematic, it would be a good fallback solution.

0 commit comments

Comments
 (0)