Skip to content

Isolate PSES dependencies from PowerShell on load + make PSES a pure binary module #1118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 62 commits into from
Dec 10, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
55485e7
First work
Nov 19, 2019
57ce346
More progress
Nov 20, 2019
ce7f50f
Enable compilation
Nov 21, 2019
9814824
Build increments
Nov 22, 2019
a2ab488
Add initial load context
Nov 23, 2019
1be54f2
Fix loading issues
Nov 25, 2019
23abd87
Write session file
Nov 26, 2019
e50e95c
Fix startup connection issue
Nov 26, 2019
884a742
Add logging
Nov 26, 2019
3ab36d5
Add doc comments + some cleanup
Nov 27, 2019
dbd3081
Plumb host logging into PSES
Nov 27, 2019
8fb83f9
Fix compile issues
Nov 27, 2019
7f04198
First part of host logger handover
Nov 27, 2019
0580aea
Complete host logging deregistration
rjmholt Nov 27, 2019
b073e53
Add more logging on startup
rjmholt Nov 27, 2019
b2eebcc
Migrate startup script
rjmholt Nov 27, 2019
fd6d25e
Fix Windows PowerShell startup
rjmholt Nov 27, 2019
14fc131
Add event handler logging
rjmholt Nov 27, 2019
accfa52
Add warning for low .NET versions
rjmholt Nov 27, 2019
1cebbb6
Fix VSCode and Command module imports
rjmholt Nov 27, 2019
640f97b
Add constrained language mode check
rjmholt Nov 27, 2019
9bd8789
Add copyright headers
rjmholt Nov 27, 2019
c18b5e7
Remove extraneous host files
rjmholt Nov 27, 2019
72da273
Warn user about read key on crash
rjmholt Nov 28, 2019
97bbae7
Add configuration validation
rjmholt Nov 28, 2019
d744818
Add more logging
rjmholt Nov 28, 2019
8c34147
Fix profile path issue
Dec 2, 2019
267f5d2
Fix host logging
Dec 2, 2019
51491af
Improve asm loading logging
Dec 2, 2019
8582be4
Attempt async logging
Dec 2, 2019
d7c6981
Fix UnixConsoleEcho dependencies in build
rjmholt Dec 3, 2019
7680ccc
Update src/PowerShellEditorServices.Hosting/StartEditorServicesComman…
rjmholt Dec 3, 2019
fb703c1
Address some feedback
rjmholt Dec 3, 2019
3929b81
Fix Serilog file logging issue
rjmholt Dec 3, 2019
2c3f4a2
Add PowerShell version to log
rjmholt Dec 3, 2019
c0e0e64
Fix log flushing issue
rjmholt Dec 3, 2019
a14a2b6
Configure PSReadLine correctly
rjmholt Dec 3, 2019
ed702c2
Fix log thread collision issue
rjmholt Dec 3, 2019
bfe4430
Contract log level names
rjmholt Dec 3, 2019
a515b64
Fix integration tests
Dec 4, 2019
877a887
Fix manifest import in build
Dec 4, 2019
b7476f3
Use full version for build info
Dec 4, 2019
9fe2f32
Actually fix buildinfo
Dec 4, 2019
ed15096
Address @SeeminglyScience's remaining feedback
Dec 4, 2019
33b7881
Code tweaks
Dec 5, 2019
b0c845d
Fix lint issues
Dec 6, 2019
6a3c637
Make simple feedback changes
rjmholt Dec 7, 2019
ff2b058
Improve startup logging
rjmholt Dec 7, 2019
9ea332b
Load PSES at right time
rjmholt Dec 7, 2019
6b0ee5b
Fix PSModulePath in setup
rjmholt Dec 7, 2019
f9f4643
Increase startup banner spacing
rjmholt Dec 7, 2019
36bc03a
Move BuildInfo
rjmholt Dec 7, 2019
8d7f7cd
Minor buildinfo change
rjmholt Dec 7, 2019
dd921d3
Add comment to .NET Framework dependency resolution
rjmholt Dec 7, 2019
ef720f1
Update BuildInfo unindex
rjmholt Dec 7, 2019
2ab5db7
Small fixes
rjmholt Dec 7, 2019
f20d1af
Move files into nice dirs
rjmholt Dec 7, 2019
edf7bbd
Fix tests
rjmholt Dec 7, 2019
69b9583
Update PowerShellEditorServices.build.ps1
rjmholt Dec 9, 2019
7951fb3
Update src/PowerShellEditorServices.Hosting/Internal/EditorServicesRu…
rjmholt Dec 9, 2019
dde7ea7
Skip adding SMA asms to module
Dec 9, 2019
ddb97af
Fix codacy bits
Dec 9, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,6 @@ trim_trailing_whitespace = true

[*.{ps1xml,props,xml,yaml}]
indent_size = 2

# CA1303: Do not pass literals as localized parameters
dotnet_diagnostic.CA1303.severity = none
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ docs/metadata/
*.zip

# Generated build info file
src/PowerShellEditorServices/Hosting/BuildInfo.cs
src/PowerShellEditorServices.Hosting/BuildInfo.cs

# quickbuild.exe
/VersionGeneratingLogs/
Expand Down
280 changes: 101 additions & 179 deletions PowerShellEditorServices.build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -19,149 +19,23 @@ param(
#Requires -Modules @{ModuleName="InvokeBuild";ModuleVersion="3.2.1"}

$script:IsUnix = $PSVersionTable.PSEdition -and $PSVersionTable.PSEdition -eq "Core" -and !$IsWindows
$script:TargetPlatform = "netstandard2.0"
$script:TargetFrameworksParam = "/p:TargetFrameworks=`"$script:TargetPlatform`""
$script:RequiredSdkVersion = (Get-Content (Join-Path $PSScriptRoot 'global.json') | ConvertFrom-Json).sdk.version
$script:NugetApiUriBase = 'https://www.nuget.org/api/v2/package'
$script:ModuleBinPath = "$PSScriptRoot/module/PowerShellEditorServices/bin/"
$script:VSCodeModuleBinPath = "$PSScriptRoot/module/PowerShellEditorServices.VSCode/bin/"
$script:WindowsPowerShellFrameworkTarget = 'net461'
$script:NetFrameworkPlatformId = 'win'
$script:BuildInfoPath = [System.IO.Path]::Combine($PSScriptRoot, "src", "PowerShellEditorServices", "Hosting", "BuildInfo.cs")

$script:PSCoreModulePath = $null

$script:TestRuntime = @{
'Core' = 'netcoreapp2.1'
'Desktop' = 'net461'
}
$script:BuildInfoPath = [System.IO.Path]::Combine($PSScriptRoot, "src", "PowerShellEditorServices.Hosting", "BuildInfo.cs")

<#
Declarative specification of binary assets produced
in the build that need to be binplaced in the module.
Schema is:
{
<Output Path>: {
<Project Name>: [
<FilePath From Project Build Folder>
]
}
$script:NetRuntime = @{
Core = 'netcoreapp2.1'
Desktop = 'net461'
Standard = 'netstandard2.0'
}
#>
$script:RequiredBuildAssets = @{
$script:ModuleBinPath = @{
'PowerShellEditorServices' = @(
'publish/Microsoft.Extensions.DependencyInjection.Abstractions.dll',
'publish/Microsoft.Extensions.DependencyInjection.dll',
'publish/Microsoft.Extensions.FileSystemGlobbing.dll',
'publish/Microsoft.Extensions.Logging.Abstractions.dll',
'publish/Microsoft.Extensions.Logging.dll',
'publish/Microsoft.Extensions.Options.dll',
'publish/Microsoft.Extensions.Primitives.dll',
'publish/Microsoft.PowerShell.EditorServices.dll',
'publish/Microsoft.PowerShell.EditorServices.pdb',
'publish/Newtonsoft.Json.dll',
'publish/OmniSharp.Extensions.JsonRpc.dll',
'publish/OmniSharp.Extensions.LanguageProtocol.dll',
'publish/OmniSharp.Extensions.LanguageServer.dll',
'publish/OmniSharp.Extensions.DebugAdapter.dll',
'publish/OmniSharp.Extensions.DebugAdapter.Server.dll',
'publish/MediatR.dll',
'publish/MediatR.Extensions.Microsoft.DependencyInjection.dll',
'publish/runtimes/linux-64/native/libdisablekeyecho.so',
'publish/runtimes/osx-64/native/libdisablekeyecho.dylib',
'publish/Serilog.dll',
'publish/Serilog.Extensions.Logging.dll',
'publish/Serilog.Sinks.File.dll',
'publish/System.Reactive.dll',
'publish/UnixConsoleEcho.dll'
)
}

$script:VSCodeModuleBinPath = @{
'PowerShellEditorServices.VSCode' = @(
'Microsoft.PowerShell.EditorServices.VSCode.dll',
'Microsoft.PowerShell.EditorServices.VSCode.pdb'
)
}
}

<#
Declares the binary shims we need to make the netstandard DLLs hook into .NET Framework.
Schema is:
{
<Destination Bin Directory>: [{
'PackageName': <Package Name>,
'PackageVersion': <Package Version>,
'TargetRuntime': <Target .NET Runtime>,
'DllName'?: <Name of DLL to extract>
}]
}
#>
$script:RequiredNugetBinaries = @{
'Desktop' = @(
@{ PackageName = 'System.Security.Principal.Windows'; PackageVersion = '4.5.0'; TargetRuntime = 'net461' },
@{ PackageName = 'System.Security.AccessControl'; PackageVersion = '4.5.0'; TargetRuntime = 'net461' },
@{ PackageName = 'System.IO.Pipes.AccessControl'; PackageVersion = '4.5.1'; TargetRuntime = 'net461' }
)
}
$script:HostCoreOutput = "$PSScriptRoot/src/PowerShellEditorServices.Hosting/bin/$Configuration/$($script:NetRuntime.Core)/publish"
$script:HostDeskOutput = "$PSScriptRoot/src/PowerShellEditorServices.Hosting/bin/$Configuration/$($script:NetRuntime.Desktop)/publish"
$script:PsesOutput = "$PSScriptRoot/src/PowerShellEditorServices/bin/$Configuration/$($script:NetRuntime.Standard)/publish"
$script:VSCodeOutput = "$PSScriptRoot/src/PowerShellEditorServices.VSCode/bin/$Configuration/$($script:NetRuntime.Standard)/publish"

if (Get-Command git -ErrorAction SilentlyContinue) {
# ignore changes to this file
git update-index --assume-unchanged "$PSScriptRoot/src/PowerShellEditorServices.Host/BuildInfo/BuildInfo.cs"
}

if ($PSVersionTable.PSEdition -ne "Core") {
Add-Type -Assembly System.IO.Compression.FileSystem
}

function Restore-NugetAsmForRuntime {
param(
[ValidateNotNull()][string]$PackageName,
[ValidateNotNull()][string]$PackageVersion,
[string]$DllName,
[string]$DestinationPath,
[string]$TargetPlatform = $script:NetFrameworkPlatformId,
[string]$TargetRuntime = $script:WindowsPowerShellFrameworkTarget
)

$tmpDir = Join-Path $PSScriptRoot '.tmp'
if (-not (Test-Path $tmpDir)) {
New-Item -ItemType Directory -Path $tmpDir
}

if (-not $DllName) {
$DllName = "$PackageName.dll"
}

if ($DestinationPath -eq $null) {
$DestinationPath = Join-Path $tmpDir $DllName
} elseif (Test-Path $DestinationPath -PathType Container) {
$DestinationPath = Join-Path $DestinationPath $DllName
}

$packageDirPath = Join-Path $tmpDir "$PackageName.$PackageVersion"
if (-not (Test-Path $packageDirPath)) {
$guid = New-Guid
$tmpNupkgPath = Join-Path $tmpDir "$guid.zip"
if (Test-Path $tmpNupkgPath) {
Remove-Item -Force $tmpNupkgPath
}

try {
$packageUri = "$script:NugetApiUriBase/$PackageName/$PackageVersion"
Invoke-WebRequest -Uri $packageUri -OutFile $tmpNupkgPath
Expand-Archive -Path $tmpNupkgPath -DestinationPath $packageDirPath
} finally {
Remove-Item -Force $tmpNupkgPath -ErrorAction SilentlyContinue
}
}

$internalPath = [System.IO.Path]::Combine($packageDirPath, 'runtimes', $TargetPlatform, 'lib', $TargetRuntime, $DllName)

Copy-Item -Path $internalPath -Destination $DestinationPath -Force

return $DestinationPath
git update-index --assume-unchanged "$PSScriptRoot/src/PowerShellEditorServices.Hosting/BuildInfo.cs"
}

function Invoke-WithCreateDefaultHook {
Expand Down Expand Up @@ -284,11 +158,17 @@ task CreateBuildInfo -Before Build {
$buildVersion = "<development-build>"
$buildOrigin = "<development>"

if ($propsBody.VersionSuffix)
{
$propsXml = [xml](Get-Content -Raw -LiteralPath "$PSScriptRoot/PowerShellEditorServices.Common.props")
$propsBody = $propsXml.Project.PropertyGroup
$buildVersion = $propsBody.VersionPrefix
$buildVersion += '-' + $propsBody.VersionSuffix
}

# Set build info fields on build platforms
if ($env:TF_BUILD)
{
$psd1Path = [System.IO.Path]::Combine($PSScriptRoot, "module", "PowerShellEditorServices", "PowerShellEditorServices.psd1")
$buildVersion = (Import-PowerShellDataFile -LiteralPath $psd1Path).Version
$buildOrigin = "VSTS"
}

Expand All @@ -310,8 +190,8 @@ namespace Microsoft.PowerShell.EditorServices.Hosting
{
public static class BuildInfo
{
public const string BuildVersion = "$buildVersion";
public const string BuildOrigin = "$buildOrigin";
public static readonly string BuildVersion = "$buildVersion";
public static readonly string BuildOrigin = "$buildOrigin";
public static readonly System.DateTime? BuildTime = System.DateTime.Parse("$buildTime");
}
}
Expand All @@ -327,8 +207,15 @@ task SetupHelpForTests -Before Test {
}

task Build {
exec { & $script:dotnetExe publish -c $Configuration .\src\PowerShellEditorServices\PowerShellEditorServices.csproj -f $script:TargetPlatform }
exec { & $script:dotnetExe build -c $Configuration .\src\PowerShellEditorServices.VSCode\PowerShellEditorServices.VSCode.csproj $script:TargetFrameworksParam }
exec { & $script:dotnetExe publish -c $Configuration .\src\PowerShellEditorServices\PowerShellEditorServices.csproj -f $script:NetRuntime.Standard }
exec { & $script:dotnetExe publish -c $Configuration .\src\PowerShellEditorServices.Hosting\PowerShellEditorServices.Hosting.csproj -f $script:NetRuntime.Core }
if (-not $script:IsUnix)
{
exec { & $script:dotnetExe publish -c $Configuration .\src\PowerShellEditorServices.Hosting\PowerShellEditorServices.Hosting.csproj -f $script:NetRuntime.Desktop }
}

# Build PowerShellEditorServices.VSCode module
exec { & $script:dotnetExe publish -c $Configuration .\src\PowerShellEditorServices.VSCode\PowerShellEditorServices.VSCode.csproj -f $script:NetRuntime.Standard }
}

function DotNetTestFilter {
Expand All @@ -343,80 +230,115 @@ task TestServer {
Set-Location .\test\PowerShellEditorServices.Test\

if (-not $script:IsUnix) {
exec { & $script:dotnetExe test --logger trx -f $script:TestRuntime.Desktop (DotNetTestFilter) }
exec { & $script:dotnetExe test --logger trx -f $script:NetRuntime.Desktop (DotNetTestFilter) }
}

Invoke-WithCreateDefaultHook -NewModulePath $script:PSCoreModulePath {
exec { & $script:dotnetExe test --logger trx -f $script:TestRuntime.Core (DotNetTestFilter) }
exec { & $script:dotnetExe test --logger trx -f $script:NetRuntime.Core (DotNetTestFilter) }
}
}

task TestProtocol {
Set-Location .\test\PowerShellEditorServices.Test.Protocol\

if (-not $script:IsUnix) {
exec { & $script:dotnetExe test --logger trx -f $script:TestRuntime.Desktop (DotNetTestFilter) }
exec { & $script:dotnetExe test --logger trx -f $script:NetRuntime.Desktop (DotNetTestFilter) }
}

Invoke-WithCreateDefaultHook {
exec { & $script:dotnetExe test --logger trx -f $script:TestRuntime.Core (DotNetTestFilter) }
exec { & $script:dotnetExe test --logger trx -f $script:NetRuntime.Core (DotNetTestFilter) }
}
}

task TestHost {
Set-Location .\test\PowerShellEditorServices.Test.Host\

if (-not $script:IsUnix) {
exec { & $script:dotnetExe build -f $script:TestRuntime.Desktop }
exec { & $script:dotnetExe test -f $script:TestRuntime.Desktop (DotNetTestFilter) }
exec { & $script:dotnetExe build -f $script:NetRuntime.Desktop }
exec { & $script:dotnetExe test -f $script:NetRuntime.Desktop (DotNetTestFilter) }
}

exec { & $script:dotnetExe build -c $Configuration -f $script:TestRuntime.Core }
exec { & $script:dotnetExe test -f $script:TestRuntime.Core (DotNetTestFilter) }
exec { & $script:dotnetExe build -c $Configuration -f $script:NetRuntime.Core }
exec { & $script:dotnetExe test -f $script:NetRuntime.Core (DotNetTestFilter) }
}

task TestE2E {
Set-Location .\test\PowerShellEditorServices.Test.E2E\

$env:PWSH_EXE_NAME = if ($IsCoreCLR) { "pwsh" } else { "powershell" }
exec { & $script:dotnetExe test --logger trx -f $script:TestRuntime.Core (DotNetTestFilter) }
exec { & $script:dotnetExe test --logger trx -f $script:NetRuntime.Core (DotNetTestFilter) }
}

task LayoutModule -After Build {
$modulesDir = "$PSScriptRoot/module"
$psesVSCodeBinOutputPath = "$modulesDir/PowerShellEditorServices.VSCode/bin"
$psesOutputPath = "$modulesDir/PowerShellEditorServices"
$psesBinOutputPath = "$PSScriptRoot/module/PowerShellEditorServices/bin"
$psesDepsPath = "$psesBinOutputPath/Common"
$psesCoreHostPath = "$psesBinOutputPath/Core"
$psesDeskHostPath = "$psesBinOutputPath/Desktop"

foreach ($dir in $psesDepsPath,$psesCoreHostPath,$psesDeskHostPath,$psesVSCodeBinOutputPath)
{
New-Item -Force -Path $dir -ItemType Directory
}

# Copy Third Party Notices.txt to module folder
Copy-Item -Force -Path "$PSScriptRoot\Third Party Notices.txt" -Destination $PSScriptRoot\module\PowerShellEditorServices

# Lay out the PowerShellEditorServices module's binaries
# For each binplace destination
foreach ($destDir in $script:RequiredBuildAssets.Keys) {
# Create the destination dir
$null = New-Item -Force $destDir -Type Directory

# For each PSES subproject
foreach ($projectName in $script:RequiredBuildAssets[$destDir].Keys) {
# Get the project build dir path
$basePath = [System.IO.Path]::Combine($PSScriptRoot, 'src', $projectName, 'bin', $Configuration, $script:TargetPlatform)

# For each asset in the subproject
foreach ($bin in $script:RequiredBuildAssets[$destDir][$projectName]) {
# Get the asset path
$binPath = Join-Path $basePath $bin

# Binplace the asset
Copy-Item -Force -Verbose $binPath $destDir
}
Copy-Item -Force -Path "$PSScriptRoot\Third Party Notices.txt" -Destination $psesOutputPath

# Copy UnixConsoleEcho native libraries
Copy-Item -Path "$script:PsesOutput/runtimes/osx-64/native/*" -Destination $psesDepsPath
Copy-Item -Path "$script:PsesOutput/runtimes/linux-64/native/*" -Destination $psesDepsPath

# Assemble PSES module

$includedDlls = [System.Collections.Generic.HashSet[string]]::new()
[void]$includedDlls.Add('System.Management.Automation.dll')

# PSES/bin/Common
foreach ($psesComponent in Get-ChildItem $script:PsesOutput)
{
if ($psesComponent.Name -eq 'System.Management.Automation.dll' -or
$psesComponent.Name -eq 'System.Runtime.InteropServices.RuntimeInformation.dll')
{
continue
}

if ($psesComponent.Extension)
{
[void]$includedDlls.Add($psesComponent.Name)
Copy-Item -Path $psesComponent.FullName -Destination $psesDepsPath
}
}

# PSES/bin/Core
foreach ($hostComponent in Get-ChildItem $script:HostCoreOutput)
{
if (-not $includedDlls.Contains($hostComponent.Name))
{
Copy-Item -Path $hostComponent.FullName -Destination $psesCoreHostPath
}
}

# Get and place the shim bins for net461
foreach ($binDestinationDir in $script:RequiredNugetBinaries.Keys) {
$binDestPath = Join-Path $script:ModuleBinPath $binDestinationDir
if (-not (Test-Path $binDestPath)) {
New-Item -Path $binDestPath -ItemType Directory
# PSES/bin/Desktop
if (-not $script:IsUnix)
{
foreach ($hostComponent in Get-ChildItem $script:HostDeskOutput)
{
if (-not $includedDlls.Contains($hostComponent.Name))
{
Copy-Item -Path $hostComponent.FullName -Destination $psesDeskHostPath
}
}
}

foreach ($packageDetails in $script:RequiredNugetBinaries[$binDestinationDir]) {
Restore-NugetAsmForRuntime -DestinationPath $binDestPath @packageDetails
# Assemble the PowerShellEditorServices.VSCode module

foreach ($vscodeComponent in Get-ChildItem $script:VSCodeOutput)
{
if (-not $includedDlls.Contains($vscodeComponent.Name))
{
Copy-Item -Path $vscodeComponent.FullName -Destination $psesVSCodeBinOutputPath
}
}
}
Expand Down
Loading