Dissociation between master branch and Microsoft.PowerShell.EditorServices nuget package

[mhintzke]

Hey PowerShell Team!

This is not so much an "issue", but rather a question.

My team and I are wanting to build a PSES backed ASP .NET Core application that can perform a number of supplementory operations for our Monaco Editor implementation in our application. These features include things like:

• Script Analysis
• Script Formatting
• AST Generation, Querying, and Building
• (Probably more in the future)

The first 2 can be achieved using the PSScriptAnalyzer which we have gotten working both a Powershell Azure Function and as a PoC in a ASP .NET Core Console Application by building our own DLLs and interfacing with the ScriptAnalyzer directly.

The 3rd feature is something we think we can achieve via the System.Management.Automation.Language library which offers the Parser.ParserInput(...) method that constructs us an AST of an arbtrary script. We would also need a way to mutate the AST and get back a script as a string. I assume all of this can be done via this library.

Since we want to support all of these functionalities, it seemed best to build a single service for all things "Powershell" that utilized PSES. I noticed that PSES makes use of the PSScriptAnalyzer by generating a RunespacePool and taking care of a lot of the boilerplate logic needed to orchestrate these operations at scale. With that said, I tried standing up a .NET Core API using the latest Microsoft.PowerShell.EditorServices library, but the library seems to be outdated from what I am seeing in master today. Do you intend on releasing the latest to that NuGet package anytime soon or is it better to compile our own DLLs?

As a whole, it seems like using PSES over using PSScriptAnalyzer directly is the smarter move for the longevity and featureset of our application. Any suggestions on what we should target as a dependency going forward are welcomed.

Our application is still in Alpha so we don't really have to worry about the code being production ready. We just love what you guys are doing over there and want to take advantage of some of these cool technologies to give our user's an ideal PowerShell IDE experience.

Thanks!

[rjmholt]

Hi @mhintzke13, thanks for opening an issue.

Do you intend on releasing the latest to that NuGet package anytime soon or is it better to compile our own DLLs?

PSES is currently undergoing a fair amount of change to meet short/medium-term goals that lie on the LSP/DAP API front, as well as in the Integrated Terminal. As a result, we don't consider the .NET APIs stable currently and while we do eventually intend to publish/update the NuGet package, it's not currently a priority. Also, the .NET APIs exist either because of the old architecture (which is no longer in master) or to be exposed in the Integrated Console; they're not intended for rehosting, and probably wouldn't be enough for that.

However, PSES is designed to be a standalone, PowerShell-integrated LSP server, and its primary stable interface is language server protocol. My recommendation would be to stand up a PSES process around a currently stable or preview release, available here and integrate with it over LSP. There's a fairly high-level description of how to get started with that in the README.

it seemed best to build a single service for all things "Powershell" that utilized PSES

So on that front, PSES is designed to already be a service like this, at least as far as editor experience is concerned. Certainly that's how it's embedded by other editors.

The 3rd feature is something we think we can achieve via the System.Management.Automation.Language library which offers the Parser.ParserInput(...) method that constructs us an AST of an arbtrary script. We would also need a way to mutate the AST and get back a script as a string. I assume all of this can be done via this library.

To this point, it's worth noting that PowerShell AST objects are immutable. There are some solutions here, but it depends on the goal. One way is to have an AST builder that extends AstVisitor2 or ICustomAstVisitor2 and builds a new AST from the old one as it goes. But it just depends.

[mhintzke]

Hey @rjmholt, thanks for the quick reply!

So we actually already are using PSES on one of our services, but that service is distributed 1-1 with our users. We do this primarily for Intellisense within our browser IDE which allows each user to have a completely different experience due to the fact that they are all running different instances of PS and therefore have different modules loaded and available for their IDE.

I was able to get a simple ScriptAnalyzer() solution working in a .NET Core console app with the below code:

var script = "function";
var analyzer = new ScriptAnalyzer();

using (var runspace = RunspaceFactory.CreateRunspace(InitialSessionState.Create()))
{
    var streamOutput = new StringOutputWriter();
    runspace.Open();

    analyzer.Initialize(runspace, streamOutput, customizedRulePath: null, includeRuleNames: null, excludeRuleNames: null, severity: null, includeDefaultRules: true, suppressedOnly: false, profile: null);

    var results = analyzer.AnalyzeScriptDefinition(script);
}

So I guess we could either:

1. Try to do all of our analysis and parsing on the same PSES services that we already have it running on. This is a docker image running linux withi PS Core and a NodeJS app spawning node-pty psuedo-terminals that users can connect to via a Websocket, so any LSP communication would have to go through this Node app.

2. Find a way to have a single service running ASP .NET Core that could also spin up a PowerShell process on-demand, run the single analysis that needs to be done using Named Pipes and LSP, and return results to the client in a more stateless way. (Not sure if this would even be possible)

3. Try to do basically what I did above and not use LSP at all since I'm not using the PSES library by going straight to the ScriptAnalyzer itself. This feels more "hacky" and might yield side effects by abusing the Runespace like this.

It sounds like although 1 above would take the most effort, that is your preferred approach, correct? I'm curious if you th ink 2 is possible, where we basically do the same thing as 1, but have a single process running rather than 1-1 to the user. And it would be abstracted away by a .NET Core API.

[rjmholt]

So we actually already are using PSES on one of our services, but that service is distributed 1-1 with our users

Because PSES is designed as an LSP server, it's not designed to be a multi-client server. In fact, the LSP spec says the server process must shutdown before the client does.

Of the possibilities you list, (1) is the closest to the intended scenario of PSES, where a user connects to a single PowerShell session and PSES runs that session and provides LSP interfaces to it over another port.

(2) is possible, but PSES is very heavyweight for this, since you would need to create a new one per analysis request. Even the PowerShell startup here would be prohibitive. Pursuing this I think you would be better off hosting PowerShell with the PowerShell SDK and providing your own analysis using AST visitors.

Along these lines, work is being done to make (3) easier, so that PSSA is easier to use from C#.

But the important part is that things like the initialize and shutdown requests being part of the LSP means that LSP servers are inherently client-attached and stateful. LSP is not intended for many-to-one scenarios (and nor is PSES). Even PSSA is not well-suited for servicing multiple PowerShells, and it does run PowerShell commands in some cases. For lightweight analysis in many-to-one scenarios, my suggestion would be to host PowerShell using the PowerShell SDK and run AST visitors for analysis. It might be quite possible to reuse the rules from PSScriptAnalyzer directly without PSSA too (they're built as their own assembly).

[mhintzke]

Ok great, thanks for all of the insight @rjmholt !

Looks like I will spend the next few days reading up on LSP :)

We can close this issue if you want and if I have more follow-up questions, I will open up another.

Thanks!