New syntax coloring doesn't work correctly (preview)

[PrzemyslawKlys]

Issue Description

I am experiencing a problem with... coloring syntax.

I know Monokai is not yet working properly (I saw a PR fix for this) but it was showing more or less consistently. But now I noticed that something is weird

Describe 'Select-Properties' {
    It 'Select-Properties - Testing Array of PSCustomObjects' {

        $Object1 = [PSCustomobject] @{
            Name1 = '1'
            Name2 = '3'
            Name3 = '5'
        }
        $Object2 = [PSCustomobject] @{
            Name4 = '2'
            Name5 = '6'
            Name6 = '7'
        }

        Select-Properties -Objects $Object1, $Object2 -AllProperties | Should -Be Name1, Name2, Name3, Name4, Name5, Name6
        $Object1, $Object2 | Select-Properties -AllProperties | Should -Be Name1, Name2, Name3, Name4, Name5, Name6
        $Object1, $Object2 | Select-Properties -AllProperties -ExcludeProperty Name6 -Property Name3 | Should -Be Name3

    }
    It 'Select-Properties - Testing Array of OrderedDictionary' {
        $Object3 = [Ordered] @{
            Name1 = '1'
            Name2 = '3'
            Name3 = '5'
        }
        $Object4 = [Ordered] @{
            Name4 = '2'
            Name5 = '6'
            Name6 = '7'
        }

        Select-Properties -Objects $Object3, $Object4 -AllProperties | Should -Be Name1, Name2, Name3, Name4, Name5, Name6
        $Object3, $Object4 | Select-Properties -AllProperties | Should -Be Name1, Name2, Name3, Name4, Name5, Name6
    }
}

I can continue adding new lines:

And it seems like it stopped working. Restarting PS session fixes this issue.

Attached Logs

1596304553-9b1c68c9-c15f-43c6-9739-e1b537cd361b1596304549455.zip

Environment Information

Visual Studio Code

Name	Version
Operating System	Windows_NT x64 10.0.19041
VSCode	1.47.3
PowerShell Extension Version	2020.7.0

PowerShell Information

Name	Value
PSVersion	5.1.19041.1
PSEdition	Desktop
PSCompatibleVersions	1.0 2.0 3.0 4.0 5.0 5.1.19041.1
BuildVersion	10.0.19041.1
CLRVersion	4.0.30319.42000
WSManStackVersion	3.0
PSRemotingProtocolVersion	2.3
SerializationVersion	1.1.0.1

Visual Studio Code Extensions

Visual Studio Code Extensions(Click to Expand)

Extension	Author	Version
bracket-pair-colorizer-2	CoenraadS	0.2.0
code-settings-sync	Shan	3.4.3
github-linker	gimenete	0.2.3
gitlens	eamodio	10.2.2
markdown-all-in-one	yzhang	3.2.0
material-icon-theme	PKief	4.2.0
powershell-preview	ms-vscode	2020.7.0
project-manager	alefragnani	11.1.0
run-in-powershell	tobysmith568	1.0.1
vscode-markdownlint	DavidAnson	0.36.2
vscode-pull-request-github	GitHub	0.18.0
vscode-toggle-quotes	BriteSnow	0.3.3
vscode-wakatime	WakaTime	4.0.0
vscode-yaml	redhat	0.9.1
xml	DotJoshJohnson	2.5.1

[rjmholt]

One thing I spot in the logs seems to be an error in Pester here. Don't think it's related to the syntax highlighting though

[SydneyhSmith]

Thanks @PrzemyslawKlys if this happens again could you see if you are able to get feedback from the extension in any other way (like completions)?

[PrzemyslawKlys]

I'm pretty sure it's related to a known issues that I'm having quite often when everything stops working (completions etc). With this new approach is much easier to spot when there are issues with performance.

[PrzemyslawKlys]

In less than 30 minutes - I didn't even run the code, did like 3 edits, and looked away. When I came back adding anything to code would not highlight properly (check lines 22,23,24).

Here are the logs:
1596785024-7bca635e-a6e1-4236-bf08-6ae63cc3f7861596731164353.zip

We really need improvement in stability because more and more features are reliant on it. Maybe consider adding an option to use "old" highlighting and new highlighting so people can switch/choose?

I know you're hard at work with it, but as it stands I have to restart my session every 30 minutes or less if I want to have things running.

[TylerLeonhardt]

@PrzemyslawKlys when you sent those last logs, did you capture the logs while the extension was running or did you close vscode and then copy the logs?

The reason I ask is because it looks like the language server was asked to shut down and I wanna know if that was intentional or not.

Also, this was an odd thing I spotted in the log:

2020-08-07 09:23:51.101 +02:00 [VRB] Loaded 'pf50wdtn, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' from ''

It's also odd that I only see queuing of the Semantic Token request and not handling. Probably need to add some logging somewhere.

[PrzemyslawKlys]

The last one was "intentional" sort of. I went to Open Logs Folder and clicked Restart session by accident.

[PrzemyslawKlys]

1596894776-765f03f3-0486-43d7-9e74-5a2f57e7ef801596891995787.zip

This is like 2 minutes into session.

[PrzemyslawKlys]

Also - not sure if this matters for comparison purposes but it's showing weird coloring here and there. But maybe Extension doesn't touch it.

[PrzemyslawKlys]

It seems when this happens all other functionality is in place. As in completion works.

[TylerLeonhardt]

We've updated to the latest Semantic Highlighting changes so it's possible that this will be addressed in the next preview (probably next week)

[ili101]

I started using the following configurations

"editor.codeLens": false,
"powershell.scriptAnalysis.enable": false,
"editor.semanticHighlighting.enabled": false

Honestly I don't understand way bother adding new features to this extension, it just make it even slower and unstable so the overall experience worst.
If it was up to me I would go with full feature freeze and focus 100% on stability improvements.

[PrzemyslawKlys]

I believe that's what @rjmholt is doing right now, and a reason why we didn't see new releases for a while now. That they are working on making it more robust. I guess it's so much you can do in current implementation that they need to rewrite huge portions of it, in which case things take time. I feel your pain every day and I restart PowerShell instance every X minutes because something stops working.

Right now intellisense stops working, highlighting stops working, types completion stops working but there's not much I can do.

[ghost]

This issue was closed automatically as repro info was indicated as needed, but there has been no activity in over a week. Please feel free to reopen with any available information!

[andyleejordan]

I know the bot closed this automatically, but I wanted to add that we intentionally disabled semantic highlighting by default a while back in #3377 because it was too buggy. Eventually when the rest of the backlog is cleared out I plan to work on stabilizing that feature and re-releasing it with proper test coverage.