Find out why 9 known failing tests on PowerShell Core pass in AppVeyor environment

[bergmeister]

We recently added PowerShell Core CI on Windows and Ubuntu and the number of tests that I had to fix for that were surprisingly small.
Locally, I always have 9 failures for PowerShell Core (both OS) but not in AppVeyor and when running the tests in the hosted Linux agent of VSTS confirm that it is probably AppVeyor that is hiding something.
We should find out about this BEFORE we fix the tests. The tests seem to have always failed for PowerShell Core by rerunning them on an old commit from January this year, therefore this is just technical debt due to the non-existence of PowerShell Core CI before.
The tests in question are listed in #944:

P.S. Below is a snippet to run the tests in the PowerShell v2 task in VSTS:

Import-Module .\tools\appveyor.psm1; Invoke-AppveyorInstall
if ($IsCoreClr) {
  Invoke-AppveyorBuild -CheckoutPath $env:BUILD_SOURCESDIRECTORY -BuildConfiguration Release -BuildType 'NetStandard'
}
else {
  Invoke-AppveyorBuild -CheckoutPath $env:BUILD_SOURCESDIRECTORY -BuildConfiguration Release -BuildType 'FullCLR'
}

$modulePath = $env:PSModulePath.Split([System.IO.Path]::PathSeparator) | Where-Object { Test-Path $_} | Select-Object -First 1
Copy-Item "$env:BUILD_SOURCESDIRECTORY\out\PSScriptAnalyzer" "$modulePath\" -Recurse -Force
$testScripts = "$env:BUILD_SOURCESDIRECTORY\Tests\Engine","$env:BUILD_SOURCESDIRECTORY\Tests\Rules"
$testResults = Invoke-Pester -Script $testScripts -OutputFormat NUnitXml -OutputFile "TestResults.xml" -PassThru

[bergmeister]

@kalgiz If you are interested, this would be a good and important issue to look into. It would be good to find out why the tests pass in AppVeyor or find a way of making them fail. Only after we have clarity about what happens in AppVeyor, the tests should be fixed.

[bergmeister]

Note that I found that at least on Windows, there can be environments (OS or hardware) where all tests pass for some reason...

[bergmeister]

Issue is outdated