diff --git a/2-Draft-Accepted/RFC0036-AdditionalTelemetry.md b/2-Draft-Accepted/RFC0036-AdditionalTelemetry.md new file mode 100644 index 00000000..4affaa73 --- /dev/null +++ b/2-Draft-Accepted/RFC0036-AdditionalTelemetry.md @@ -0,0 +1,196 @@ +--- +RFC: RFC0036 +Author: Sydney Smith +Status: Draft-Accepted +SupercededBy: N/A +Version: 1.0 +Area: Telemetry +Comments Due: 3/29/19 +Plan to implement: Yes +--- + +# Additional Telemetry in PowerShell + +Add usage telemetry across PSEdition, Platform, Application, session type and command types. +Improve the opt-out mechanisms for PowerShell telemetry.Provide community access to aggregated +statistics. + +## Motivation + +To make the best possible investments in PowerShell we need answers backed by data to the following questions: + +- Is PowerShell Core usage growing (in terms of number of starts)? +- Is the PowerShell Core user-base growing? +- How is PowerShell being used? What is the usage distribution across command types and session type? +- What are impediments to PowerShell Core usage growth? +- What are issues that customers are hitting in PowerShell Core? +- What versions of PowerShell tools and services should Microsoft continue to support? +- What PowerShell integration scenarios are people using? How many people are using PowerShell + Editor services? +- Which experimental features are being used and tested? Which experimental features should we invest in? +- How can we optimize the engine size and efficiency of PowerShell for cloud scenarios? + +To ensure we are getting an accurate picture of how everyone uses PowerShell, not just those most +vocal/involved in the community, we need to make improvements in our telemetry. +PowerShell usage telemetry will allow us to better prioritize testing, support, and investments. +It should be straightforward and easy to opt-out of PowerShell telemetry (presently the process is +not intuitive nor well documented). +To better share our plans and successes with the community we want more complete telemetry on our +[Public PowerBi report](https://msit.powerbi.com/view?r=eyJrIjoiYTYyN2U3ODgtMjBlMi00MGM1LWI0ZjctMmQ3MzE2ZDNkMzIyIiwidCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsImMiOjV9&pageName=ReportSection5). +Simply put, we want to make PowerShell better and believe this can be achieved by better +understanding how PowerShell is being used. + +### Non-Goals + +The goal of this is **not** to collect any personal data of PowerShell users or infringe on +users' privacy. +Respecting the privacy of our users has been the principal factor in determining which metrics we +plan to collect. +The goal of this is not to collect patterns of individual users but to look at aggregate usage trends. + +## Specification + +Add telemetry to track the following metrics: + +- Count of unique users (PowerShell) +- Count of execution types (ex. cmdlets, native binaries/applications) (PowerShell) +- Count of enabled PSEngine experimental features (only those created by the PowerShell Team) (PowerShell) +- Count of session types (ex. hosted vs not hosted sessions) (PowerShell) +- Hosting application of the of the PowerShell session (ex. VSCode, ISE, AZDevOps) (PowerShell) +- Version of PowerShell, PowerShellGet, PackageManagement, Nuget and platform + used to install from PSGallery (PowerShellGet) + +### Other features + +- Public PowerBi DashBoard to make the aggregated statistics available to the community. + Find a mocked-up PowerBi DashBoard with randomly generated data at the end of the document. + It includes the following reports: + - Unique user trends + - Types of user —based on types of executions taking place + - VSCode usage trends + - Application trends + - Hosted Scenario Trends + - PowerShellGet installation trends +- A cmdlet for providing feedback (Send-PSFeedback): + - This cmdlet would take in a string array and send it back to the PowerShell team + - The cmdlet will also send Platform data (OS/version) and Shell data ($PSVersionTable) + - The cmdlet will have an -OpenGitHubIssue switch that will open a browser to a new issue template + in the PowerShell/PowerShell repository + - The user will have the option to include the the last error using an -IncludeError switch + - If the command is not successful (ex. there is no Web Connection) the cmdlet will return a + warning and alternative instructions for support through GitHub + - If no location switch is used the cmdlet will default send the feedback to the PowerShell + User Voice + - Based on usage of this cmdlet there would potential for additional parameters +- Disabling, and re-enabling telemetry will be available through a new cmdlet (Set-PSTelemetry): + - The cmdlet will stop all collection of telemetry for the remainder of that session and stop + telemetry from being collected on subsequent starts + - The user can request their telemetry is deleted by using a Remove-PSTelemetry cmdlet + - In order to have their telemetry deleted the user will be required to provide their User GUID + - If the cmdlet is unable to send the information the cmdlet will return a warning and alternative + instructions for support through GitHub. It will be specified that GUIDs are unique identifiers + and should not be shared publicly on GitHub. +- There will be two options for the telemetry environment variable which can be set manually + or using the Set-PSTelemetry cmdlet + - $True indicates all telemetry will be collected + - $False indicates no telemetry will be collected +- The existing disabling [mechanisms for PowerShell telemetry](https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-61?view=powershell-6#telemetry-can-only-be-disabled-with-an-environment-variable) will remain including the ability to + set the telemetry environment variable before ever needing to launch PowerShell. + Setting the telemetry environment variable to $False will block telemetry from being sent on + every session launch. + +### Future work + +A public API for module authors and PowerShell hosted application owners to use the same +infrastructure to collect telemetry for their module or application. +Making the API public is a phase two goal of this project. +We will make the telemetry publicly available once it is internally validated. +The intent of this API is to provide a simple consistent way for PowerShell authors to collect their own telemetry. +If external PowerShell authors want to use the API they will need to provide their own storage for the data. +Validating compliance/privacy will be up to the user of the API and not something PowerShell team/Microsoft will cover. + +## Design + +### PowerShell Changes + +- Count of unique users will be based on a User GUID generated at PowerShell start time and stored + in the user's configuration directory +- On first start up notification that telemetry is enabled will display in the console header +- The Application ID will be used to disambiguate PowerShell Hosted applications. It will be a + standardized method for applications to identify themselves in our telemetry. It will be up to + Application owners to tag their usage for inclusion in our counts. +- The count execution type will be collected throughout the PowerShell session +- The type of PowerShell session (hosted vs non-hosted) will be collected at PowerShell + engine start time +- The count of enabled PSEngine experiemental features will be collected at PowerShell + engine start time. The count will be based on the experimental feature list in the + powershell.config.json file under $PSHOME, and will only count features created by the PowerShell Team +- An example of what would be collected on PowerShell Start up: + - User GUID: c58030d3-1a91-4086-9cb1-5bddd342056d + - The ApplicationID: VSCode + - The Platform being used: Windows 10 Pro + - The Version of PowerShell: 6.2.0-preview.4 + - The type of PS session: Hosted + - Experimental Features enabled: "A-Experimental-Feature-name", "Another-Experimental-Feature-Name" +- An example of what might be collected during a PowerShell Session: + - Count of cmdlets and functions: 10 + - Count of native binaries/applications: 5 +- Currently telemetry is collected at the console startup but we would move collection as close to + the PowerShell engine start time as possible in order to capture hosted PowerShell scenarios in + the telemetry +- Telemetry will be collected through Azure Application Insights and will be stored using Azure + Storage Tables + +### PowerShellGet Changes + +Create a header when packages are installed from the PowerShell Gallery, +to collect/send the following information: + +- Version of PowerShellGet, PackageManagement, and NugetProvider +- Version and Edition of PowerShell +- Identifier for Operating System + +- An example of what would be collected at Package Installation: + - The Version of PowerShellGet: 2.0.3 + - The Version of PackageManagement: 1.2.4 + - The NugetProvider: 3.0.0.1 + - The version of PowerShell: Core 6.2 + - The platform being used: macOS 10.3 + +## Alternate Proposals and Considerations + +Privacy concerns including GDPR regulations are a major consideration of this RFC. +We underwent a privacy review before drafting this RFC to ensure that we are respecting +the privacy of all users. The User GUID will be stored in $PSHome so that the user +can provide it and have their data deleted at anytime. +Performance impact is also a major consideration of these changes, and the metrics chosen are +designed to have as nominal an impact on end user experience as possible. + +We considered collecting the module/function/cmdlet usage from only modules that are downloaded by the PowerShell Gallery. +The reason we did not chose to propose this is because of the large cost (in terms of performance, data collection, and data management) +compared with the relatively smaller benefit of actionable questions we could answer with the information. + +We also considered automatically collecting a count of fully qualified error ids. The reason we decided against this was because +of the potential performance impact and high cost. + +[Microsoft Privacy Policy](https://privacy.microsoft.com/en-US/privacystatement) + +## PowerShell Committee Decision + +### Voting Results + +Joey Aiello: Accept + +Steve Lee: Accept + +Dongbo Wang: Accept + +Jim Truher: Accept + +### Majority Decision + +Committee agrees that this RFC addresses the need for additional telemetry to help guide future decisions. + +### Minority Decision + +N/A