File become unreadable once it is staged in git

[Stephanevg]

System Details

Copy / paste the following commands into the PowerShell Integrated Console, and paste the output here:

PS D:\Repository\Scripts\SCS.Logging\Tests> code -v
code : The term 'code' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ code -v
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (code:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS D:\Repository\Scripts\SCS.Logging\Tests> $pseditor.EditorServicesVersion

Major  Minor  Build  Revision
-----  -----  -----  --------
1      7      0      0


PS D:\Repository\Scripts\SCS.Logging\Tests> code --list-extensions --show-versions
code : The term 'code' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ code --list-extensions --show-versions
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (code:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS D:\Repository\Scripts\SCS.Logging\Tests> code -v
code : The term 'code' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ code -v
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (code:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS D:\Repository\Scripts\SCS.Logging\Tests> $pseditor.EditorServicesVersion

Major  Minor  Build  Revision
-----  -----  -----  --------
1      7      0      0


PS D:\Repository\Scripts\SCS.Logging\Tests> code --list-extensions --show-versions
code : The term 'code' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ code --list-extensions --show-versions
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (code:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS D:\Repository\Scripts\SCS.Logging\Tests> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.2189
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.2189
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Issue Description

I am experiencing a problem with VSCode, when I stage a document.

Then, wenn I add stage the document: and go look in the 'staged changes'

[rjmholt]

This looks like an encoding problem -- something is encoding the file to UTF-16 I suspect?

This issue might be slightly helpful: #1308.

But it's unlikely that PowerShell or the PowerShell VSCode extension is doing this. Much more likely is that VSCode is configured to some encoding, or git has a configuration encoded, or that a collaborator is changing the encoding.

VSCode allows you to specify an encoding, so it might be worth finding out what your current setting is (if any). It defaults to UTF-8, but if it finds a BOM for another encoding it will switch over.

Do you know what your setting for "files.encoding" in VSCode is?

But the real puzzling part here is the fact that VSCode can read the file when it's written, then forgets when it's staged. Do you have any git hooks set?

The best way to work out what's going on would be to work out what the encoding of the file is before you stage it and after. This might be useful: https://stackoverflow.com/questions/3710374/get-encoding-of-a-file-in-windows.

[rjmholt]

Ah here's a question - what git diff tool are you using?

I saw that the code command you entered in the console didn't work. The intent of that is that it will provide a list of the VSCode extensions installed. Does code not register in your Integrated Console? Can you try code --list-extensions --show-versions in the Integrated Console once more and if that doesn't work, in PowerShell?

[Stephanevg]

I tried to check the encoding before and after and the staging:
O both cases, I have this: Unicode UTF-16 Little-Endian (I used this function to get the value https://gist.github.com/jpoehls/2406504 )

I tried your command, it seems I have an issue with the 'code' command:

PS D:\Repository\Scripts\SCS.Logging\Tests> code --list-extensions --show-versions
code : The term 'code' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ code --list-extensions --show-versions
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (code:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Otherwise, in the GUI, these are the extensions I have:

EDIT: "files.encoding": "utf8"

[rjmholt]

Maybe try code.cmd? code is the command line invocation for VSCode. If you just run code.cmd it will start a new VSCode session, for example. Having VSCode installed should mean it works.

[Stephanevg]

"git noob quesiton coming a head"I was rechecking the Git installation I did, could did perhaps have an influence?

[rjmholt]

That's possible I guess, but unless you're pushing and pulling the file and another coworker is changing it, that shouldn't be the problem. I've seen examples where other people have editors that change the encoding, but git itself doesn't care about encodings (it only sees bytes) and tends not to touch the encoded file. Windows git has that line endings feature, but that wouldn't be as extensive as this issue.

I notice in your screenshots that the first one has file encoding set to UTF-16LE and the second one doesn't list it at all. Maybe try setting `"files.encoding: utf16le" (or whatever the suggestion is that looks like it's UTF-16LE)?

[Stephanevg]

Yes, that did the TricK (Adding the "files.encoding": "utf16le"), for that first file (the psd1).

(I also fixed the code error thing. I simply added the path C:\Program Files\Microsoft VS Code\ to the $Path environment variable. (Let me know if you still need these command results).

I am currently working alone on this project. I double checkd, nobody actually changed anything besides of me.

I started this file with ISE, installed VScode afterwards, and then continue with VSCode. I thought for a moment that ISE would enforce utf16le.
So I created a script in ISE, and saved it.
Them I opened it VsCode, it is utf8.
When I staged it.

I get a nice little Chinese class:

(I checked using the Get-FilenEcoding - it says UTF8)

[rjmholt]

Ah excellent! No don't need the output of the command.

You should decide what encoding you want to use based on your context, convert the files in the repo to that encoding, and configure both VSCode and the ISE to honour it. Just don't want my advice to break your repo. Ideally everything should be in one encoding.

[Stephanevg]

@rjmholt You were to quick closing the ticket. I didn't have enough time to edit it correctly. I reproduced the issue on another file. Which was UTF8 at the begining.

[rjmholt]

Oh sorry. Well this isn't actually an issue in the PowerShell VSCode extension though. We don't control VSCode's file encoding handling.

[rjmholt]

Can you provide a screenshot of the UTF-8 file problem?

[Stephanevg]

Yeah, actually, setting that setting from files.encoding": "utf16le" to files.encoding": "utf8" actually corrects it, and shows the file as expected, so we can actually close the ticket :)

I guess I have to find out my file gets converted into a UTF16le in the first place.
Thanks for your help @rjmholt

[rjmholt]

My suggestion would be to run a utility over the whole repo and change it to the encoding of your choice and then see if you still have issues. Some editors and things get confused by moving between encodings. 😄