Skip to content

Generated files are not UTF-8 encoded #2077

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HugCoder opened this issue Sep 13, 2016 · 14 comments
Closed

Generated files are not UTF-8 encoded #2077

HugCoder opened this issue Sep 13, 2016 · 14 comments
Labels
needs: repro steps We cannot reproduce the issue with the information given

Comments

@HugCoder
Copy link

Please provide us with the following information:

  1. OS? Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)
    Win10

  2. Versions. Please run ng --version. If there's nothing outputted, please run
    in a Terminal: node --version and paste the result here:
    angular-cli: 1.0.0-beta.11-webpack.8
    node: 6.5.0
    os: win32 x64

  3. Repro steps. Was this an app that wasn't created using the CLI? What change did you
    do on your code? etc.
    Generate a component or a new project. The files do not support UTF-8 chars, like åäö (Swedish). When rendered in HTML they appear as symbols. Changing the file encoding to UTF-8 (with or without BOM) fixes this.

  4. The log given by the failure. Normally this include a stack trace and some
    more information.
    N/A

  5. Mention any other details that might be useful.
    See step 3.


    Thanks! We'll be in touch soon.

@HugCoder
Copy link
Author

HugCoder commented Sep 13, 2016

I did some more tests now and it turns out that even if I manually change the encoding on the .ts and .html files, it seems like it reverts back to ANSI encoding again when webpack compiles it all, at least the .ts files.
Searching on this issue made me find this:
"webpack-encoding-plugin" (https://github.com/quackes/webpack-encoding-plugin)
and the issue seems like it might be related to this:
webpack/webpack-dev-server#432

The problem with angular-cli though is that it seems like there are no webpack.config files to add our own plugins or settings to. Maybe there's a way through angular-cli.json and adding it in "addons" or "packages"? So currently I cannot test if the plugin solves this issue. It would probably be good to add it natively to angular-cli though, since utf-8 encoding should be the base for everything in a modern web project.

@filipesilva filipesilva added needs: investigation Requires some digging to determine if action is needed needs: more info Reporter must clarify the issue needs: repro steps We cannot reproduce the issue with the information given and removed needs: investigation Requires some digging to determine if action is needed needs: more info Reporter must clarify the issue labels Sep 21, 2016
@filipesilva
Copy link
Contributor

I tried to repro and got UTF-8 encoded files througout.
image

Also on Win10, using VSCode myself. It is maybe your editor?

@HugCoder
Copy link
Author

HugCoder commented Sep 21, 2016

Thanks for answering!

Hmm, Yes, Visual Studio 2015 seems to re-save everything in ANSI. I found a "solution" by installing the "ForceUTF8" VS Extension:
https://visualstudiogallery.msdn.microsoft.com/222add7a-54a6-4fdd-b6f1-5d95cc596b42
but it doesn't exactly feel like a good/solid solution since if this would suddenly stop working/get disabled, things get worse. This is because I also noticed while manually changing the encoding, and VS re-saving to ANSI, I got weird symbols in the editor as well, which could be a pain to keep restoring to the original chars. I guess it's VS to blame though, no idea why they cannot simply default to UTF-8 for everyone instead of ANSI.
I guess I could develop the Angular part of the project in VS Code, while using VS2015 for the server code only. VS Code seems more compatible with the Angular CLI style of taking in new generated files automatically anyway. VS2015 requires you to add them to the project manually all the time.

@filipesilva
Copy link
Contributor

Sorry I can't help you more than this, I don't think there's much we can do on the CLI side :/

@keivankashani
Copy link

I have the same problem in CLI when I transfer Dist files to server and use IIS all persian fonts have bad encoding , and They are not UTF8
What Can I Do ???

@nomykk
Copy link

nomykk commented Mar 5, 2018

Having the same encoding problem with ng build --prod, fonts encoding is wrong, any solution for this?

@nomykk
Copy link

nomykk commented Mar 5, 2018

For Spring and Spring Boot Apps, I found the solution here:
#7053

including these properties resolved the encoding issue for me.

spring.messages.encoding=UTF-8
server.tomcat.uri-encoding=UTF-8
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true

@keivankashani
Copy link

Your answer is for Tomcat Web Server. Do you know a solution for IIS ?

@nomykk
Copy link

nomykk commented Mar 5, 2018

I don't know any solution for IIS, sorry dear.

@Sadlowski
Copy link

Sadlowski commented Mar 5, 2018 via email

@chpoit
Copy link

chpoit commented Mar 7, 2018

We have this issue here with files created with the CLI and then imported into visual studio (2015 and 2017). After saving in VS, when looking at the encoding the files claim they are utf-8, however they are in reality encoded in ISO-8859-1.
The issue seems to arrise from the fact that VS expects UTF-8 with BOM, and not utf-8, then it finds that the file is not correctly encoded and saves it as ISO-8859-1.

@kostia24
Copy link

For Visual Studio 2017, I solved this by turning off Auto-detect UTF-8 encoding without signature and Follow project coding conventions.

Got to: Tools | Options | Text Editor | General: then uncheck Auto-detect UTF-8 encoding without signature and Follow project coding conventions.

@progmars
Copy link

progmars commented Jul 18, 2019

Got to: Tools | Options | Text Editor | General: then uncheck Auto-detect UTF-8 encoding

This will make Visual Studio incorrectly display non-ASCII characters in files that were saved without BOM. I usually want to leave "Auto-detect UTF-8 encoding" enabled, otherwise when I open a file with non-ASCII multibyte characters in Visual Studio and save it, VS decides its ASCII and corrupts the non-ASCII multibyte characters.

So, in general this solution does not work. I don't know a clean way to force Visual Studio to automatically save non-BOM as UTF8 if you happen to be the first person to add non-ASCII characters to the file. I just tested - in such scenario, VS saves non-ASCII chars as a single byte. You have to manually resave the file picking UTF8 without BOM in VS to make it work correctly.

@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Sep 9, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs: repro steps We cannot reproduce the issue with the information given
Projects
None yet
Development

No branches or pull requests

8 participants