Codec utf-16 aliases do not work in read_csv with c engine

[Anaphory]

Code Sample, a copy-pastable example if possible

import pandas

path = "test.csv"
pandas.DataFrame({"A": [0,1], "B": [2,3]}).to_csv(
    path, encoding="utf-16")

for encoding in ["utf-16","utf_16","UTF_16","UTF-16"]:
    try:
        pandas.io.parsers.read_csv(
            path,
            engine='c',
            encoding=encoding)
        print(encoding, "succeeded in c-pandas")
    except UnicodeDecodeError:
        print(encoding, "failed in c-pandas")

    try:
        pandas.io.parsers.read_csv(
            path,
            engine='python',
            encoding=encoding)
        print(encoding, "succeeded in pandas")
    except UnicodeDecodeError:
        print(encoding, "failed in pandas")

    try:
        with open(path, encoding=encoding) as file:
            file.read()
        print(encoding, "succeeded in open")
    except UnicodeDecodeError:
        print(encoding, "failed in open")

Expected Output

utf-16 succeeded in c-pandas
utf-16 succeeded in pandas
utf-16 succeeded in open
utf_16 succeeded in c-pandas
utf_16 succeeded in pandas
utf_16 succeeded in open
UTF_16 succeeded in c-pandas
UTF_16 succeeded in pandas
UTF_16 succeeded in open
UTF-16 succeeded in c-pandas
UTF-16 succeeded in pandas
UTF-16 succeeded in open

Actual Output

utf-16 succeeded in c-pandas
utf-16 succeeded in pandas
utf-16 succeeded in open
utf_16 failed in c-pandas
utf_16 succeeded in pandas
utf_16 succeeded in open
UTF_16 failed in c-pandas
UTF_16 succeeded in pandas
UTF_16 succeeded in open
UTF-16 failed in c-pandas
UTF-16 succeeded in pandas
UTF-16 succeeded in open

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.6.2-1-ARCH
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.1
setuptools: 23.0.0
Cython: None
numpy: 1.11.0
scipy: 0.17.1
statsmodels: None
xarray: None
IPython: 4.2.0
sphinx: None
patsy: None
dateutil: 2.4.2
pytz: 2016.4
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.4.1
html5lib: None
httplib2: 0.9.2
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None

[Anaphory]

Without error handling to run through all test cases, the traceback is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/gereon/python/bugreport/test.py", line 11, in <module>
    engine='c',
  File "/usr/lib/python3.5/site-packages/pandas/io/parsers.py", line 562, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/lib/python3.5/site-packages/pandas/io/parsers.py", line 325, in _read
    return parser.read()
  File "/usr/lib/python3.5/site-packages/pandas/io/parsers.py", line 815, in read
    ret = self._engine.read(nrows)
  File "/usr/lib/python3.5/site-packages/pandas/io/parsers.py", line 1314, in read
    data = self._reader.read(nrows)
  File "pandas/parser.pyx", line 805, in pandas.parser.TextReader.read (pandas/parser.c:8748)
  File "pandas/parser.pyx", line 827, in pandas.parser.TextReader._read_low_memory (pandas/parser.c:9003)
  File "pandas/parser.pyx", line 904, in pandas.parser.TextReader._read_rows (pandas/parser.c:10022)
  File "pandas/parser.pyx", line 1011, in pandas.parser.TextReader._convert_column_data (pandas/parser.c:11397)
  File "pandas/parser.pyx", line 1071, in pandas.parser.TextReader._convert_tokens (pandas/parser.c:12302)
  File "pandas/parser.pyx", line 1157, in pandas.parser.TextReader._convert_with_dtype (pandas/parser.c:13740)
  File "pandas/parser.pyx", line 1173, in pandas.parser.TextReader._string_convert (pandas/parser.c:13950)
  File "pandas/parser.pyx", line 1460, in pandas.parser._string_box_decode (pandas/parser.c:19767)
UnicodeDecodeError: 'utf-16-le' codec can't decode byte 0x41 in position 0: truncated data

[Anaphory]

In case it is relevant:

7.2.3. Standard Encodings

[…]Notice that spelling alternatives that only differ in case or use a hyphen instead of an underscore are also valid aliases; therefore, e.g. 'utf-8' is a valid alias for the 'utf_8' codec.

CPython implementation detail: Some common encodings can bypass the codecs lookup machinery […]: […] utf-16 […]

[Anaphory]

This also holds if the writing encoding is taken to be utf_16:

>>> pandas.DataFrame({"A": [0,1], "B": [2,3]}).to_csv(
        path, encoding="utf_16")
[…]
utf_16 failed in c-pandas
utf_16 succeeded in pandas
utf_16 succeeded in open

[jreback]

yeah this has a string definition of utf-16. Should be easy to make more general.

want to take a crack at it?