Bug in read_csv and read_excel not applying dtype to second col with dup cols

[phofl]

• closes BUG: in read_excel for mangled columns only the original/first column dtype is correct, col.N is not parsed correctly #35211
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

[jreback]

thanks @phofl

[jreback]

would be nice to unify this code between here and the python parser (followon)

[phofl]

Yes definitely, but will have to refactor the PythonParser quite a bit and split into 2 classes to be able to inherit from TextReader respectively a generic cython class where TextReader and something like PythonTextReader can inherit from.

I am planning to do this in the (probably medium-term) future

[jreback]

sounds great!

feels free to open an issue for tracking

[phofl]

Though about using #39345 for this

[fiendish]

FYI this breaks parsing with a non-dict dtype.

[phofl]

Could you provide an example?

[fiendish]

Documentation for read_csv says dtype may be "a type name or dict".
pandas.read_csv("test.csv", engine="python", dtype="str") and pandas.read_csv("test.csv", engine="python", dtype=str) now error because this code's new calls to .get assume a dict.

[phofl]

Your test file has duplicate columns too? Will look into this later

[fiendish]

Your test file has duplicate columns too? Will look into this later

Indeed it does, good eye. That's the unfortunate reality when non-programmer people make documents.

Here's my test file:
test.csv

But of course

A,B,B
1,1,1

shows it too.

[fiendish]

I should have filed an issue first. I've done that now: #42022