ENH: Allow poorly formatted stata files to be read #25967
Conversation
Codecov Report
@@ Coverage Diff @@
## master #25967 +/- ##
==========================================
- Coverage 91.84% 91.83% -0.01%
==========================================
Files 175 175
Lines 52550 52550
==========================================
- Hits 48266 48261 -5
- Misses 4284 4289 +5
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## master #25967 +/- ##
==========================================
- Coverage 91.84% 91.84% -0.01%
==========================================
Files 175 175
Lines 52550 52550
==========================================
- Hits 48266 48262 -4
- Misses 4284 4288 +4
Continue to review full report at Codecov.
|
Add a fall back decode path that allows improperly formatted Stata files written in 118 format but using latin-1 encoded strings to be read closes pandas-dev#25960
bashtage
force-pushed
the
latin-1-fallback
branch
2 times, most recently
from
3f711fc to
ddc806f
Compare
| has been incorrectly encoded by Stata or some other software. You should verify | ||
| the string values returned are correct.""" | ||
| with pytest.warns(UnicodeWarning, match=msg): | ||
| encoded = read_stata(self.dta_encoding_118) |
There was a problem hiding this comment.
Use tm.assert_produces_warning. Our implementation also checks stacklevel.
There was a problem hiding this comment.
actually this already checks
Refactor decode and null terminate to use file encoding
bashtage
force-pushed
the
latin-1-fallback
branch
from
ddc806f to
27a173d
Compare
|
lgtm. @bashtage if you can get the check_stacklevel to work would be great. |
|
I switched to |
|
actually this does check, thanks @bashtage |
Add a fall back decode path that allows improperly formatted Stata
files written in 118 format but using latin-1 encoded strings to be
read
closes #25960
git diff upstream/master -u -- "*.py" | flake8 --diff