BUG: Missing value code not recognised for Stata format version 105 a… #59325
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mroeschke
reviewed
Jul 26, 2024
pandas/io/stata.py
Outdated
| # recode instances of this to the currently used value | ||
| if self._format_version <= 105 and fmt == "d": | ||
| data.iloc[:, i] = data.iloc[:, i].replace( | ||
| float.fromhex("0x1.0p333"), self.MISSING_VALUES["d"] |
There was a problem hiding this comment.
Nit: Could you define float.fromhex("0x1.0p333") outside the loop?
There was a problem hiding this comment.
Yes, that should be straightforward, I'll make that change.
mroeschke
approved these changes
Jul 26, 2024
|
Thanks @cmjcharlton |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…nd earlier
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.This is an initial attempt at fixing this bug, although a more efficient implementation probably exists.
As Pandas does not write in any of these version formats I have simply replaced instances of the old missing code with the current one for these versions before carrying on as before.
I believe that the range of valid values for float/double was also changed when the missing code was changed, however as this was widened any existing files written in these formats should still be within the current range and therefore will be read correctly. If an older format version is created with values outside the documented range then the current version of Stata (18) reads them as valid values, rather than converting them to missing, so I think the behaviour here is consistent with that.