FIX: Workaround for integer hashing #8982
Conversation
|
@yarikoptic This might fix it by forcing int. I'm not completely sure if this isn't a NumPy bug (or at least idiosyncracy) on 32 bit platforms with hashing or casting this value to native Python types. |
|
iirc this is a numpyism that dtype hashes are not the same |
jreback
added
IO Stata
|
@yarikoptic can you try this out? |
|
well i'll merge and see what happens @bashtage thanks |
| @@ -643,6 +643,7 @@ class StataMissingValue(StringMixin): | |||
|
|
|||
| def __init__(self, value): | |||
| self._value = value | |||
| value = int(value) if value < 2147483648 else value | |||
There was a problem hiding this comment.
maybe put this as 2147......L e.g. long?
can you test somehow?
|
There is some cleaning to do in the stata code, so I'll think about how to make this more robust in the future - perhaps making all integer values always longs when creating the dictionary, and then doing the same when doing a look up might be the more stable across bitnesses. |
|
ok, if this fixes @yarikoptic then ok to put in |
bashtage
force-pushed
the
stata-large-int-missing-value
branch
from
04d4308 to
b7fc3c6
Compare
|
Have made a few minor changes to ensure that the hashes in the missing value dictionary should always have the correct type for any bitness. |
bashtage
force-pushed
the
stata-large-int-missing-value
branch
2 times, most recently
from
bd85b73 to
802917c
Compare
Force conversion to integer for missing values when they must be integer to avoid hash errors on 32 bit platforms. closes pandas-dev#8968
FIX: Workaround for integer hashing
Force conversion to integer for missing values when
they must be integer to avoid hash errors on 32 bit
platforms.
closes #8968