-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
BUG: SIGBUS in test_float_byteswap test on arm #54391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
There's definitely unaligned access there (using the offsets), but the byteswap functions look like possible aliasing violations too, e.g. c855be8#diff-e64873a81f0bfbd0c2b519bed04f66272560b25e1d78f65a4f685230364a2b0dR84. You can't cast between e.g. |
Thanks for pointing this out! I didn't know about this aliasing rule. I will fix this. For the unaligned accesses, is there any way to avoid them? The SAS7BDAT file format effectively requires unaligned reads, eg. when reading columns of non-power-of-2 length |
I looked into portable-endianness but it doesn't seem to compile to efficient code with MSVC. |
Unaligned access is UB, even on x86. On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
The
pandas/tests/io/sas/test_byteswap.py::test_float_byteswap
tests SIGBUS for me on arm (arm64 host, armv7 chroot).When building with UBSAN (
-fsanitize=undefined
), I get the following:Both these tests and this functionality were introduced in c855be8 (cc @jonashaag).
Expected Behavior
All tests pass.
Installed Versions
INSTALLED VERSIONS
commit : 0f43794
python : 3.11.4.final.0
python-bits : 32
OS : Linux
OS-release : 5.15.117-gentoo-dist
Version : #1 SMP Wed Jun 14 13:14:49 -00 2023
machine : armv8l
processor : ARMv8 Processor rev 1 (v8l)
byteorder : little
LC_ALL : None
LANG : C.UTF8
LOCALE : en_US.UTF-8
pandas : 2.0.3
numpy : 1.25.2
pytz : 2023.3
dateutil : 2.8.2
setuptools : 68.0.0
pip : None
Cython : 3.0.0
pytest : 7.4.0
hypothesis : 6.82.0
sphinx : None
blosc : 1.11.1
feather : None
xlsxwriter : 3.1.2
lxml.etree : 4.9.3
html5lib : 1.1
pymysql : 1.4.6
psycopg2 : 2.9.4
jinja2 : 3.1.2
IPython : None
pandas_datareader: None
bs4 : 4.12.2
bottleneck : 1.3.7
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.7.2
numba : None
numexpr : 2.8.4
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.11.1
snappy : None
sqlalchemy : 2.0.19
tables : 3.8.0
tabulate : 0.9.0
xarray : 2023.7.0
xlrd : 2.0.1
zstandard : None
tzdata : None
qtpy : None
pyqt5 : None
The text was updated successfully, but these errors were encountered: