BUG: SIGBUS in test_float_byteswap test on arm #54391
Comments
thesamesam
added
Bug
Needs Triage
|
There's definitely unaligned access there (using the offsets), but the byteswap functions look like possible aliasing violations too, e.g. c855be8#diff-e64873a81f0bfbd0c2b519bed04f66272560b25e1d78f65a4f685230364a2b0dR84. You can't cast between e.g. |
3 tasks
|
Thanks for pointing this out! I didn't know about this aliasing rule. I will fix this. For the unaligned accesses, is there any way to avoid them? The SAS7BDAT file format effectively requires unaligned reads, eg. when reading columns of non-power-of-2 length |
|
I looked into portable-endianness but it doesn't seem to compile to efficient code with MSVC. |
5 tasks
jonashaag
added a commit
to jonashaag/pandas
that referenced
this issue
Aug 4, 2023
jonashaag
added a commit
to jonashaag/pandas
that referenced
this issue
Aug 4, 2023
jonashaag
added a commit
to jonashaag/pandas
that referenced
this issue
Aug 4, 2023
2 tasks
lithomas1
added
IO SAS
mroeschke
pushed a commit
that referenced
this issue
Aug 4, 2023
mroeschke
pushed a commit
to mroeschke/pandas
that referenced
this issue
Aug 18, 2023
thesamesam
added a commit
to thesamesam/c-blosc2
that referenced
this issue
Aug 21, 2023
Unaligned access is UB, even on x86. On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
thesamesam
added a commit
to thesamesam/c-blosc2
that referenced
this issue
Aug 21, 2023
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
thesamesam
added a commit
to thesamesam/c-blosc2
that referenced
this issue
Aug 21, 2023
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
thesamesam
added a commit
to thesamesam/c-blosc2
that referenced
this issue
Aug 21, 2023
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
thesamesam
added a commit
to thesamesam/c-blosc2
that referenced
this issue
Aug 21, 2023
Unaligned access is UB, even on x86. UBsan will also detect this (-fsanitize=undefined or -fsanitize=alignment). On arm, I hit SIGBUSes in pandas's test suite via pandas->pytables->c-blosc2 because of unaligned loads and stores. Modern compilers are capable of optimising the "slow" path bitshifts and memcpy into faster alternatives where it is legal. Bug: https://bugs.gentoo.org/911660 Bug: pandas-dev/pandas#54391 Bug: pandas-dev/pandas#54396 Signed-off-by: Sam James <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
The
pandas/tests/io/sas/test_byteswap.py::test_float_byteswaptests SIGBUS for me on arm (arm64 host, armv7 chroot).When building with UBSAN (
-fsanitize=undefined), I get the following:Both these tests and this functionality were introduced in c855be8 (cc @jonashaag).
Expected Behavior
All tests pass.
Installed Versions
INSTALLED VERSIONS
commit : 0f43794
python : 3.11.4.final.0
python-bits : 32
OS : Linux
OS-release : 5.15.117-gentoo-dist
Version : #1 SMP Wed Jun 14 13:14:49 -00 2023
machine : armv8l
processor : ARMv8 Processor rev 1 (v8l)
byteorder : little
LC_ALL : None
LANG : C.UTF8
LOCALE : en_US.UTF-8
pandas : 2.0.3
numpy : 1.25.2
pytz : 2023.3
dateutil : 2.8.2
setuptools : 68.0.0
pip : None
Cython : 3.0.0
pytest : 7.4.0
hypothesis : 6.82.0
sphinx : None
blosc : 1.11.1
feather : None
xlsxwriter : 3.1.2
lxml.etree : 4.9.3
html5lib : 1.1
pymysql : 1.4.6
psycopg2 : 2.9.4
jinja2 : 3.1.2
IPython : None
pandas_datareader: None
bs4 : 4.12.2
bottleneck : 1.3.7
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.7.2
numba : None
numexpr : 2.8.4
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.11.1
snappy : None
sqlalchemy : 2.0.19
tables : 3.8.0
tabulate : 0.9.0
xarray : 2023.7.0
xlrd : 2.0.1
zstandard : None
tzdata : None
qtpy : None
pyqt5 : None
The text was updated successfully, but these errors were encountered: