Redis 6.2.12 crashed by signal: 6

[jcq15]

I'm using redis-om-python with redis-stack. Here is the report. Thanks for any help!

=== REDIS BUG REPORT START: Cut & paste starting from here ===
9:M 28 Jul 2023 01:15:23.374 # Redis 6.2.12 crashed by signal: 6, si_code: -6
9:M 28 Jul 2023 01:15:23.374 # Crashed running the instruction at: 0x7f801bdf600b

------ STACK TRACE ------
EIP:
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f801bdf600b]

Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f801bfb9420]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f801bdf600b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f801bdd5859]
/opt/redis-stack/lib/rejson.so(+0x1c85f7)[0x7f80146985f7]
/opt/redis-stack/lib/rejson.so(+0x1c6af2)[0x7f8014696af2]
/opt/redis-stack/lib/rejson.so(+0x1c697a)[0x7f801469697a]
/opt/redis-stack/lib/rejson.so(+0x1c6672)[0x7f8014696672]
/opt/redis-stack/lib/rejson.so(+0x1c5726)[0x7f8014695726]
/opt/redis-stack/lib/rejson.so(+0x1c63c2)[0x7f80146963c2]
/opt/redis-stack/lib/rejson.so(+0x3f203)[0x7f801450f203]
/opt/redis-stack/lib/rejson.so(+0x3f29d)[0x7f801450f29d]
/opt/redis-stack/lib/rejson.so(+0x930c3)[0x7f80145630c3]
/opt/redis-stack/lib/rejson.so(JSONAPI_getInt+0x6b)[0x7f80145bbbab]
/opt/redis-stack/lib/redisearch.so(JSON_StoreInDocField+0x113)[0x7f801b1915d3]
/opt/redis-stack/lib/redisearch.so(JSON_LoadDocumentField+0xd1)[0x7f801b191841]
/opt/redis-stack/lib/redisearch.so(Document_LoadSchemaFieldJson+0x11d)[0x7f801b17417d]
/opt/redis-stack/lib/redisearch.so(IndexSpec_UpdateDoc+0x1a0)[0x7f801b1bb3e0]
/opt/redis-stack/lib/redisearch.so(Indexes_UpdateMatchingWithSchemaRules+0xd6)[0x7f801b1bbb06]
/opt/redis-stack/lib/redisearch.so(HashNotificationCallback+0x1fd)[0x7f801b1964fd]
/opt/redis-stack/bin/redis-server *:6379(moduleNotifyKeyspaceEvent+0xfc)[0x55bac94697ec]
/opt/redis-stack/bin/redis-server *:6379(notifyKeyspaceEvent+0x35)[0x55bac9455775]
/opt/redis-stack/bin/redis-server *:6379(RM_NotifyKeyspaceEvent+0x33)[0x55bac94604b3]
/opt/redis-stack/lib/rejson.so(+0x10bf62)[0x7f80145dbf62]
/opt/redis-stack/lib/rejson.so(+0x4ad51)[0x7f801451ad51]
/opt/redis-stack/lib/rejson.so(+0xee697)[0x7f80145be697]
/opt/redis-stack/bin/redis-server *:6379(RedisModuleCommandDispatcher+0x67)[0x55bac9467417]
/opt/redis-stack/bin/redis-server *:6379(call+0xf0)[0x55bac93d8940]
/opt/redis-stack/bin/redis-server *:6379(processCommand+0x643)[0x55bac93da643]
/opt/redis-stack/bin/redis-server *:6379(processCommandAndResetClient+0x20)[0x55bac93edd20]
/opt/redis-stack/bin/redis-server *:6379(processInputBuffer+0xea)[0x55bac93f067a]
/opt/redis-stack/bin/redis-server *:6379(+0x1018ac)[0x55bac94898ac]
/opt/redis-stack/bin/redis-server *:6379(aeProcessEvents+0x2ca)[0x55bac93d0e6a]
/opt/redis-stack/bin/redis-server *:6379(aeMain+0x1d)[0x55bac93d10fd]
/opt/redis-stack/bin/redis-server *:6379(main+0x33a)[0x55bac93cd50a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f801bdd7083]
/opt/redis-stack/bin/redis-server *:6379(_start+0x2e)[0x55bac93cda0e]

------ REGISTERS ------
9:M 28 Jul 2023 01:15:23.375 #
RAX:0000000000000000 RBX:00007f801bdaef80
RCX:00007f801bdf600b RDX:0000000000000000
RDI:0000000000000002 RSI:00007ffee932c5e0
RBP:00007ffee932ca58 RSP:00007ffee932c5e0
R8 :0000000000000000 R9 :00007ffee932c5e0
R10:0000000000000008 R11:0000000000000246
R12:0000000000000001 R13:0000000000000001
R14:00007f8014786528 R15:00007ffee932c990
RIP:00007f801bdf600b EFL:0000000000000246
CSGSFS:002b000000000033
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5ef) -> 00007ffee932c720
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5ee) -> 0000000000000000
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5ed) -> 0000000000000000
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5ec) -> 0000000000000010
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5eb) -> 00007f80147865c0
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5ea) -> 00007ffee932c7e0
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e9) -> 00007f80146931ed
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e8) -> 00007ffee932c697
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e7) -> 0000000000000001
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e6) -> 00007ffee932c697
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e5) -> 0000000000000001
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e4) -> 0000000000000001
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e3) -> 00007f801bfb8344
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e2) -> 0000000000000000
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e1) -> 00007ffee932c697
9:M 28 Jul 2023 01:15:23.375 # (00007ffee932c5e0) -> 0000000000000000

------ INFO OUTPUT ------

Server

redis_version:6.2.12
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:4dbc2487343b0024
redis_mode:standalone
os:Linux 5.19.0-1025-aws x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:9.4.0
process_id:9
process_supervised:no
run_id:1f7c0d1b5078c36ec6fdd4e84b6807690e574d81
tcp_port:6379
server_time_usec:1690506923358641
uptime_in_seconds:45628
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:12785323
executable:/opt/redis-stack/bin/redis-server
config_file:/redis-stack.conf
io_threads_active:0

Clients

connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:168
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

Memory

used_memory:35747072
used_memory_human:34.09M
used_memory_rss:52899840
used_memory_rss_human:50.45M
used_memory_peak:35747072
used_memory_peak_human:34.09M
used_memory_peak_perc:100.01%
used_memory_overhead:2267336
used_memory_startup:1014472
used_memory_dataset:33479736
used_memory_dataset_perc:96.39%
allocator_allocated:35827568
allocator_active:36835328
allocator_resident:39825408
total_system_memory:4110970880
total_system_memory_human:3.83G
used_memory_lua:30720
used_memory_lua_human:30.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.03
allocator_frag_bytes:1007760
allocator_rss_ratio:1.08
allocator_rss_bytes:2990080
rss_overhead_ratio:1.33
rss_overhead_bytes:13074432
mem_fragmentation_ratio:1.48
mem_fragmentation_bytes:17197296
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:20648
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

Persistence

loading:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:60
rdb_bgsave_in_progress:0
rdb_last_save_time:1690506920
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:1265664
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

Stats

total_connections_received:2
total_commands_processed:212
instantaneous_ops_per_sec:6
total_net_input_bytes:74545
total_net_output_bytes:83358
instantaneous_input_kbps:2.63
instantaneous_output_kbps:0.85
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:941
evicted_keys:0
keyspace_hits:282995
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:1347
total_forks:21
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:212
total_writes_processed:210
io_threaded_reads_processed:0
io_threaded_writes_processed:0

Replication

role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:8296e900be401189944f8ea16c1f6142d9585112
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

CPU

used_cpu_sys:6.724217
used_cpu_user:64.395662
used_cpu_sys_children:0.550217
used_cpu_user_children:4.194333
used_cpu_sys_main_thread:6.578226
used_cpu_user_main_thread:62.951034

Modules

module:name=ReJSON,ver=20407,api=1,filters=0,usedby=[search|graph],using=[],options=[handle-io-errors]
module:name=search,ver=20609,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors]
module:name=graph,ver=21010,api=1,filters=0,usedby=[],using=[ReJSON],options=[]
module:name=timeseries,ver=10810,api=1,filters=0,usedby=[],using=[],options=[handle-io-errors]
module:name=bf,ver=20405,api=1,filters=0,usedby=[],using=[],options=[]

Commandstats

cmdstat_command:calls=2,usec=20,usec_per_call=10.00,rejected_calls=0,failed_calls=0
cmdstat_hset:calls=132,usec=15934,usec_per_call=120.71,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=2,usec=11,usec_per_call=5.50,rejected_calls=0,failed_calls=0
cmdstat_FT.SEARCH:calls=51,usec=392995,usec_per_call=7705.78,rejected_calls=0,failed_calls=0
cmdstat_info:calls=2,usec=38,usec_per_call=19.00,rejected_calls=0,failed_calls=0
cmdstat_json.set:calls=11,usec=1603,usec_per_call=145.73,rejected_calls=0,failed_calls=0
cmdstat_FT.INFO:calls=6,usec=288,usec_per_call=48.00,rejected_calls=0,failed_calls=0
cmdstat_get:calls=6,usec=23,usec_per_call=3.83,rejected_calls=0,failed_calls=0

Errorstats

Cluster

cluster_enabled:0

Keyspace

db0:keys=24251,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=25 addr=172.17.0.1:45744 laddr=172.17.0.3:6379 fd=12 name= age=45543 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=512 qbuf-free=40442 argv-mem=481 obl=0 oll=0 omem=0 tot-mem=61953 events=r cmd=json.set user=default redir=-1

------ CURRENT CLIENT INFO ------
id=25 addr=172.17.0.1:45744 laddr=172.17.0.3:6379 fd=12 name= age=45543 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=512 qbuf-free=40442 argv-mem=481 obl=0 oll=0 omem=0 tot-mem=61953 events=r cmd=json.set user=default redir=-1
argv[0]: 'JSON.SET'
argv[1]: 'karma:bots.karma.UserTable:01H698K7M2BPAFF93T2GNXAQ39'
argv[2]: '.'
argv[3]: '{"pk": "01H698K7M2BPAFF93T2GNXAQ39", "user_id": "98acba66", "name": "\u5c3c\u83ab\u514b", "word": "4", "bought": [21, 21, 24, 22, 21, 101], "items": [25616506980096071, 59826576399327, 113507698638, 165643800, 169791, 101], "prices": [12800, 256000, 10240000, 102400000, 1536000000, 1374389534720000000], "eff": [1, 1, 1, 1, 1, 1], "level": 5, "karma": 2368245463503591660, "total": 9240193141634191760, "base_eff": 10}'
9:M 28 Jul 2023 01:15:23.375 # key 'karma:bots.karma.UserTable:01H698K7M2BPAFF93T2GNXAQ39' found in DB containing the following object:
9:M 28 Jul 2023 01:15:23.375 # Object type: 5
9:M 28 Jul 2023 01:15:23.375 # Object encoding: 0
9:M 28 Jul 2023 01:15:23.375 # Object refcount: 1

------ MODULES INFO OUTPUT ------

ReJSON_trace

ReJSON_trace: 0: redis_module::base_info_func
1: rejson::__info_func
2: modulesCollectInfo
at /__w/redis-stack/redis-stack/redis/src/module.c:7100:9
3: logModulesInfo
at /__w/redis-stack/redis-stack/redis/src/debug.c:1624:22
4: printCrashReport
at /__w/redis-stack/redis-stack/redis/src/debug.c:1896:5
5: sigsegvHandler
at /__w/redis-stack/redis-stack/redis/src/debug.c:1878:5
6:
7: gsignal
8: abort
9: std::sys::unix::abort_internal
10: rust_panic
11: std::panicking::rust_panic_with_hook
12: std::panicking::begin_panic_handler::{{closure}}
13: std::sys_common::backtrace::__rust_end_short_backtrace
14: rust_begin_unwind
15: core::panicking::panic_fmt
16: core::panicking::panic
17: rejson::jsonpath::json_node::::get_long
18: JSONAPI_getInt
19: JSON_StoreInDocField
20: JSON_LoadDocumentField
21: Document_LoadSchemaFieldJson
22: IndexSpec_UpdateDoc
23: Indexes_UpdateMatchingWithSchemaRules
24: HashNotificationCallback
25: moduleNotifyKeyspaceEvent
at /__w/redis-stack/redis-stack/redis/src/module.c:5983:13
26: notifyKeyspaceEvent
at /__w/redis-stack/redis-stack/redis/src/notify.c:111:6
27: RM_NotifyKeyspaceEvent
at /__w/redis-stack/redis-stack/redis/src/module.c:5951:5
28: redis_module::raw::notify_keyspace_event
29: rejson::commands::json_set
30: rejson::RedisModule_OnLoad::__do_command
31: RedisModuleCommandDispatcher
at /__w/redis-stack/redis-stack/redis/src/module.c:695:5
32: call
at /__w/redis-stack/redis-stack/redis/src/server.c:3750:5
33: processCommand
at /__w/redis-stack/redis-stack/redis/src/server.c:4297:9
34: processCommandAndResetClient
at /__w/redis-stack/redis-stack/redis/src/networking.c:2105:9
35: processInputBuffer
at /__w/redis-stack/redis-stack/redis/src/networking.c:2206:17
36: callHandler
at /__w/redis-stack/redis-stack/redis/src/connhelpers.h:79:18
connSocketEventHandler
at /__w/redis-stack/redis-stack/redis/src/connection.c:295:14
37: aeProcessEvents
at /__w/redis-stack/redis-stack/redis/src/ae.c:427:17
38: aeMain
at /__w/redis-stack/redis-stack/redis/src/ae.c:487:9
39: main
at /__w/redis-stack/redis-stack/redis/src/server.c:6474:5
40: __libc_start_main
41: _start

search_version

search_version:2.6.9
search_redis_version:6.2.12 - oss

search_index

search_number_of_indexes:3

search_fields_statistics

search_fields_numeric:Numeric=7
search_fields_tag:Tag=9

search_dialect_statistics

search_dialect_1:1
search_dialect_2:0
search_dialect_3:0

search_runtime_configurations

search_concurrent_mode:OFF
search_enableGC:ON
search_minimal_term_prefix:2
search_maximal_prefix_expansions:200
search_query_timeout_ms:500
search_timeout_policy:return
search_cursor_read_size:1000
search_cursor_max_idle_time:300000
search_max_doc_table_size:1000000
search_max_search_results:10000
search_max_aggregate_results:10000
search_search_pool_size:20
search_index_pool_size:8
search_gc_scan_size:100
search_min_phonetic_term_length:3

graph_executing commands

------ FAST MEMORY TEST ------
9:M 28 Jul 2023 01:15:23.401 # Bio thread for job type #0 terminated
9:M 28 Jul 2023 01:15:23.401 # Bio thread for job type #1 terminated
9:M 28 Jul 2023 01:15:23.401 # Bio thread for job type #2 terminated
*** Preparing to test memory region 55bac95a1000 (2281472 bytes)
*** Preparing to test memory region 55bacabc0000 (405504 bytes)
*** Preparing to test memory region 7f8008000000 (135168 bytes)
*** Preparing to test memory region 7f800f000000 (8388608 bytes)
*** Preparing to test memory region 7f800f800000 (4194304 bytes)
*** Preparing to test memory region 7f800fdcc000 (40894464 bytes)
*** Preparing to test memory region 7f80124cd000 (8388608 bytes)
*** Preparing to test memory region 7f8012cce000 (8388608 bytes)
*** Preparing to test memory region 7f80134cf000 (8388608 bytes)
*** Preparing to test memory region 7f8013cd0000 (8388608 bytes)
*** Preparing to test memory region 7f801478c000 (8388608 bytes)
*** Preparing to test memory region 7f8014f8d000 (8388608 bytes)
*** Preparing to test memory region 7f801578e000 (8388608 bytes)
*** Preparing to test memory region 7f8015f8f000 (8388608 bytes)
*** Preparing to test memory region 7f8016790000 (8388608 bytes)
*** Preparing to test memory region 7f8016f91000 (8388608 bytes)
*** Preparing to test memory region 7f8017792000 (8388608 bytes)
*** Preparing to test memory region 7f8017f93000 (8388608 bytes)
*** Preparing to test memory region 7f8019ec3000 (12288 bytes)
*** Preparing to test memory region 7f8019ec7000 (8388608 bytes)
*** Preparing to test memory region 7f801a6c8000 (8388608 bytes)
*** Preparing to test memory region 7f801b0a7000 (12288 bytes)
*** Preparing to test memory region 7f801b3fd000 (12288 bytes)
*** Preparing to test memory region 7f801b400000 (8388608 bytes)
*** Preparing to test memory region 7f801bd4f000 (4096 bytes)
*** Preparing to test memory region 7f801bdad000 (24576 bytes)
*** Preparing to test memory region 7f801bfa1000 (16384 bytes)
*** Preparing to test memory region 7f801bfc4000 (16384 bytes)
*** Preparing to test memory region 7f801c29b000 (16384 bytes)
*** Preparing to test memory region 7f801c487000 (8192 bytes)
*** Preparing to test memory region 7f801c4ba000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: gsignal (base: 0x7f801bdf5f40)
Module: /lib/x86_64-linux-gnu/libc.so.6 (base 0x7f801bdb3000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x7f801bdf5f40 -D -b binary -m i386:x86-64 /tmp/dump.bin

9:M 28 Jul 2023 01:15:24.038 # dump of function (hexdump of 331 bytes):
f30f1efa4881ec180100004189f831ff660f6f058898170064488b042528000000488984240801000031c04989e1b80e000000488db424800000000f29842480000000660f76c041ba080000004c89ca0f298424900000000f298424a00000000f298424b00000000f298424c00000000f298424d00000000f298424e00000000f298424f00000000f05b92700000089c80f054889c7b8ba0000000f0589c64489c2b8ea0000000f05483d00f0ffff773f4189c041ba0800000031d24c89cebf02000000b80e0000000f05488b84240801000064483304252800000075264489c04881c418010000c30f1f8000000000488b15398e1a00f7d841b8ffffffff648902ebb0e827ca0e000f1f8000000000f30f1efa85ff7808f7dfe97103000090488b05098e1a0064c70016000000b8ffffffffc3662e0f1f84000000000066909066662e0f1f8400000000
Function at 0x7f801bee2a70 is __stack_chk_fail

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

[jcq15]

It's solved. There is a long int (larger than 2^64) in my python script which caused some basic service crashed.