Skip to content

Commit ca38ba4

Browse files
bors[bot]saik0
andauthored
131: Append perf bugfix r=Kerollmops a=saik0 Fixes RoaringBitmap#130 ## Cause Where _N_ is the length of the iterator: _N_ calls to `RoaringBitmap::push` required N calls to`RoaringBitmap::max`, which is not cheap. ## Solution Compare only the first item in the iterator to the bitmap max, then compare each element to its predecessor. ## Future work This could be strictly faster than inserting, given that it's always inserting into the last container (or pushing a new container) ## Before ![Screen Shot 2022-01-09 at 9 35 20 PM](https://user-images.githubusercontent.com/997050/148730686-152f7232-4ef0-43da-b37f-7bef6d1502da.png) ![Screen Shot 2022-01-09 at 9 35 36 PM](https://user-images.githubusercontent.com/997050/148730691-a15045a7-0abc-47c3-8208-b95aa406a681.png) ## After ![Screen Shot 2022-01-09 at 11 13 35 PM](https://user-images.githubusercontent.com/997050/148729666-55c48265-d38a-4865-a92d-ceb83efdb500.png) ![Screen Shot 2022-01-09 at 11 13 43 PM](https://user-images.githubusercontent.com/997050/148729673-41dfc7e1-62ca-4cc4-8c7c-8cd3b59adc4c.png) Co-authored-by: saik0 <[email protected]>
2 parents 5101a10 + b098dcd commit ca38ba4

File tree

2 files changed

+52
-4
lines changed

2 files changed

+52
-4
lines changed

src/bitmap/iter.rs

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -184,13 +184,35 @@ impl RoaringBitmap {
184184
&mut self,
185185
iterator: I,
186186
) -> Result<u64, NonSortedIntegers> {
187-
let mut count = 0;
187+
// Name shadowed to prevent accidentally referencing the param
188+
let mut iterator = iterator.into_iter();
189+
190+
let mut prev: u32 = match iterator.next() {
191+
None => return Ok(0),
192+
Some(first) => {
193+
if let Some(max) = self.max() {
194+
if first <= max {
195+
return Err(NonSortedIntegers { valid_until: 0 });
196+
}
197+
}
198+
199+
first
200+
}
201+
};
202+
203+
self.insert(prev);
204+
let mut count = 1;
205+
206+
// It is now guaranteed that so long as the values are iterator are monotonically
207+
// increasing they must also be the greatest in the set.
188208

189209
for value in iterator {
190-
if self.push(value) {
191-
count += 1;
192-
} else {
210+
if value <= prev {
193211
return Err(NonSortedIntegers { valid_until: count });
212+
} else {
213+
self.insert(value);
214+
prev = value;
215+
count += 1;
194216
}
195217
}
196218

tests/push.rs

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,32 @@ fn append() {
2525
test_from_sorted_iter!(vec![1, 2, 4, 5, 7, 8, 9], RoaringBitmap);
2626
}
2727

28+
#[test]
29+
fn append_empty() {
30+
assert_eq!(RoaringBitmap::new().append(vec![]), Ok(0u64))
31+
}
32+
33+
#[test]
34+
fn append_error() {
35+
match [100u32].iter().cloned().collect::<RoaringBitmap>().append(vec![10, 20, 0]) {
36+
Ok(_) => {
37+
panic!("The 0th element in the iterator was < the max of the bitmap")
38+
}
39+
Err(non_sorted_error) => {
40+
assert_eq!(non_sorted_error.valid_until(), 0)
41+
}
42+
}
43+
44+
match [100u32].iter().cloned().collect::<RoaringBitmap>().append(vec![200, 201, 201]) {
45+
Ok(_) => {
46+
panic!("The 3rd element in the iterator was < 2nd")
47+
}
48+
Err(non_sorted_error) => {
49+
assert_eq!(non_sorted_error.valid_until(), 2)
50+
}
51+
}
52+
}
53+
2854
#[test]
2955
fn append_tree() {
3056
test_from_sorted_iter!((0..1_000_000).map(|x| 13 * x).collect::<Vec<u64>>(), RoaringTreemap);

0 commit comments

Comments
 (0)