Skip to content

Commit 90ab4ca

Browse files
committed
Document SAC and super stream usage in performance tool
References #46
1 parent 32746d6 commit 90ab4ca

File tree

4 files changed

+74
-4
lines changed

4 files changed

+74
-4
lines changed

src/docs/asciidoc/performance-tool.adoc

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -391,6 +391,7 @@ The accepted values for `--offset` are `first`, `last`, `next` (the default),
391391
an unsigned long for a given offset, and an ISO 8601 formatted timestamp
392392
(eg. `2020-06-03T07:45:54Z`).
393393

394+
[[performance-tool-offset-tracking]]
394395
===== Offset Tracking (Consumer)
395396

396397
A consumer can <<api.adoc#consumer-offset-tracking,track the point>> it has reached
@@ -465,6 +466,74 @@ java -jar stream-perf-test.jar --producer-names %s-%d
465466

466467
The run will start one producer and will use the `stream-1` producer reference (default stream is `stream` and the number of the producer is 1.)
467468

469+
[[performance-tool-sac]]
470+
===== Single Active Consumer
471+
472+
If the `--single-active-consumer` flag is set, the performance tool will create <<api.adoc#single-active-consumer, single active consumer>> instances.
473+
This means that if there are more consumers than streams, there will be only one active consumer at a time on a stream, _if they share the same name_.
474+
Note <<performance-tool-offset-tracking, offset tracking>> gets enabled automatically if it's not with `--single-active-consumer` (using 10,000 for `--store-every`).
475+
Let's see a couple of examples.
476+
477+
In the following command we have 1 producer publishing to 1 stream and 3 consumers on this stream.
478+
As `--single-active-consumer` is used, only one of these consumers will be active at a time.
479+
480+
----
481+
java -jar stream-perf-test.jar --producers 1 --consumers 3 --single-active-consumer \
482+
--consumer-names my-app
483+
----
484+
485+
Note we use a fixed value for the consumer names: if they don't have the same name, the broker will not consider them as a group of consumers, so they will all get messages, like regular consumers.
486+
487+
In the following example we have 2 producers for 2 streams and 6 consumers overall (3 for each stream).
488+
Note the consumers have the same name on their streams with the use of `--consumer-names my-app-%s`, as `%s` is a <<consumer-names, placeholder for the stream name>>.
489+
490+
----
491+
java -jar stream-perf-test.jar --producers 2 --consumers 6 --stream-count 2 \
492+
--single-active-consumer --consumer-names my-app-%s
493+
----
494+
495+
496+
===== Super Streams
497+
498+
The performance tool has a `--super-streams` flag to enable <<super-streams.adoc#super-streams, super streams>> on the publisher and consumer sides.
499+
This support is meant to be used with the <<performance-tool-sac, `--single-active-consumer` flag>>, to <<super-streams.adoc#super-stream-sac, benefit from both features>>.
500+
We recommend reading the appropriate sections of the documentation to understand the semantics of the flags before using them.
501+
Let's see some examples.
502+
503+
The example below creates 1 producer and 3 consumers on the default `stream`, which is now a _super stream_ because of the `--super-streams` flag:
504+
505+
----
506+
java -jar stream-perf-test.jar --producers 1 --consumers 3 --single-active-consumer \
507+
--super-streams --consumer-names my-app
508+
----
509+
510+
The performance tool creates 3 individual streams by default, they are the partitions of the super stream.
511+
They are named `stream-0`, `stream-1`, and `stream-2`, after the name of the super stream, `stream`.
512+
The producer will publish to each of them, using a <<super-streams.adoc#super-stream-producer, hash-based routing strategy>>.
513+
514+
A consumer is _composite_ with `--super-streams`: it creates a consumer instance for each partition.
515+
This is 9 consumer instances overall – 3 composite consumers and 3 partitions – spread evenly across the partitions, but with only one active at a time on a given stream.
516+
517+
Note we use a fixed consumer name so that the broker considers the consumers belong to the same group and enforce the single active consumer behavior.
518+
519+
The next example is more convoluted.
520+
We are going to work with 2 super streams (`--stream-count 2` and `--super-streams`).
521+
Each super stream will have 5 partitions (`--super-stream-partitions 5`), so this is 10 streams overall (`stream-1-0` to `stream-1-4` and `stream-2-0` to `stream-2-4`).
522+
Here is the command line:
523+
524+
----
525+
java -jar stream-perf-test.jar --producers 2 --consumers 6 --stream-count 2 \
526+
--super-streams --super-stream-partitions 5 \
527+
--single-active-consumer \
528+
--consumer-names my-app-%s
529+
----
530+
531+
We see also that each super stream has 1 producer (`--producers 2`) and 3 consumers (`--consumers 6`).
532+
The composite consumers will spread their consumer instances across the partitions.
533+
Each partition will have 3 consumers but only 1 active at a time with `--single-active-consumer` and `--consumer-names my-app-%s` (the consumers on a given stream have the same name, so the broker make sure only one consumes at a time).
534+
535+
536+
468537
===== Monitoring
469538

470539
The tool can expose some runtime information on HTTP.

src/docs/asciidoc/super-streams.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ When a super stream is in use, the stream Java client queries this information t
4747
From the application code point of view, using a super stream is mostly configuration-based.
4848
Some logic must also be provided to extract routing information from messages.
4949

50+
[[super-stream-producer]]
5051
===== Publishing to a Super Stream
5152

5253
When the topology of a super stream like the one described above has been set, creating a producer for it is straightforward:

src/main/java/com/rabbitmq/stream/perf/StreamPerfTest.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -358,7 +358,7 @@ public class StreamPerfTest implements Callable<Integer> {
358358
private boolean superStreams;
359359

360360
@CommandLine.Option(
361-
names = {"--super-streams-partitions", "-ssp"},
361+
names = {"--super-stream-partitions", "-ssp"},
362362
description = "number of partitions for the super streams",
363363
defaultValue = "3",
364364
converter = Utils.PositiveIntegerTypeConverter.class)

src/test/java/com/rabbitmq/stream/perf/StreamPerfTestTest.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,7 @@ void singleActiveConsumersOnSuperStream() throws Exception {
382382
.storeEvery(10)
383383
.streamCount(2)
384384
.superStreams()
385-
.superStreamsPartitions(3)
385+
.superStreamPartitions(3)
386386
.singleActiveConsumer()
387387
.storeEvery(10)
388388
.consumerNames(consumerName)
@@ -589,8 +589,8 @@ ArgumentsBuilder superStreams() {
589589
return this;
590590
}
591591

592-
ArgumentsBuilder superStreamsPartitions(int partitions) {
593-
arguments.put("super-streams-partitions", String.valueOf(partitions));
592+
ArgumentsBuilder superStreamPartitions(int partitions) {
593+
arguments.put("super-stream-partitions", String.valueOf(partitions));
594594
return this;
595595
}
596596

0 commit comments

Comments
 (0)