Skip to content

Commit dfe7cc7

Browse files
committed
Document SAC and super stream usage in performance tool
References #46
1 parent 6807639 commit dfe7cc7

File tree

4 files changed

+74
-4
lines changed

4 files changed

+74
-4
lines changed

src/docs/asciidoc/performance-tool.adoc

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,7 @@ The accepted values for `--offset` are `first`, `last`, `next` (the default),
411411
an unsigned long for a given offset, and an ISO 8601 formatted timestamp
412412
(eg. `2020-06-03T07:45:54Z`).
413413

414+
[[performance-tool-offset-tracking]]
414415
===== Offset Tracking (Consumer)
415416

416417
A consumer can <<api.adoc#consumer-offset-tracking,track the point>> it has reached
@@ -485,6 +486,74 @@ java -jar stream-perf-test.jar --producer-names %s-%d
485486

486487
The run will start one producer and will use the `stream-1` producer reference (default stream is `stream` and the number of the producer is 1.)
487488

489+
[[performance-tool-sac]]
490+
===== Single Active Consumer
491+
492+
If the `--single-active-consumer` flag is set, the performance tool will create <<api.adoc#single-active-consumer, single active consumer>> instances.
493+
This means that if there are more consumers than streams, there will be only one active consumer at a time on a stream, _if they share the same name_.
494+
Note <<performance-tool-offset-tracking, offset tracking>> gets enabled automatically if it's not with `--single-active-consumer` (using 10,000 for `--store-every`).
495+
Let's see a couple of examples.
496+
497+
In the following command we have 1 producer publishing to 1 stream and 3 consumers on this stream.
498+
As `--single-active-consumer` is used, only one of these consumers will be active at a time.
499+
500+
----
501+
java -jar stream-perf-test.jar --producers 1 --consumers 3 --single-active-consumer \
502+
--consumer-names my-app
503+
----
504+
505+
Note we use a fixed value for the consumer names: if they don't have the same name, the broker will not consider them as a group of consumers, so they will all get messages, like regular consumers.
506+
507+
In the following example we have 2 producers for 2 streams and 6 consumers overall (3 for each stream).
508+
Note the consumers have the same name on their streams with the use of `--consumer-names my-app-%s`, as `%s` is a <<consumer-names, placeholder for the stream name>>.
509+
510+
----
511+
java -jar stream-perf-test.jar --producers 2 --consumers 6 --stream-count 2 \
512+
--single-active-consumer --consumer-names my-app-%s
513+
----
514+
515+
516+
===== Super Streams
517+
518+
The performance tool has a `--super-streams` flag to enable <<super-streams.adoc#super-streams, super streams>> on the publisher and consumer sides.
519+
This support is meant to be used with the <<performance-tool-sac, `--single-active-consumer` flag>>, to <<super-streams.adoc#super-stream-sac, benefit from both features>>.
520+
We recommend reading the appropriate sections of the documentation to understand the semantics of the flags before using them.
521+
Let's see some examples.
522+
523+
The example below creates 1 producer and 3 consumers on the default `stream`, which is now a _super stream_ because of the `--super-streams` flag:
524+
525+
----
526+
java -jar stream-perf-test.jar --producers 1 --consumers 3 --single-active-consumer \
527+
--super-streams --consumer-names my-app
528+
----
529+
530+
The performance tool creates 3 individual streams by default, they are the partitions of the super stream.
531+
They are named `stream-0`, `stream-1`, and `stream-2`, after the name of the super stream, `stream`.
532+
The producer will publish to each of them, using a <<super-streams.adoc#super-stream-producer, hash-based routing strategy>>.
533+
534+
A consumer is _composite_ with `--super-streams`: it creates a consumer instance for each partition.
535+
This is 9 consumer instances overall – 3 composite consumers and 3 partitions – spread evenly across the partitions, but with only one active at a time on a given stream.
536+
537+
Note we use a fixed consumer name so that the broker considers the consumers belong to the same group and enforce the single active consumer behavior.
538+
539+
The next example is more convoluted.
540+
We are going to work with 2 super streams (`--stream-count 2` and `--super-streams`).
541+
Each super stream will have 5 partitions (`--super-stream-partitions 5`), so this is 10 streams overall (`stream-1-0` to `stream-1-4` and `stream-2-0` to `stream-2-4`).
542+
Here is the command line:
543+
544+
----
545+
java -jar stream-perf-test.jar --producers 2 --consumers 6 --stream-count 2 \
546+
--super-streams --super-stream-partitions 5 \
547+
--single-active-consumer \
548+
--consumer-names my-app-%s
549+
----
550+
551+
We see also that each super stream has 1 producer (`--producers 2`) and 3 consumers (`--consumers 6`).
552+
The composite consumers will spread their consumer instances across the partitions.
553+
Each partition will have 3 consumers but only 1 active at a time with `--single-active-consumer` and `--consumer-names my-app-%s` (the consumers on a given stream have the same name, so the broker make sure only one consumes at a time).
554+
555+
556+
488557
===== Monitoring
489558

490559
The tool can expose some runtime information on HTTP.

src/docs/asciidoc/super-streams.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ When a super stream is in use, the stream Java client queries this information t
4747
From the application code point of view, using a super stream is mostly configuration-based.
4848
Some logic must also be provided to extract routing information from messages.
4949

50+
[[super-stream-producer]]
5051
===== Publishing to a Super Stream
5152

5253
When the topology of a super stream like the one described above has been set, creating a producer for it is straightforward:

src/main/java/com/rabbitmq/stream/perf/StreamPerfTest.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -358,7 +358,7 @@ public class StreamPerfTest implements Callable<Integer> {
358358
private boolean superStreams;
359359

360360
@CommandLine.Option(
361-
names = {"--super-streams-partitions", "-ssp"},
361+
names = {"--super-stream-partitions", "-ssp"},
362362
description = "number of partitions for the super streams",
363363
defaultValue = "3",
364364
converter = Utils.PositiveIntegerTypeConverter.class)

src/test/java/com/rabbitmq/stream/perf/StreamPerfTestTest.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,7 @@ void singleActiveConsumersOnSuperStream() throws Exception {
382382
.storeEvery(10)
383383
.streamCount(2)
384384
.superStreams()
385-
.superStreamsPartitions(3)
385+
.superStreamPartitions(3)
386386
.singleActiveConsumer()
387387
.storeEvery(10)
388388
.consumerNames(consumerName)
@@ -589,8 +589,8 @@ ArgumentsBuilder superStreams() {
589589
return this;
590590
}
591591

592-
ArgumentsBuilder superStreamsPartitions(int partitions) {
593-
arguments.put("super-streams-partitions", String.valueOf(partitions));
592+
ArgumentsBuilder superStreamPartitions(int partitions) {
593+
arguments.put("super-stream-partitions", String.valueOf(partitions));
594594
return this;
595595
}
596596

0 commit comments

Comments
 (0)