The library contains also a performance tool to test the RabbitMQ Stream plugin. It is usable as an uber JAR downloadable from GitHub Release or as a Docker image. It can be built separately as well.
Snapshots are on GitHub release as well. Use the pivotalrabbitmq/stream-perf-test:dev image to use the latest snapshot in Docker.
The performance tool is available as a Docker image. You can use the Docker image to list the available options:
docker run -it --rm pivotalrabbitmq/stream-perf-test --help
There are all sorts of options, if none is provided, the tool will start publishing to and consuming from a stream created only for the test.
When using Docker, the container running the performance tool must be able to connect to the broker, so you have to figure out the appropriate Docker configuration to make this possible. You can have a look at the Docker network documentation to find out more.
|
Note
|
Docker on macOS
Docker runs on a virtual machine when using macOS, so do not expect high performance when using RabbitMQ Stream and the performance tool inside Docker on a Mac. |
We show next a couple of options to easily use the Docker image.
This is the simplest way to run the image locally, with a local broker running in Docker as well. The containers use the host network, this is perfect for experimenting locally.
# run the broker docker run -it --rm --name rabbitmq --network host rabbitmq:3.9 # open another terminal and enable the stream plugin docker exec rabbitmq rabbitmq-plugins enable rabbitmq_stream # run the performance tool docker run -it --rm --network host pivotalrabbitmq/stream-perf-test
|
Note
|
Docker Host Network Driver Support
According to Docker’s documentation, the host networking driver only works on Linux hosts. Nevertheless, the commands above work on some Mac hosts. |
Containers need to be able to communicate with each other with the bridge network driver, this can be done by defining a network and running the containers in this network.
# create a network
docker network create stream-perf-test
# run the broker
docker run -it --rm --network stream-perf-test --name rabbitmq rabbitmq:3.9
# open another terminal and enable the stream plugin
docker exec rabbitmq rabbitmq-plugins enable rabbitmq_stream
# run the performance tool
docker run -it --rm --network stream-perf-test pivotalrabbitmq/stream-perf-test \
--uris rabbitmq-stream://rabbitmq:5552
The Java binary is available on GitHub Release. Snaphots are available as well. To use the latest snapshot:
wget https://github.com/rabbitmq/rabbitmq-java-tools-binaries-dev/releases/download/v-stream-perf-test-latest/stream-perf-test-latest.jar
To launch a run:
$ java -jar stream-perf-test-latest.jar 17:51:26.207 [main] INFO c.r.stream.perf.StreamPerfTest - Starting producer 1, published 560277 msg/s, confirmed 554088 msg/s, consumed 556983 msg/s, latency min/median/75th/95th/99th 2663/9799/13940/52304/57995 µs, chunk size 1125 2, published 770722 msg/s, confirmed 768209 msg/s, consumed 768585 msg/s, latency min/median/75th/95th/99th 2454/9599/12206/23940/55519 µs, chunk size 1755 3, published 915895 msg/s, confirmed 914079 msg/s, consumed 916103 msg/s, latency min/median/75th/95th/99th 2338/8820/11311/16750/52985 µs, chunk size 2121 4, published 1004257 msg/s, confirmed 1003307 msg/s, consumed 1004981 msg/s, latency min/median/75th/95th/99th 2131/8322/10639/14368/45094 µs, chunk size 2228 5, published 1061380 msg/s, confirmed 1060131 msg/s, consumed 1061610 msg/s, latency min/median/75th/95th/99th 2131/8247/10420/13905/37202 µs, chunk size 2379 6, published 1096345 msg/s, confirmed 1095947 msg/s, consumed 1097447 msg/s, latency min/median/75th/95th/99th 2131/8225/10334/13722/33109 µs, chunk size 2454 7, published 1127791 msg/s, confirmed 1127032 msg/s, consumed 1128039 msg/s, latency min/median/75th/95th/99th 1966/8150/10172/13500/23940 µs, chunk size 2513 8, published 1148846 msg/s, confirmed 1148086 msg/s, consumed 1149121 msg/s, latency min/median/75th/95th/99th 1966/8079/10135/13248/16771 µs, chunk size 2558 9, published 1167067 msg/s, confirmed 1166369 msg/s, consumed 1167311 msg/s, latency min/median/75th/95th/99th 1966/8063/9986/12977/16757 µs, chunk size 2631 10, published 1182554 msg/s, confirmed 1181938 msg/s, consumed 1182804 msg/s, latency min/median/75th/95th/99th 1966/7963/9949/12632/16619 µs, chunk size 2664 11, published 1197069 msg/s, confirmed 1196495 msg/s, consumed 1197291 msg/s, latency min/median/75th/95th/99th 1966/7917/9955/12503/15386 µs, chunk size 2761 12, published 1206687 msg/s, confirmed 1206176 msg/s, consumed 1206917 msg/s, latency min/median/75th/95th/99th 1966/7893/9975/12503/15280 µs, chunk size 2771 ... ^C Summary: published 1279444 msg/s, confirmed 1279019 msg/s, consumed 1279019 msg/s, latency 95th 12161 µs, chunk size 2910
The previous command will start publishing to and consuming from a stream stream that
will be created. The tool outputs live metrics on the console and write more
detailed metrics in a stream-perf-test-current.txt file that get renamed to
stream-perf-test-yyyy-MM-dd-HHmmss.txt when the run ends.
To see the options:
java -jar stream-perf-test-latest.jar --help
The performance tool comes also with a
completion script.
You can download it and enable it in
your ~/.zshrc file:
alias stream-perf-test='java -jar target/stream-perf-test.jar' source ~/.zsh/stream-perf-test_completion
Note the activation requires an alias which must be stream-perf-test. The command can be anything
though.
The performance tool connects by default to localhost, on port 5552, with
default credentials (guest/guest), on the default / virtual host.
This can be changed with the --uris option:
java -jar stream-perf-test.jar --uris rabbitmq-stream://rabbitmq-1:5552
The URI follows the same rules as the
AMQP 0.9.1 URI,
except the protocol must be rabbitmq-stream.
The next command shows how to set up the different elements of the URI:
java -jar stream-perf-test.jar \ --uris rabbitmq-stream://guest:guest@localhost:5552/%2f
The option accepts several values, separated by commas. By doing so, the tool will be able to pick another URI for its "locator" connection, in case a node crashes:
java -jar stream-perf-test.jar \ --uris rabbitmq-stream://rabbitmq-1:5552,rabbitmq-stream://rabbitmq-2:5552
Note the tool uses those URIs only for management purposes, it does not use them to distribute publishers and consumers across a cluster.
It is also possible to enable TLS by using the rabbitmq-stream+tls scheme:
java -jar stream-perf-test.jar \ --uris rabbitmq-stream+tls://guest:guest@localhost:5551/%2f
Note the performance tool will automatically configure the client to trust all server certificates and to not use a private key (for client authentication).
Have a look at the connection logic section in case of connection problem.
It is possible to limit the publishing rate with the --rate option:
java -jar stream-perf-test.jar --rate 10000
RabbitMQ Stream can easily saturate the resources of the hardware, it can especially max out the storage IO. Reasoning when a system is under severe constraints can be difficult, so setting a low publishing rate can be a good idea to get familiar with the performance tool and the semantics of streams.
You can set the number of producers and consumers with the --producers and
--consumers options, respectively:
java -jar stream-perf-test.jar --producers 5 --consumers 5
With the previous command, you should see a higher consuming rate than publishing rate. It is because the 5 producers publish as fast as they can and each consumer consume the messages from the 5 publishers. In theory the consumer rate should be 5 times the publishing rate, but as stated previously, the performance tool may put the broker under severe constraints, so the numbers may not add up.
You can set a low publishing rate to verify this theory:
java -jar stream-perf-test.jar --producers 5 --consumers 5 --rate 10000
With the previous command, each publisher should publish 10,000 messages per second, that is 50,000 messages per second overall. As each consumer consumes each published messages, the consuming rate should be 5 times the publishing rate, that is 250,000 messages per second. Using a small publishing rate should let plenty of resources to the system, so the rates should tend towards those values.
The performance tool uses a stream stream by default, the --streams option allows
specifying streams that the tool will try to create. Note producer
and consumer counts must be set accordingly, as they are not spread across the
stream automatically. The following command will run a test with 3 streams, with
a producer and a consumer on each of them:
java -jar stream-perf-test.jar --streams stream1,stream2,stream3 \
--producers 3 --consumers 3
If you do not want the tool to create and delete streams for a run, because they are already created,
use the --pre-declared option:
java -jar stream-perf-test.jar --streams stream1,stream2,stream3 \
--producers 3 --consumers 3 \
--pre-declared
The stream creation process has the following semantics:
-
the tool always tries to create streams.
-
if the target streams already exist and have the exact same properties as the ones the tool uses (see retention below), the run will start normally as stream creation is idempotent.
-
if the target streams already exist but do not have the exact same properties as the ones the tool uses, the creation process will fail, and the run will not start.
-
the streams are not deleted after the run.
-
if you want the tool to delete the streams after a run, use the
--delete-streamsflag.
Specifying streams one by one can become tedious as their number grows, so the --stream-count
option can be combined with the --streams option to specify a number or a range and a stream name
pattern, respectively. The following table shows the usage of these 2 options and the resulting
exercised streams. Do not forget to also specify the appropriate number of producers and
consumers if you want all the declared streams to be used.
| Options | Computed Streams | Details |
|---|---|---|
|
|
Stream count starts at 1. |
|
|
Possible to specify a Java printf-style format string. |
|
|
Not bad, but not correctly sorted alphabetically. |
|
|
Better for sorting. |
|
|
The default format string handles the sorting issue. |
|
|
Ranges are accepted. |
|
|
Default format string. |
The default publishing batch size is 100, that is a publishing frame is sent every 100 messages.
The following command sets the batch size to 50 with the --batch-size option:
java -jar stream-perf-test.jar --batch-size 50
There is no ideal batch size, it is a tradeoff between throughput and latency. High batch size values should increase throughput (usually good) and latency (usually not so good), whereas low batch size should decrease throughput (usually not good) and latency (usually good).
A publisher can have at most 10,000 unconfirmed messages at some point. If it reaches this value,
it has to wait until the broker confirms some messages. This avoids fast publishers overwhelming
the broker. The --confirms option allows changing the default value:
java -jar stream-perf-test.jar --confirms 20000
High values should increase throughput at the cost of consuming more memory, whereas low values should decrease throughput and memory consumption.
The default size of a message is 10 bytes, which is rather small. The --size option lets you
specify a different size, usually higher, to have a value close to your use case. The next command
sets a size of 1 KB:
java -jar stream-perf-test.jar --size 1024
Note the message body size cannot be smaller that 8 bytes, as the performance tool stores a long in each message to calculate the latency. Note also the actual size of a message will be slightly higher, as the body is wrapped in an AMQP 1.0 message.
If you run performance tests for a long time, you might be interested in setting
a retention strategy for
the streams the performance tool creates for a run. This
would typically avoid saturating the storage devices of your servers.
The default values are 20 GB for the maximum size of a stream and
500 MB for each segment files that composes a stream. You can change
these values with the --max-length-bytes and --stream-max-segment-size-bytes options:
java -jar stream-perf-test.jar --max-length-bytes 10gb \
--stream-max-segment-size-bytes 250mb
Both options accept units (kb, mb, gb, tb), as well as no unit to
specify a number of bytes.
It is also possible to use the time-based retention strategy with the --max-age option.
This can be less predictable than --max-length-bytes in the context of performance tests though.
The following command shows how to set the maximum age of segments to 5 minutes with
a maximum segment size of 250 MB:
java -jar stream-perf-test.jar --max-age PT5M \
--stream-max-segment-size-bytes 250mb
The --max-age option uses the
ISO 8601 duration format.
Consumers start by default at the very end of a stream (offset next).
It is possible to specify an offset
to start from with the --offset option,
if you have existing streams, and you want to consume from them at a specific offset.
The following command sets the consumer to start consuming at the beginning of
a stream:
java -jar stream-perf-test.jar --offset first
The accepted values for --offset are first, last, next (the default),
an unsigned long for a given offset, and an ISO 8601 formatted timestamp
(eg. 2020-06-03T07:45:54Z).
A consumer can track the point it has reached
in a stream to be able to restart where it left off in a new incarnation.
The performance tool has the --store-every option to tell consumers to store
the offset every x messages to be able to measure the impact of offset tracking
in terms of throughput and storage. This feature is disabled by default.
The following command shows how to store the offset every 100,000 messages:
java -jar stream-perf-test.jar --store-every 100000
When using --store-every (see above) for offset tracking,
the performance tool uses a default name using the pattern {stream-name}-{consumer-number}.
So the default name of a single tracking consumer consuming from stream will be stream-1.
The consumer names pattern can be set with the --consumer-names option, which uses
the Java printf-style format string.
The stream name and the consumer number are injected as arguments, in this order.
The following table illustrates some examples for the --consumer-names option
for a s1 stream and a second consumer:
| Option | Computed Name | Details |
|---|---|---|
|
|
Default pattern. |
|
|
|
|
|
The argument indexes ( |
|
|
Random UUID that changes for every run. |
Note you can use --consumer-names uuid to change the consumer names for every run. This
can be useful when you want to use tracking consumers in different runs but you want to
force the offset they start consuming from. With consumer names that do not change between runs,
tracking consumers would ignore the specified offset and would start where they left off
(this is the purpose of offset tracking).
You can use the --producer-names option to set the producer names pattern and therefore
enable message deduplication (using the default
publishing sequence starting at 0 and incremented for each message).
The same naming options apply as above in consumer names with the only
difference that the default pattern is empty (i.e. no deduplication).
Here is an example of the usage of the --producer-names option:
java -jar stream-perf-test.jar --producer-names %s-%d
The run will start one producer and will use the stream-1 producer reference (default stream is stream and the number of the producer is 1.)
The tool can expose some runtime information on HTTP. The default port is 8080. The following options are available:
-
--monitoring: add athreaddumpendpoint to display a thread dump of the process. This can be useful to inspect threads if the tool seems blocked. -
--prometheus: add ametricsendpoint to expose metrics using the Prometheus format. The endpoint can then be declared in a Prometheus instance to scrape the metrics. -
--monitoring-port: set the port to use for the web server.
The performance tool binary uses Logback with an internal configuration file.
The default log level is warn with a console appender.
It is possible to define loggers directly from the command line, this is useful for quick debugging.
Use the rabbitmq.streamperftest.loggers system property with name=level pairs, e.g.:
java -Drabbitmq.streamperftest.loggers=com.rabbitmq.stream=debug -jar stream-perf-test.jar
It is possible to define several loggers by separating them with commas, e.g. -Drabbitmq.streamperftest.loggers=com.rabbitmq.stream=debug,com.rabbitmq.stream.perf=info.
It is also possible to use an environment variable:
export RABBITMQ_STREAM_PERF_TEST_LOGGERS=com.rabbitmq.stream=debug
The system property takes precedence over the environment variable.
Use the environment variable with the Docker image:
docker run -it --rm --network host \
--env RABBITMQ_STREAM_PERF_TEST_LOGGERS=com.rabbitmq.stream=debug \
pivotalrabbitmq/stream-perf-test