用了几年了,还是四五年前做的性能测试。目前遇到性能瓶颈,为了找出真因,不得不测试下,得到基准线,再根据测试基准线寻求优化。
测试结果:
- direct 类型 比 topic 吞吐率快2倍左右;
- 自动ack 比 手动ack 吞吐率快2倍左右;
- 短期消息持久化对吞吐率无明显影响;
rabbitmq使用建议:无额外需求,建议采用direct类型,且自动ACK;
备注:消息量超过2KB后,吞吐较低的原因目前来看,大部分因素的网卡流量问题,4*2KB队列,网卡流量已达到400Mb/S,与峰值几乎一致。其CPU与内存用量未达顶峰。
一、工具准备
服务器环境:Docker Rabbitmq,垃圾硬件,CPU:4U,RAM:6092MB,硬盘:HDD
客户端环境:Windows11
工具:PerfTest (rabbitmq官方推荐)
下载:https://www.rabbitmq.com/java-tools.html#using-perf-test
二、工具使用
博主帮助:
命令:runjava com.rabbitmq.perf.PerfTest
参数:
-x:生产者计数
-y:消费者计数
-h:mq服务地址
-u:队列名
-a:自动ack回复
--id:测试id
-f persistent:消息持久化
原生帮助:
\rabbitmq-perf-test-2.17.0.RC4-bin\rabbitmq-perf-test-2.17.0.RC4\bin>runjava com.rabbitmq.perf.PerfTest --help
usage: <program>
-?,--help show usage
-a,--autoack auto ack
-A,--multi-ack-every <arg> multi ack every
-ad,--auto-delete <arg> should the queue be
auto-deleted, default is true
-b,--heartbeat <arg> heartbeat interval
-B,--body <arg> comma-separated list of files
to use in message bodies
-bc,--body-count <arg> number of pre-generated
message bodies. Use with
--json-body. Default is 100.
-bfc,--body-field-count <arg> number of pre-generated
fields and values for body.
Use with --json-body. Default
is 1000.
-c,--confirm <arg> max unconfirmed publishes
-C,--pmessages <arg> producer message count
-ca,--consumer-args <arg> consumer arguments as
key/values pairs, separated
by commas, e.g. x-priority=10
-cri,--connection-recovery-interval <arg> connection recovery interval
in seconds. Default is 5
seconds. Interval syntax,
e.g. 30-60, is supported to
specify an random interval
between 2 values between each
attempt.
-ct,--confirm-timeout <arg> waiting timeout for
unconfirmed publishes before
failing (in seconds)
-ctp,--consumers-thread-pools <arg> number of thread pools to use
for all consumers, default is
to use a thread pool for each
consumer
-d,--id <arg> test ID
-D,--cmessages <arg> consumer message count
-dcr,--disable-connection-recovery disable automatic connection
recovery
-e,--exchange <arg> exchange name
-E,--exclusive use server-named exclusive
queues. Such queues can only
be used by their declaring
connection!
-env,--environment-variables show usage with environment
variables
-ew,--exit-when <arg> exit when queue(s) empty or
consumer(s) idle for 1
second, valid values are
empty or idle
-f,--flag <arg> message flag(s), supported
values: persistent and
mandatory. Use the option
several times to specify
several values.
-h,--uri <arg> connection URI
-H,--uris <arg> connection URIs (separated by
commas)
-hst,--heartbeat-sender-threads <arg> number of threads for
producers and consumers
heartbeat senders
-i,--interval <arg> sampling interval in seconds
-jb,--json-body generate a random JSON
document for message body.
Use with --size.
-k,--routing-key <arg> routing key
-K,--random-routing-key use random routing key per
message
-l,--legacy-metrics display legacy metrics
(min/avg/max latency)
-L,--consumer-latency <arg> consumer latency in
microseconds
-m,--ptxsize <arg> producer tx size
-M,--framemax <arg> frame max
-mh,--metrics-help show metrics usage
-mp,--message-properties <arg> message properties as
key/value pairs, separated by
commas, e.g. priority=5
-ms,--use-millis should latency be collected
in milliseconds, default is
false. Set to true if
producers are consumers run
on different machines.
-n,--ctxsize <arg> consumer tx size
-na,--nack nack messages, requeue them
by default.
-niot,--nio-threads <arg> number of NIO threads to use
-niotp,--nio-thread-pool <arg> size of NIO thread pool,
should be slightly higher
than number of NIO threads
-o,--output-file <arg> output file for timing
results
-p,--predeclared allow use of predeclared
objects
-P,--publishing-interval <arg> publishing interval in
seconds (opposite of producer
rate limit)
-pi,--polling-interval <arg> time to wait before polling
with basic.get, in
millisecond, default is 0.
-po,--polling use basic.get to consume
messages. Do not use this in
real applications.
-prsd,--producer-random-start-delay <arg> max random delay in seconds
to start producers
-pst,--producer-scheduler-threads <arg> number of threads to use when
using --publishing-interval
-q,--qos <arg> consumer prefetch count
-Q,--global-qos <arg> channel prefetch count
-qa,--queue-args <arg> queue arguments as key/value
pairs, separated by commas,
e.g. x-max-length=10
-qf,--queue-file <arg> file to look up queue names
from
-qp,--queue-pattern <arg> queue name pattern for
creating queues in sequence
-qpf,--queue-pattern-from <arg> queue name pattern range
start (inclusive)
-qpt,--queue-pattern-to <arg> queue name pattern range end
(inclusive)
-qq,--quorum-queue create quorum queue(s)
-r,--rate <arg> producer rate limit
-R,--consumer-rate <arg> consumer rate limit
-re,--requeue <arg> should nacked messages be
requeued, default is true.
-rkcs,--routing-key-cache-size <arg> size of the random routing
keys cache. See
--random-routing-key.
-S,--slow-start start consumers slowly (1 sec
delay between each)
-s,--size <arg> message size in bytes
-sb,--skip-binding-queues don't bind queues to the
exchange
-se,--sasl-external use SASL EXTERNAL
authentication, default is
false. Set to true if using
client certificate
authentication with the
rabbitmq_auth_mechanism_ssl
plugin.
-sni,--server-name-indication <arg> server names for Server Name
Indication TLS parameter,
separated by commas
-sst,--servers-startup-timeout <arg> start timeout in seconds (in
case the servers(s) is (are)
not available when the run
starts). Default is to fail
immediately if the servers(s)
is (are) not available.
-st,--shutdown-timeout <arg> shutdown timeout, default is
5 seconds
-sul,--servers-up-limit <arg> number of available servers
needed before starting the
run. Used in conjunction with
--servers-start-timeout.
Default is deduced from --uri
or --uris.
-t,--type <arg> exchange type
-T,--body-content-type <arg> body content-type
-u,--queue <arg> queue name
-udsc,--use-default-ssl-context use JVM default SSL context
-v,--version print version information
-vl,--variable-latency <arg> variable consumer processing
latency with
[MICROSECONDS]:[DURATION]
syntax, where [MICROSECONDS]
integer >= 0 and [DURATION]
integer > 0. Use the option
several times to specify
several values.
-vr,--variable-rate <arg> variable publishing rate with
[RATE]:[DURATION] syntax,
where [RATE] integer >= 0 and
[DURATION] integer > 0. Use
the option several times to
specify several values.
-vs,--variable-size <arg> variable message size with
[SIZE]:[DURATION] syntax,
where [SIZE] integer > 0 and
[DURATION] integer > 0. Use
the option several times to
specify several values.
-x,--producers <arg> producer count
-X,--producer-channel-count <arg> channels per producer
-y,--consumers <arg> consumer count
-Y,--consumer-channel-count <arg> channels per consumer
-z,--time <arg> run duration in seconds
(unlimited by default)
三、测试过程
2.1 测试1
条件:使用2个发布者和4个消费者:
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://user:xxx@192.168.6.5:5672" -u "perf-test1" -x 2 -y 4 -a --id "test 2"
结果:
条件:使用1个发布者和1个消费者:
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1-1" -a --id "test 1-1"
结果:
条件:一个生产者,一个消费者,消息大小从默认(12字节)更改为4 kB
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1-1-4096" -a --id "test 1-1 4096" -s 4096
结果:
条件:一个生产者,一个消费者,消息大小默认12字节,手动确认
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1-1" --id "test 1-1"
结果:
条件:一个生产者,一个消费者, 消息大小默认12字节,手动确认,使用持久队列和持久消息
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1-1" --id "test 1-1" -f persistent
结果:
条件:一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1-1" --id "test 1-1" -f persistent -a
结果:
条件:二个生产者,四个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxxxx@192.168.6.5:5672" -x 2 -y 4 -u "throughput-test-1-1" --id "test 1-1" -f persistent -a
结果:
条件:二个生产者,四个消费者, 消息大小默认12字节 ,手动确认,使用持久队列和持久消息
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxx@192.168.6.5:5672" -x 2 -y 4 -u "throughput-test-1-1" --id "test 1-1" -f persistent
结果:
条件:二个生产者,四个消费者, 消息大小默认12字节 ,手动确认,使用持久队列和持久消息 ,Topic类型
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxx@192.168.6.5:5672" -x 2 -y 4 -u "throughput-test-1-1" --id "test 1-1" -f persistent -t "topic"
结果:
条件:二个生产者,四个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Topic类型
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxx@192.168.6.5:5672" -x 2 -y 4 -u "throughput-test-1-1" --id "test 1-1" -f persistent -t "topic" -a
结果:
条件:一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Topic类型
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1-1" --id "test 1-1" -f persistent -t "topic" -a
结果:
条件:四个生产者,八个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxx:xxxxxxx@192.168.6.5:5672" -x 4 -y 8 -u "throughput-test-1-1" --id "test 1-1" -f persistent -t "direct" -a
条件:四个工具一起测试* 一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://tianhengrd:gd6pYnYw3nlVrsoi@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-2-2" --id "test 2-2" -f persistent -t "direct" -a
结果:
条件:六个工具一起测试* 一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型
结果:
结果:
资源消耗:
条件:四个工具一起测试* 一个生产者,一个消费者, 消息大小4KB ,自动确认,使用持久队列和持久消息,Direct类型
命令:
runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxxx@192.168.6.5:5672" -x 1 -y 1 -u "throughput-test-1" --id "test 1" -f persistent -t "direct" -a -s 4096
结果:
条件:四个工具一起测试* 一个生产者,一个消费者, 消息大2KB ,自动确认,使用持久队列和持久消息,Direct类型
结果: