rabbitmq性能测试PerfTest吞吐率最全评测

用了几年了,还是四五年前做的性能测试。目前遇到性能瓶颈,为了找出真因,不得不测试下,得到基准线,再根据测试基准线寻求优化。

测试结果:

  • direct 类型 比 topic 吞吐率快2倍左右;
  • 自动ack 比 手动ack 吞吐率快2倍左右;
  • 短期消息持久化对吞吐率无明显影响;

rabbitmq使用建议:无额外需求,建议采用direct类型,且自动ACK;

备注:消息量超过2KB后,吞吐较低的原因目前来看,大部分因素的网卡流量问题,4*2KB队列,网卡流量已达到400Mb/S,与峰值几乎一致。其CPU与内存用量未达顶峰。

一、工具准备

服务器环境:Docker Rabbitmq,垃圾硬件,CPU:4U,RAM:6092MB,硬盘:HDD

客户端环境:Windows11

工具:PerfTest (rabbitmq官方推荐)

下载:https://www.rabbitmq.com/java-tools.html#using-perf-test

二、工具使用

博主帮助:

命令:runjava com.rabbitmq.perf.PerfTest
参数:

-x:生产者计数
-y:消费者计数
-h:mq服务地址
-u:队列名
-a:自动ack回复
--id:测试id
-f persistent:消息持久化

原生帮助:

\rabbitmq-perf-test-2.17.0.RC4-bin\rabbitmq-perf-test-2.17.0.RC4\bin>runjava com.rabbitmq.perf.PerfTest --help
usage: <program>
 -?,--help                                   show usage
 -a,--autoack                                auto ack
 -A,--multi-ack-every <arg>                  multi ack every
 -ad,--auto-delete <arg>                     should the queue be
                                             auto-deleted, default is true
 -b,--heartbeat <arg>                        heartbeat interval
 -B,--body <arg>                             comma-separated list of files
                                             to use in message bodies
 -bc,--body-count <arg>                      number of pre-generated
                                             message bodies. Use with
                                             --json-body. Default is 100.
 -bfc,--body-field-count <arg>               number of pre-generated
                                             fields and values for body.
                                             Use with --json-body. Default
                                             is 1000.
 -c,--confirm <arg>                          max unconfirmed publishes
 -C,--pmessages <arg>                        producer message count
 -ca,--consumer-args <arg>                   consumer arguments as
                                             key/values pairs, separated
                                             by commas, e.g. x-priority=10
 -cri,--connection-recovery-interval <arg>   connection recovery interval
                                             in seconds. Default is 5
                                             seconds. Interval syntax,
                                             e.g. 30-60, is supported to
                                             specify an random interval
                                             between 2 values between each
                                             attempt.
 -ct,--confirm-timeout <arg>                 waiting timeout for
                                             unconfirmed publishes before
                                             failing (in seconds)
 -ctp,--consumers-thread-pools <arg>         number of thread pools to use
                                             for all consumers, default is
                                             to use a thread pool for each
                                             consumer
 -d,--id <arg>                               test ID
 -D,--cmessages <arg>                        consumer message count
 -dcr,--disable-connection-recovery          disable automatic connection
                                             recovery
 -e,--exchange <arg>                         exchange name
 -E,--exclusive                              use server-named exclusive
                                             queues. Such queues can only
                                             be used by their declaring
                                             connection!
 -env,--environment-variables                show usage with environment
                                             variables
 -ew,--exit-when <arg>                       exit when queue(s) empty or
                                             consumer(s) idle for 1
                                             second, valid values are
                                             empty or idle
 -f,--flag <arg>                             message flag(s), supported
                                             values: persistent and
                                             mandatory. Use the option
                                             several times to specify
                                             several values.
 -h,--uri <arg>                              connection URI
 -H,--uris <arg>                             connection URIs (separated by
                                             commas)
 -hst,--heartbeat-sender-threads <arg>       number of threads for
                                             producers and consumers
                                             heartbeat senders
 -i,--interval <arg>                         sampling interval in seconds
 -jb,--json-body                             generate a random JSON
                                             document for message body.
                                             Use with --size.
 -k,--routing-key <arg>                      routing key
 -K,--random-routing-key                     use random routing key per
                                             message
 -l,--legacy-metrics                         display legacy metrics
                                             (min/avg/max latency)
 -L,--consumer-latency <arg>                 consumer latency in
                                             microseconds
 -m,--ptxsize <arg>                          producer tx size
 -M,--framemax <arg>                         frame max
 -mh,--metrics-help                          show metrics usage
 -mp,--message-properties <arg>              message properties as
                                             key/value pairs, separated by
                                             commas, e.g. priority=5
 -ms,--use-millis                            should latency be collected
                                             in milliseconds, default is
                                             false. Set to true if
                                             producers are consumers run
                                             on different machines.
 -n,--ctxsize <arg>                          consumer tx size
 -na,--nack                                  nack messages, requeue them
                                             by default.
 -niot,--nio-threads <arg>                   number of NIO threads to use
 -niotp,--nio-thread-pool <arg>              size of NIO thread pool,
                                             should be slightly higher
                                             than number of NIO threads
 -o,--output-file <arg>                      output file for timing
                                             results
 -p,--predeclared                            allow use of predeclared
                                             objects
 -P,--publishing-interval <arg>              publishing interval in
                                             seconds (opposite of producer
                                             rate limit)
 -pi,--polling-interval <arg>                time to wait before polling
                                             with basic.get, in
                                             millisecond, default is 0.
 -po,--polling                               use basic.get to consume
                                             messages. Do not use this in
                                             real applications.
 -prsd,--producer-random-start-delay <arg>   max random delay in seconds
                                             to start producers
 -pst,--producer-scheduler-threads <arg>     number of threads to use when
                                             using --publishing-interval
 -q,--qos <arg>                              consumer prefetch count
 -Q,--global-qos <arg>                       channel prefetch count
 -qa,--queue-args <arg>                      queue arguments as key/value
                                             pairs, separated by commas,
                                             e.g. x-max-length=10
 -qf,--queue-file <arg>                      file to look up queue names
                                             from
 -qp,--queue-pattern <arg>                   queue name pattern for
                                             creating queues in sequence
 -qpf,--queue-pattern-from <arg>             queue name pattern range
                                             start (inclusive)
 -qpt,--queue-pattern-to <arg>               queue name pattern range end
                                             (inclusive)
 -qq,--quorum-queue                          create quorum queue(s)
 -r,--rate <arg>                             producer rate limit
 -R,--consumer-rate <arg>                    consumer rate limit
 -re,--requeue <arg>                         should nacked messages be
                                             requeued, default is true.
 -rkcs,--routing-key-cache-size <arg>        size of the random routing
                                             keys cache. See
                                             --random-routing-key.
 -S,--slow-start                             start consumers slowly (1 sec
                                             delay between each)
 -s,--size <arg>                             message size in bytes
 -sb,--skip-binding-queues                   don't bind queues to the
                                             exchange
 -se,--sasl-external                         use SASL EXTERNAL
                                             authentication, default is
                                             false. Set to true if using
                                             client certificate
                                             authentication with the
                                             rabbitmq_auth_mechanism_ssl
                                             plugin.
 -sni,--server-name-indication <arg>         server names for Server Name
                                             Indication TLS parameter,
                                             separated by commas
 -sst,--servers-startup-timeout <arg>        start timeout in seconds (in
                                             case the servers(s) is (are)
                                             not available when the run
                                             starts). Default is to fail
                                             immediately if the servers(s)
                                             is (are) not available.
 -st,--shutdown-timeout <arg>                shutdown timeout, default is
                                             5 seconds
 -sul,--servers-up-limit <arg>               number of available servers
                                             needed before starting the
                                             run. Used in conjunction with
                                             --servers-start-timeout.
                                             Default is deduced from --uri
                                             or --uris.
 -t,--type <arg>                             exchange type
 -T,--body-content-type <arg>                body content-type
 -u,--queue <arg>                            queue name
 -udsc,--use-default-ssl-context             use JVM default SSL context
 -v,--version                                print version information
 -vl,--variable-latency <arg>                variable consumer processing
                                             latency with
                                             [MICROSECONDS]:[DURATION]
                                             syntax, where [MICROSECONDS]
                                             integer >= 0 and [DURATION]
                                             integer > 0. Use the option
                                             several times to specify
                                             several values.
 -vr,--variable-rate <arg>                   variable publishing rate with
                                             [RATE]:[DURATION] syntax,
                                             where [RATE] integer >= 0 and
                                             [DURATION] integer > 0. Use
                                             the option several times to
                                             specify several values.
 -vs,--variable-size <arg>                   variable message size with
                                             [SIZE]:[DURATION] syntax,
                                             where [SIZE] integer > 0 and
                                             [DURATION] integer > 0. Use
                                             the option several times to
                                             specify several values.
 -x,--producers <arg>                        producer count
 -X,--producer-channel-count <arg>           channels per producer
 -y,--consumers <arg>                        consumer count
 -Y,--consumer-channel-count <arg>           channels per consumer
 -z,--time <arg>                             run duration in seconds
                                             (unlimited by default)

三、测试过程

2.1 测试1

条件:使用2个发布者和4个消费者:

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://user:xxx@192.168.6.5:5672" -u "perf-test1" -x 2 -y 4  -a --id "test 2"

结果:

条件:使用1个发布者和1个消费者:

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1-1" -a --id "test 1-1"

结果:

条件:一个生产者,一个消费者,消息大小从默认(12字节)更改为4 kB

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1-1-4096" -a --id "test 1-1 4096" -s 4096

结果:

条件:一个生产者,一个消费者,消息大小默认12字节,手动确认

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1-1"  --id "test 1-1"

结果:

条件:一个生产者,一个消费者, 消息大小默认12字节,手动确认,使用持久队列和持久消息

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1-1"  --id "test 1-1" -f persistent

结果:

条件:一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1-1"  --id "test 1-1" -f persistent -a

结果:

条件:二个生产者,四个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxxxx@192.168.6.5:5672"  -x 2 -y 4 -u "throughput-test-1-1"  --id "test 1-1" -f persistent -a

结果:

条件:二个生产者,四个消费者, 消息大小默认12字节 ,手动确认,使用持久队列和持久消息

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxx@192.168.6.5:5672"  -x 2 -y 4 -u "throughput-test-1-1"  --id "test 1-1" -f persistent

结果:

条件:二个生产者,四个消费者, 消息大小默认12字节 ,手动确认,使用持久队列和持久消息 ,Topic类型

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxx@192.168.6.5:5672"  -x 2 -y 4 -u "throughput-test-1-1"  --id "test 1-1" -f persistent -t "topic"

结果:

条件:二个生产者,四个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Topic类型

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxx:xxxxxx@192.168.6.5:5672"  -x 2 -y 4 -u "throughput-test-1-1"  --id "test 1-1" -f persistent -t "topic" -a

结果:

条件:一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Topic类型

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1-1"  --id "test 1-1" -f persistent -t "topic" -a

结果:

条件:四个生产者,八个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxx:xxxxxxx@192.168.6.5:5672"  -x 4 -y 8 -u "throughput-test-1-1"  --id "test 1-1" -f persistent -t "direct" -a

条件:四个工具一起测试* 一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://tianhengrd:gd6pYnYw3nlVrsoi@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-2-2"  --id "test 2-2" -f persistent -t "direct" -a

结果:

条件:六个工具一起测试* 一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型

结果:

条件:八个工具一起测试, * 一个生产者,一个消费者, 消息大小默认12字节 ,自动确认,使用持久队列和持久消息,Direct类型

结果:

资源消耗:

带宽消耗:

条件:四个工具一起测试* 一个生产者,一个消费者, 消息大小4KB ,自动确认,使用持久队列和持久消息,Direct类型

命令:

runjava com.rabbitmq.perf.PerfTest -h "amqp://xxxxx:xxxxxx@192.168.6.5:5672"  -x 1 -y 1 -u "throughput-test-1"  --id "test 1" -f persistent -t "direct" -a -s 4096

结果:

条件:四个工具一起测试* 一个生产者,一个消费者, 消息大2KB ,自动确认,使用持久队列和持久消息,Direct类型

结果: