Support RocketMQ Monitoring

Motivation

RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications. Now that Skywalking can monitor OpenTelemetry metrics, I want to add RocketMQ monitoring via the OpenTelemetry Collector, which fetches metrics from the RocketMQ Exporter

Architecture Graph

There is no significant architecture-level change.

Proposed Changes

rocketmq-exporter collects metrics from RocketMQ and transport the data to OpenTelemetry collector, using SkyWalking openTelemetry receiver to receive these metrics。 Provide cluster, broker, and topic dimensions monitoring.

RocketMQ Cluster Supported Metrics

Monitoring Panel Unit Metric Name Description Data Source
Messages Produced Today Count meter_rocketmq_cluster_messages_produced_today The number of cluster messages produced today. RocketMQ Exporter
Messages Consumed Today Count meter_rocketmq_cluster_messages_consumed_today The number of cluster messages consumed today. RocketMQ Exporter
Total Producer Tps Msg/sec meter_rocketmq_cluster_total_producer_tps The number of messages produced per second. RocketMQ Exporter
Total Consume Tps Msg/sec meter_rocketmq_cluster_total_consumer_tps The number of messages consumed per second. RocketMQ Exporter
Producer Message Size Bytes/sec meter_rocketmq_cluster_producer_message_size The max size of a message produced per second. RocketMQ Exporter
Consumer Message Size Bytes/sec meter_rocketmq_cluster_consumer_message_size The max size of the consumed message per second. RocketMQ Exporter
Messages Produced Until Yesterday Count meter_rocketmq_cluster_messages_produced_until_yesterday The total number of messages put until 12 o’clock last night. RocketMQ Exporter
Messages Consumed Until Yesterday Count meter_rocketmq_cluster_messages_consumed_until_yesterday The total number of messages read until 12 o’clock last night. RocketMQ Exporter
Max Consumer Latency ms meter_rocketmq_cluster_max_consumer_latency The max number of consumer latency. RocketMQ Exporter
Max CommitLog Disk Ratio % meter_rocketmq_cluster_max_commitLog_disk_ratio The max utilization ratio of the commit log disk. RocketMQ Exporter
CommitLog Disk Ratio % meter_rocketmq_cluster_commitLog_disk_ratio The utilization ratio of the commit log disk per broker IP. RocketMQ Exporter
Pull ThreadPool Queue Head Wait Time ms meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time The wait time in milliseconds for pulling threadPool queue per broker IP. RocketMQ Exporter
Send ThreadPool Queue Head Wait Time ms meter_rocketmq_cluster_send_threadPool_queue_head_wait_time The wait time in milliseconds for sending threadPool queue per broker IP. RocketMQ Exporter
Topic Count Count meter_rocketmq_cluster_topic_count The number of topics that received messages from the producer. RocketMQ Exporter
Broker Count Count meter_rocketmq_cluster_broker_count The number of brokers that received messages from the producer. RocketMQ Exporter

RocketMQ Broker Supported Metrics

Monitoring Panel Unit Metric Name Description Data Source
Produce TPS Msg/sec meter_rocketmq_broker_produce_tps The number of broker produces messages per second. RocketMQ Exporter
Consume QPS Msg/sec meter_rocketmq_broker_consume_qps The number of broker consumes messages per second. RocketMQ Exporter
Producer Message Size Bytes/sec meter_rocketmq_broker_producer_message_size The max size of the messages produced per second. RocketMQ Exporter
Consumer Message Size Bytes/sec meter_rocketmq_broker_consumer_message_size The max size of the messages consumed per second. RocketMQ Exporter

RocketMQ Topic Supported Metrics

Monitoring Panel Unit Metric Name Description Data Source
Max Producer Message Size Byte meter_rocketmq_topic_max_producer_message_size The maximum number of messages produced. RocketMQ Exporter
Max Consumer Message Size Byte meter_rocketmq_topic_max_consumer_message_size The maximum number of messages consumed. RocketMQ Exporter
Consumer Latency ms meter_rocketmq_topic_consumer_latency Consumption delay time of a consumer group. RocketMQ Exporter
Producer Tps Msg/sec meter_rocketmq_topic_producer_tps The number of messages produced per second. RocketMQ Exporter
Consumer Group Tps Msg/sec meter_rocketmq_topic_consumer_group_tps The number of messages consumed per second per consumer group. RocketMQ Exporter
Producer Offset Count meter_rocketmq_topic_producer_offset The max progress of a topic’s production message. RocketMQ Exporter
Consumer Group Offset Count meter_rocketmq_topic_consumer_group_offset The max progress of a topic’s consumption message per consumer group. RocketMQ Exporter
Producer Message Size Byte/sec meter_rocketmq_topic_producer_message_size The max size of messages produced per second. RocketMQ Exporter
Consumer Message Size Byte/sec meter_rocketmq_topic_consumer_message_size The max size of messages consumed per second. RocketMQ Exporter
Consumer Group_Count Count meter_rocketmq_topic_consumer_group_count The number of consumer groups. RocketMQ Exporter
Broker Count Count meter_rocketmq_topic_broker_count The number of topics that received messages from the producer. RocketMQ Exporter

Imported Dependencies libs and their licenses.

No new dependency.

Compatibility

no breaking changes.

General usage docs

This feature is out of the box.