SkyWalking
Support RocketMQ Monitoring
Motivation
RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications. Now that Skywalking can monitor OpenTelemetry metrics, I want to add RocketMQ monitoring via the OpenTelemetry Collector, which fetches metrics from the RocketMQ Exporter
Architecture Graph
There is no significant architecture-level change.
Proposed Changes
rocketmq-exporter collects metrics from RocketMQ and transport the data to OpenTelemetry collector, using SkyWalking openTelemetry receiver to receive these metrics。
Provide cluster, broker, and topic dimensions monitoring.
RocketMQ Cluster Supported Metrics
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|---|---|---|---|---|
| Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter |
| Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter |
| Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter |
| Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o’clock last night. | RocketMQ Exporter |
| Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o’clock last night. | RocketMQ Exporter |
| Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter |
| Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter |
| CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter |
| Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter |
| Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter |
| Topic Count | Count | meter_rocketmq_cluster_topic_count | The number of topics that received messages from the producer. | RocketMQ Exporter |
| Broker Count | Count | meter_rocketmq_cluster_broker_count | The number of brokers that received messages from the producer. | RocketMQ Exporter |
RocketMQ Broker Supported Metrics
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|---|---|---|---|---|
| Produce TPS | Msg/sec | meter_rocketmq_broker_produce_tps | The number of broker produces messages per second. | RocketMQ Exporter |
| Consume QPS | Msg/sec | meter_rocketmq_broker_consume_qps | The number of broker consumes messages per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_broker_producer_message_size | The max size of the messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_broker_consumer_message_size | The max size of the messages consumed per second. | RocketMQ Exporter |
RocketMQ Topic Supported Metrics
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|---|---|---|---|---|
| Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter |
| Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter |
| Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter |
| Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter |
| Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic’s production message. | RocketMQ Exporter |
| Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic’s consumption message per consumer group. | RocketMQ Exporter |
| Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter |
| Consumer Group_Count | Count | meter_rocketmq_topic_consumer_group_count | The number of consumer groups. | RocketMQ Exporter |
| Broker Count | Count | meter_rocketmq_topic_broker_count | The number of topics that received messages from the producer. | RocketMQ Exporter |
Imported Dependencies libs and their licenses.
No new dependency.
Compatibility
no breaking changes.
General usage docs
This feature is out of the box.