Pipe Plugins
The pipe plugin configurations contain a series of pipe configuration. Each pipe configuration has 5 parts, which are common_config, gatherer, processor and the sender.
common_config
Config |
Description |
pipe_name |
The unique collect space name. |
Gatherer
The gatherer has 2 roles, which are the receiver and fetcher.
Receiver Role
Config |
Description |
server_name |
The server name in the sharing pipe, which would be used in the receiver plugin. |
receiver |
The receiver configuration. Please read the doc to find all receiver plugins. |
queue |
The queue buffers the input telemetry data. Please read the doc to find all queue plugins. |
Fetcher Role
Config |
Description |
fetch_interval |
The time interval between two fetch operations. The time unit is millisecond. |
fetcher |
The fetcher configuration. Please read the doc to find all fetcher plugins. |
queue |
The queue buffers the input telemetry data. Please read the doc to find all queue plugins. |
processor
The filter configuration. Please read the doc to find all filter plugins.
sender
Config |
Description |
flush_time |
The time interval between two flush operations. And the time unit is millisecond. |
max_buffer_size |
The maximum buffer elements. |
min_flush_events |
The minimum flush elements. |
client_name |
The client name used in the forwarders of the sharing pipe. |
forwarders |
The forwarder plugin list. Please read the doc to find all forwarders plugins. |
fallbacker |
The fallbacker plugin. Please read the doc to find all fallbacker plugins. |
Example
pipes:
- common_config:
pipe_name: pipe1
gatherer:
server_name: "grpc-server"
receiver:
plugin_name: "grpc-native-log-receiver"
queue:
plugin_name: "mmap-queue"
segment_size: ${SATELLITE_MMAP_QUEUE_SIZE:524288}
max_in_mem_segments: ${SATELLITE_MMAP_QUEUE_MAX_IN_MEM_SEGMENTS:6}
queue_dir: "pipe1-log-grpc-receiver-queue"
processor:
filters:
sender:
fallbacker:
plugin_name: none-fallbacker
flush_time: ${SATELLITE_PIPE1_SENDER_FLUSH_TIME:1000}
max_buffer_size: ${SATELLITE_PIPE1_SENDER_MAX_BUFFER_SIZE:200}
min_flush_events: ${SATELLITE_PIPE1_SENDER_MIN_FLUSH_EVENTS:100}
client_name: kafka-client
forwarders:
- plugin_name: native-log-kafka-forwarder
topic: ${SATELLITE_NATIVELOG-TOPIC:log-topic}