Compiling Your Own Rules
A native image cannot generate classes at runtime, so this distro compiles every rule (OAL, MAL, LAL, TAL and the hierarchy matching rules) when it is built, and the image reads no rule file at boot. To run your own rules, keep them in your own repository, reference this repository from it (a git submodule or a checkout pinned to a commit), and build the image with your rule files compiled in. The result is a self-contained native image, or JVM distribution, carrying your rules: nothing about the rules is deployed or configured next to it.
Your rule files are a rules overlay: a directory laid out like OAP’s config/ with a rules-overlay.yaml that
declares which bundled rule files it adds, overrides and excludes.
Rules are compiled in, never configured
Every compiled rule file is active. Upstream SkyWalking picks rule files at boot from application.yml settings
and their environment variables. In this distro, the image fills those settings with every file it compiled, and the
rules can’t be selected, added or replaced at runtime:
| Module | Setting | Environment variable |
|---|---|---|
receiver-otel |
enabledOtelMetricsRules |
SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES |
agent-analyzer |
meterAnalyzerActiveFiles |
SW_METER_ANALYZER_ACTIVE_FILES |
log-analyzer |
malFiles |
SW_LOG_MAL_FILES |
log-analyzer |
lalFiles |
SW_LOG_LAL_FILES |
trace-analyzer |
talFiles |
SW_TRACE_TAL_FILES |
trace-analyzer |
nativeTalFiles |
SW_TRACE_NATIVE_TAL_FILES |
envoy-metric |
enabledEnvoyMetricsRules |
SW_ENVOY_METRIC_ENABLED_RULES |
receiver-telegraf |
activeFiles |
SW_RECEIVER_TELEGRAF_ACTIVE_FILES |
receiver-zabbix |
activeFiles |
SW_RECEIVER_ZABBIX_ACTIVE_FILES |
- The image sets them after reading the configuration, so no value given for them takes effect.
- Giving one anyway fails the boot rather than being silently ignored, wherever it is given: in
application.yml(under any provider, selected or not), by its environment variable, or by system property under either name (-DSW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES=…,-Dreceiver-otel.<provider>.enabledOtelMetricsRules=…). A deployment carried over from upstream, such as a Helm chart that still sets one, finds out at once. - A rule file where upstream would look it up fails the boot too: under
config/(oal/,otel-rules/,meter-analyzer-config/,log-mal-rules/,envoy-metrics-rules/,telegraf-rules/,zabbix-rules/,lal/,tal/,hierarchy-definition.yml,layer-extensions.yml), in the working directory, or on the JVM distribution’s class path.
Both fail rather than being ignored, so a deployment can’t believe it changed the rules when it didn’t. The image also
sets the rule directories (lalPath, malPath, talPath) to the compiled catalogs and drops the log analyzer’s
meterConfigs cache, whatever is given for them. Everything else in application.yml, alarm-settings.yml and the
other runtime configuration files stays configurable, including whether a module runs (for example SW_RECEIVER_ZABBIX=-): a disabled module’s rules stay compiled and unused.
The image built from this repository without an overlay compiles every rule file upstream bundles.
Supported rule files
An overlay may add, override or exclude these files, the ones upstream reads rules from:
| Path | Engine |
|---|---|
oal/*.oal |
OAL |
otel-rules/** |
MAL, OpenTelemetry metrics |
meter-analyzer-config/** |
MAL, SkyWalking meter protocol |
log-mal-rules/** |
MAL, metrics from logs |
envoy-metrics-rules/** |
MAL, Envoy metrics service |
telegraf-rules/** |
MAL, Telegraf |
zabbix-rules/* |
MAL, Zabbix |
lal/* |
LAL |
tal/* |
TAL, OTLP spans and SkyWalking agent segments |
hierarchy-definition.yml |
Service hierarchy |
layer-extensions.yml |
Layer extensions, registered before any rule compiles |
Rule files are YAML, as upstream documents them: .yaml or .yml, and .oal for an OAL script. As upstream reads
them, MAL rule files sit in their directory or below it (**), the others in the directory itself. A path uses ASCII
letters, digits, ., _ and - only, at most 64 characters a segment and 128 in all, and a MAL file one extension:
the image carries the names into the rule settings lists, generated class files and the rule browser’s URLs. A
case-insensitive disk takes names differing by case only for one, so a directory differing from a bundled one by case
only (otel-rules/K8S/) fails the build, even when every bundled file under it is excluded; use the bundled spelling.
Two compiled rule files may not differ by case or extension only either, but excluding lal/default.yaml lets the
overlay add lal/DEFAULT.yaml.
Other files in the overlay directory, such as a README at its root or a file of another extension in a rule directory, are not rule files: they are ignored, the build log lists them, and a manifest entry naming one fails the build. Hidden files are skipped.
rules-overlay.yaml
The manifest at the overlay root declares every change. Each entry is a path or a glob relative to the overlay root:
add: # rule files upstream doesn't have
- otel-rules/fleet/*.yaml
- oal/fleet.oal
- lal/fleet.yaml
- tal/fleet.yaml
override: # replace a bundled file as a whole
- otel-rules/vm.yaml
- layer-extensions.yml
exclude: # bundled files not to compile, so not active
- oal/tcp.oal
- "otel-rules/**"
excludematches bundled files only. A file the overlay overrides stays compiled, sootel-rules/**plus theoverrideofotel-rules/vm.yamlabove leaves just that file and the addedotel-rules/fleet/files. This is how to compile a few of a directory’s files: exclude the directory and override or add the ones you run.- The build fails when:
- a rule file in the overlay is not declared, or is declared both as add and as override, or an entry is listed twice;
- an
addnames a file upstream has, or anoverrideone it doesn’t; - an
addoroverrideentry matches no overlay rule file, or anexcludeentry matches no bundled file the overlay doesn’t provide, so a manifest can’t go stale silently after an upstream update; - two compiled rule files differ only by extension or case (
foo.yaml,Foo.yml); - a rule file is nested in a directory read flat (
lal/sub/x.yaml), or its path breaks the naming rule above; hierarchy-definition.ymlis excluded. To turn the hierarchy off, override it withhierarchy: {},auto-matching-rules: {}andlayer-levels: {};- a rule doesn’t compile, or would fail upstream’s boot (such as an unknown input type or function, or conflicting metric shapes);
- a LAL rule starts on the same line as another rule (the image finds a rule’s class by its file and line), or, of
layer: autoor a known layer, declaresfallback: trueoff alayer: autorule or shares its name with another rule of its layer. A rule of an unknown layer is compiled and skipped at boot, as upstream skips it; - a TAL file mixes
sw-nativerules with rules of another input, or twosw-nativerules share a name: thesw-nativefiles becomenativeTalFiles, the otherstalFiles; layer-levelsnames a layer neither built in nor inlayer-extensions.yml; the hierarchy maps a layer to something other than its lower layers, relates a layer whose level is missing or isn’t an integer, puts an upper layer’s level not above a lower layer’s, or maps a lower layer to anything but a matching rule’s name (or nothing, to relate the two without automatic matching); or a matching rule is named with anything but letters, digits,.,_and-.
- Otherwise a rule file is accepted as upstream SkyWalking accepts it. What depends on the modules a deployment runs, such as a layer only another receiver’s rule file declares or a function of a module that isn’t loaded, is checked when the OAP boots, as upstream checks it.
- The same metric name in several files is allowed, as upstream allows it: the bundled OpenTelemetry, Telegraf and Zabbix VM rules share their metrics.
OAL
- An excluded script isn’t compiled: none of its metrics exist. An alarm rule in
alarm-settings.ymlon one of them (the bundled ones usecore.oalmetrics such asservice_resp_time) needs a runtimealarm-settings.ymlwithout it;rules: {}turns the alarms off. - An added script gets a define of its own and may use the sources of
org.apache.skywalking.oap.server.core.source. A script that uses another package’s sources fails the build.
Building from your own repository
A typical layout, with this repository as a submodule pinned to a release or commit:
my-oap/
├── skywalking-graalvm-distro/ # this repository
├── rules/
│ ├── rules-overlay.yaml
│ ├── otel-rules/fleet/fleet-service.yaml
│ ├── lal/fleet.yaml
│ └── layer-extensions.yml
└── build.sh
Pass the overlay as an absolute path without spaces: make -C resolves a relative one against this repository,
not yours. Keep this repository’s checkout path to letters, digits, /, ., _ and - as well: the rule loaders
find the rule directories through the class path, which does not decode characters such as # or %.
DISTRO=$PWD/skywalking-graalvm-distro
RULES=$PWD/rules
M2=$PWD/.m2/repository # optional: a Maven repository of your own, kept apart from ~/.m2
# Once per distro commit: install the bundled SkyWalking it builds on.
make -C "$DISTRO" init-skywalking M2_REPO="$M2"
# Compile the rules and run the tests that check them, without the native image: the check for CI.
make -C "$DISTRO" verify-rules RULES_OVERLAY="$RULES" M2_REPO="$M2"
# The native image, on Linux, or cross-compiled for Linux on macOS (in a GraalVM container).
make -C "$DISTRO" native-image RULES_OVERLAY="$RULES" M2_REPO="$M2"
make -C "$DISTRO" native-image-macos RULES_OVERLAY="$RULES" M2_REPO="$M2"
# The Docker image, from that native image.
make -C "$DISTRO" docker-native
RULES_OVERLAYtakes effect in every target that compiles:compile,build-distro(the JVM distribution),native-image,native-image-macosandverify-rules.verify-rulescompiles the rules and runs the precompiler’s tests and the tests over the compiled rules. It runs every overlay MAL rule through its compiled class and through a fresh compile on generated inputs, and checks that every overlay LAL rule has its class.M2_REPO=<dir>builds with that Maven repository instead of~/.m2/repository, andnative-image-macosmounts it into its container. A build installs its compiled rules into the Maven repository it uses, so builds running at the same time with different overlays each need their own.- Each build restages the rules from scratch: a build without
RULES_OVERLAYleaves nothing of an earlier overlay. - To track a new release of this repository, move the submodule and rebuild. A manifest entry that no longer matches the bundled files fails the build and names the entry.
- This repository’s CI does the same with
test/e2e/cases/rules-overlay/rules/, copied out of the checkout: a MAL, a LAL and a TAL rule under a layer of its own and an OAL script over the agent services the kept native TAL rules find, most bundled rules excluded and a few kept. Therules-overlaye2e case then checks the native image runs its rules and the kept ones, and has none of the excluded.
What the image holds
Both distributions carry compiled-rules/ at their root:
compiled-rules/
├── rules-overlay.yaml # the manifest, when built with an overlay
├── BUILD-INFO # upstream commit and the overlay digest ("none" without one)
├── oal/ otel-rules/ meter-analyzer-config/ log-mal-rules/ envoy-metrics-rules/ telegraf-rules/ zabbix-rules/
├── lal/ tal/
├── hierarchy-definition.yml
└── layer-extensions.yml
It holds copies of the rule files the image was compiled from, after add, override and exclude (a file holding no
rule, such as the bundled log-mal-rules/placeholder.yaml, is copied as it is). It is a record only:
the image runs the classes compiled from these files and never reads the folder.
At boot, the OAP logs Every compiled rule file is active: {…} with the files each rule-selecting setting got.