Observe service mesh through ALS

Envoy ALS(access log service) provides full logs about RPC routed, including HTTP and TCP.

The solution is initialized and firstly implemented by Sheng Wu, Hongtao Gao, Lizan Zhou, and Dhi Aurrahman at 17 May. 2019, and presented on KubeCon China 2019. Here is the recorded Video.

SkyWalking is the first open source project introducing this ALS based solution to the world. This provides a new way with very low payload to service mesh, but the same observability.

You need three steps to open ALS.

  1. Open envoyAccessLogService in istio by enabling envoyAccessLogService in ProxyConfig.

    Upper istio 1.6.0, if istio installed by demo profile, you can open ALS ues command:

    istioctl manifest apply --set profile=demo --set meshConfig.defaultConfig.envoyAccessLogService.address=skywalking-oap.skywalking.svc:11800 --set meshConfig.enableEnvoyAccessLogService=true
    

    Note: SkyWalking OAP service is at skywalking namespace, and the port of gRPC service is 11800

  2. (Default is ACTIVATED) Activate SkyWalking envoy receiver.

  3. Active ALS k8s-mesh analysis, set system env variable SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh

envoy-metric:
  selector: ${SW_ENVOY_METRIC:default}
  default:
    acceptMetricsService: ${SW_ENVOY_METRIC_SERVICE:true}
    alsHTTPAnalysis: ${SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS:""} # Setting the system env variable would override this. 

For multiple values,please use , symbol to concatenate.

Here’s an example to deploy SkyWalking by Helm chart.

istioctl install --set profile=demo --set meshConfig.defaultConfig.envoyAccessLogService.address=skywalking-oap.istio-system:11800 --set meshConfig.enableEnvoyAccessLogService=true

git clone https://github.com/apache/skywalking-kubernetes.git
cd skywalking-kubernetes/chart

helm repo add elastic https://helm.elastic.co

helm dep up skywalking

helm install 8.1.0 skywalking -n istio-system --set oap.env.SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh --set fullnameOverride=skywalking --set oap.envoy.als.enabled=true

Notice, only use this when Envoy is under Istio’s control, and they are in k8s environment. The OAP requires the read right to k8s API server for all pods IPs.

You can use kubectl -n istio-system logs -l app=skywalking | grep "K8sALSServiceMeshHTTPAnalysis" to ensure OAP ALS k8s-mesh analysis has been active.