系统"/>
Kubernetes平台部署Grafana Loki Promtail系统
部署结构图：
- Loki 是主服务，负责存储日志和处理查询
- promtail 是代理，负责收集日志并将其发送给 loki
- Grafana 用于 UI 展示
只要在应用程序服务器上安装promtail来收集日志然后发送给Loki存储，就可以在Grafana UI界面通过添加Loki为数据源进行日志查询（如果Loki服务器性能不够，可以部署多个Loki进行存储及查询）。作为一个日志系统不光只有查询分析日志的能力，还能对日志进行监控和报警。
文章目录
- 一、创建独立的命名空间
- 二、部署Grafana
- 2.1 编写grafana部署的配置文件
- 2.2 部署grafana
- 三、部署 Loki
- 3.1 编写Loki部署的配置文件
- 3.2 部署 Loki
- 四、部署 promtail
- 4.1 编写配置文件
- 4.2 部署 Promtail
- 五、检查部署结果
- 5.1 Kubernetes后台检查Pod部署状态
- 5.2 浏览器打开grafana界面如下，表示OK
一、创建独立的命名空间
为日志系统创建一个独立的命令空间
kubectl create ns logging
二、部署Grafana
2.1 编写grafana部署的配置文件
创建 grafana 目录，并在目录中创建 grafana-deploy.yaml 文件，内容如下：
apiVersion: apps/v1
kind: Deployment
metadata:name: grafanalabels:app: grafananamespace: logging
spec:replicas: 1selector:matchLabels:app: grafanatemplate:metadata:labels:app: grafanaspec:containers:- name: grafanaimage: grafana/grafana:8.4.7imagePullPolicy: IfNotPresentsecurityContext:runAsUser: 0env:- name: GF_AUTH_BASIC_ENABLEDvalue: "true"- name: GF_AUTH_ANONYMOUS_ENABLEDvalue: "false"
# resources:
# requests:
# cpu: 100m
# memory: 200Mi
# limits:
# cpu: '1'
# memory: 2GireadinessProbe:httpGet:path: /loginport: 3000volumeMounts:- name: storagemountPath: /var/lib/grafanavolumes:- name: storagehostPath:path: /hostpath/grafana
---
apiVersion: v1
kind: Service
metadata:name: grafanalabels:app: grafananamespace: logging
spec:type: NodePortports:- port: 3000targetPort: 3000nodePort: 30200selector:app: grafana
上述配置修改注意点：
（1）上述有两处namespace，此处需要使用步骤一中创建的命名空间，这里是logging
（2）挂载的目录需要在服务器上创建并且设置为足够的访问权限，比如这里：
mkdir -p /hostpath/grafana
chmod 777 -R /hostpath/grafana
（3）对外开发端口可以自行设置，这里比如 30200
2.2 部署grafana
进入grafana目录，然后执行部署命令
cd grafana
kubectl apply -f grafana-deploy.yaml
三、部署 Loki
3.1 编写Loki部署的配置文件
创建 loki文件夹，然后再 loki 文件夹中创建三个配置文件，分别是：loki-rbac.yaml、loki-configmap.yaml和loki-statefulset.yaml
loki-rbac.yaml文件内容如下：
apiVersion: v1
kind: Namespace
metadata:name: logging
---
apiVersion: v1
kind: ServiceAccount
metadata:name: lokinamespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:name: lokinamespace: logging
rules:
- apiGroups: ["extensions"]resources: ["podsecuritypolicies"]verbs: ["use"]resourceNames: [loki]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: lokinamespace: logging
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: loki
subjects:
- kind: ServiceAccountname: loki上述配置修改注意点：
（1）只需保证其中的namespace雨步骤一中设置的一致即可，比如都是logging时，此文件不需要修改
loki-configmap.yaml文件的内容如下：
apiVersion: v1
kind: ConfigMap
metadata:name: lokinamespace: logginglabels:app: loki
data:loki.yaml: |auth_enabled: falseingester:chunk_idle_period: 3mchunk_block_size: 262144chunk_retain_period: 1mmax_transfer_retries: 0lifecycler:ring:kvstore:store: inmemoryreplication_factor: 1limits_config:enforce_metric_name: falsereject_old_samples: truereject_old_samples_max_age: 168hschema_config:configs:- from: "2022-05-15"store: boltdb-shipperobject_store: filesystemschema: v11index:prefix: index_period: 24hserver:http_listen_port: 3100storage_config:boltdb_shipper:active_index_directory: /data/loki/boltdb-shipper-activecache_location: /data/loki/boltdb-shipper-cachecache_ttl: 24hshared_store: filesystemfilesystem:directory: /data/loki/chunkschunk_store_config:max_look_back_period: 0stable_manager:retention_deletes_enabled: trueretention_period: 48hcompactor:working_directory: /data/loki/boltdb-shipper-compactorshared_store: filesystem上述配置修改注意点：
（1）namespace同样需要和步骤一中的设置保持一致
loki-statefulset.yaml文件的内容如下：
apiVersion: v1
kind: Service
metadata:name: lokinamespace: logginglabels:app: loki
spec:type: NodePortports:- port: 3100protocol: TCPname: http-metricstargetPort: http-metricsnodePort: 30201selector:app: loki
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: lokinamespace: logginglabels:app: loki
spec:podManagementPolicy: OrderedReadyreplicas: 1selector:matchLabels:app: lokiserviceName: lokiupdateStrategy:type: RollingUpdatetemplate:metadata:labels:app: lokispec:serviceAccountName: lokiinitContainers:- name: chmod-dataimage: busybox:1.28.4imagePullPolicy: IfNotPresentcommand: ["chmod","-R","777","/loki/data"]volumeMounts:- name: storagemountPath: /loki/datacontainers:- name: lokiimage: grafana/loki:2.3.0imagePullPolicy: IfNotPresentargs:- -config.file=/etc/loki/loki.yamlvolumeMounts:- name: configmountPath: /etc/loki- name: storagemountPath: /dataports:- name: http-metricscontainerPort: 3100protocol: TCPlivenessProbe:httpGet: path: /readyport: http-metricsscheme: HTTPinitialDelaySeconds: 45readinessProbe:httpGet: path: /readyport: http-metricsscheme: HTTPinitialDelaySeconds: 45securityContext:readOnlyRootFilesystem: trueterminationGracePeriodSeconds: 4800volumes:- name: configconfigMap:name: loki- name: storagehostPath:path: /app/loki上述配置修改注意点：
（1）同样，namespace需要和步骤一中设置为一致
（2）设置Loki对外开放的端口，比如这里设置为30201
3.2 部署 Loki
在loki目录中执行如下命令进行部署
cd loki
kubectl apply -f .
四、部署 promtail
4.1 编写配置文件
首先创建 promtail 文件夹，然后再文件夹中创建 promtail-rbac.yaml、promtail-configmap.yaml和promtail-daemonset.yaml三个文件
promtail-rbac.yaml 文件内容如下，同样，这里不需要修改什么配置，只需要保证namespace和步骤一中创建的一致即可。
apiVersion: v1
kind: ServiceAccount
metadata:name: loki-promtaillabels:app: promtailnamespace: logging
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:labels:app: promtailname: promtail-clusterrolenamespace: logging
rules:
- apiGroups: [""]resources: ["nodes","nodes/proxy","services","endpoints","pods"]verbs: ["get", "watch", "list"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: promtail-clusterrolebindinglabels:app: promtailnamespace: logging
subjects:- kind: ServiceAccountname: loki-promtailnamespace: logging
roleRef:kind: ClusterRolename: promtail-clusterroleapiGroup: rbac.authorization.k8s.iopromtail-configmap.yaml 文件内容如下，此文件同样只需要保证namespace与步骤一中一致即可，不需要做其他修改
apiVersion: v1
kind: ConfigMap
metadata:name: loki-promtailnamespace: logginglabels:app: promtail
data:promtail.yaml: |client:backoff_config:max_period: 5m max_retries: 10min_period: 500msbatchsize: 1048576batchwait: 1sexternal_labels: {}timeout: 10spositions:filename: /run/promtail/positions.yamlserver:http_listen_port: 3101target_config:sync_period: 10sscrape_configs:- job_name: kubernetes-pods-namepipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- source_labels:- __meta_kubernetes_pod_label_nametarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-apppipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: .+source_labels:- __meta_kubernetes_pod_label_name- source_labels:- __meta_kubernetes_pod_label_apptarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-direct-controllerspipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: .+separator: ''source_labels:- __meta_kubernetes_pod_label_name- __meta_kubernetes_pod_label_app- action: dropregex: '[0-9a-z-.]+-[0-9a-f]{8,10}'source_labels:- __meta_kubernetes_pod_controller_name- source_labels:- __meta_kubernetes_pod_controller_nametarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-indirect-controllerpipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: .+separator: ''source_labels:- __meta_kubernetes_pod_label_name- __meta_kubernetes_pod_label_app- action: keepregex: '[0-9a-z-.]+-[0-9a-f]{8,10}'source_labels:- __meta_kubernetes_pod_controller_name- action: replaceregex: '([0-9a-z-.]+)-[0-9a-f]{8,10}'source_labels:- __meta_kubernetes_pod_controller_nametarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__- job_name: kubernetes-pods-staticpipeline_stages:- docker: {}kubernetes_sd_configs:- role: podrelabel_configs:- action: dropregex: ''source_labels:- __meta_kubernetes_pod_annotation_kubernetes_io_config_mirror- action: replacesource_labels:- __meta_kubernetes_pod_label_componenttarget_label: __service__- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: dropregex: ''source_labels:- __service__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __service__target_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_annotation_kubernetes_io_config_mirror- __meta_kubernetes_pod_container_nametarget_label: __path__promtail-daemonset.yaml 配置文件的内容如下：
apiVersion: apps/v1
kind: DaemonSet
metadata:name: loki-promtailnamespace: logginglabels:app: promtail
spec:selector:matchLabels:app: promtailupdateStrategy:rollingUpdate:maxUnavailable: 1type: RollingUpdatetemplate:metadata:labels:app: promtailspec:serviceAccountName: loki-promtailcontainers:- name: promtailimage: grafana/promtail:2.3.0imagePullPolicy: IfNotPresentargs: - -config.file=/etc/promtail/promtail.yaml- -client.url=http://192.168.16.40:30201/loki/api/v1/pushenv: - name: HOSTNAMEvalueFrom: fieldRef: apiVersion: v1fieldPath: spec.nodeNamevolumeMounts:- mountPath: /etc/promtailname: config- mountPath: /run/promtailname: run- mountPath: /var/lib/docker/containersname: dockerreadOnly: true- mountPath: /var/log/podsname: podsreadOnly: trueports:- containerPort: 3101name: http-metricsprotocol: TCPsecurityContext:readOnlyRootFilesystem: truerunAsGroup: 0runAsUser: 0readinessProbe:failureThreshold: 5httpGet:path: /readyport: http-metricsscheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1tolerations:- effect: NoSchedulekey: node-role.kubernetes.io/masteroperator: Existsvolumes:- name: configconfigMap:name: loki-promtail- name: runhostPath:path: /run/promtailtype: ""- name: dockerhostPath:path: /var/lib/docker/containers- name: podshostPath:path: /var/log/pods
上述配置修改注意点：
（1）同样namespace需要与步骤一中的保持一致
（2）如下位置需要修改为步骤三中配置的 Loki 的地址
4.2 部署 Promtail
进入 promtail 目录，然后执行部署命令即可
cd promtail
kubectl apply -f .
五、检查部署结果
5.1 Kubernetes后台检查Pod部署状态
执行如下命令即可查看pod状态，均为running时表示部署OK
[root@master ~]# kubectl get pod -n logging
NAME READY STATUS RESTARTS AGE
grafana-66496d957f-ngq5g 1/1 Running 0 21h
loki-0 1/1 Running 0 24h
loki-promtail-8vjd6 1/1 Running 0 20h
loki-promtail-gmr5f 1/1 Running 0 20h
loki-promtail-svwtn 1/1 Running 0 20h
loki-promtail-tnvr9 1/1 Running 0 20h
[root@master ~]#
5.2 浏览器打开grafana界面如下，表示OK
比如 192.168.16.40:30200，如下表示部署OK
Loki数据源
参考链接：
更多推荐
Kubernetes平台部署Grafana Loki Promtail系统
发布评论