prometheus☞04redis监控

前言

Awesome Prometheus alerts | Collection of alerting rules (grep.to)

oliver006/redis_exporter: Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, 5.x and 6.x (github.com)

oliver006/redis_exporter (docker.com)

安装

docker run -d --net promsnet --name redis_exporter -p 9121:9121 oliver006/redis_exporter -web.listen-address 0.0.0.0:9121 --redis.addr=

–redis.addr= 后面不用加任何地址，就是这么写的，如果不加这个选项，则 redis_exporter 会自动添加 redis://localhost:6379，如果 redis_exporter 部署的宿主机无法访问 redis://localhost:6379，则会导致 proms 报一个实例 down 掉。

具体redis_exporter需要监控的redis地址，在prometheus中配置即可。

prometheus 主配置

scrape_configs:
  - job_name: 'redis'
    metrics_path: /scrape
    relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: <redis_exporter地址>:9121
    static_configs:
    - targets:
      - redis://<被监控的redis地址01>:6379
      - redis://<被监控的redis地址02>:6379

  - job_name: 'redis_exporter'
    static_configs:
      - targets:
        - redis_exporter:9121

prometheus 告警配置

groups:
  - name: redis-alert
    rules:
    - alert: RedisDown
      expr: redis_up == 0
      for: 0m
      labels:
        severity: critical
      annotations:
        summary: Redis down (instance {{ $labels.instance }})
        description: "Redis instance is down\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

    - alert: RedisOutOfSystemMemory
      expr: redis_memory_used_bytes / redis_total_system_memory_bytes * 100 > 90
      for: 2m
      labels:
        severity: warning
      annotations:
        summary: Redis out of system memory (instance {{ $labels.instance }})
        description: "Redis is running out of system memory (> 90%)\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

    - alert: RedisOutOfConfiguredMaxmemory
      expr: redis_memory_used_bytes / redis_memory_max_bytes * 100 > 90
      for: 2m
      labels:
        severity: warning
      annotations:
        summary: Redis out of configured maxmemory (instance {{ $labels.instance }})
        description: "Redis is running out of configured maxmemory (> 90%)\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

    - alert: RedisTooManyConnections
      expr: redis_connected_clients > 100
      for: 2m
      labels:
        severity: warning
      annotations:
        summary: Redis too many connections (instance {{ $labels.instance }})
        description: "Redis instance has too many connections\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

    - alert: RedisNotEnoughConnections
      expr: redis_connected_clients < 5
      for: 2m
      labels:
        severity: warning
      annotations:
        summary: Redis not enough connections (instance {{ $labels.instance }})
        description: "Redis instance should have more connections (> 5)\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

    - alert: RedisRejectedConnections
      expr: increase(redis_rejected_connections_total[1m]) > 0
      for: 0m
      labels:
        severity: critical
      annotations:
        summary: Redis rejected connections (instance {{ $labels.instance }})
        description: "Some connections to Redis has been rejected\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

grafana

https://grafana.com/dashboards/763

redis_exporter 作为边车容器

# prome-redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: monitor
spec:
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:4
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 6379
      - name: redis-exporter
        image: oliver006/redis_exporter:latest
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 9121
---
kind: Service
apiVersion: v1
metadata:
  name: redis
  namespace: monitor
spec:
  selector:
    app: redis
  ports:
  - name: redis
    port: 6379
    targetPort: 6379
  - name: prom
    port: 9121
    targetPort: 9121