====== Prometheus monitoring ====== * Grafana prometheus dashboard: https://grafana.com/grafana/dashboards/1860 * Grafana prometheus percona mysqld dashboard: https://grafana.com/grafana/dashboards/7362 * Prometheus rules collection: https://awesome-prometheus-alerts.grep.to/rules.html ===== Agent node_exporter install ===== sudo useradd node_exporter -s /sbin/nologin mkdir -p /opt/prom-exporters wget https://github.com/prometheus/node_exporter/releases/download/v*/node_exporter-*.*-amd64.tar.gz tar xvfz node_exporter-*.*-amd64.tar.gz sudo mv node_exporter-*.*-amd64/node_exporter /opt/prom-exporters/ rm -rf node_exporter-*.*-amd64 Create a systemd service file ''/etc/systemd/system/node_exporter.service'': [Unit] Description=Node Exporter [Service] User=node_exporter EnvironmentFile=/etc/sysconfig/node_exporter ExecStart=/opt/prom-exporters/node_exporter $OPTIONS [Install] WantedBy=multi-user.target Create a sysconfig file: sudo mkdir -p /etc/sysconfig sudo echo "OPTIONS=\"--collector.textfile.directory /var/lib/node_exporter/textfile_collector\"" > /etc/sysconfig/node_exporter Reload systemd configuration and configure node_exporter to auto-start on system boot: sudo systemctl daemon-reload sudo systemctl enable node_exporter sudo systemctl start node_exporter Once the Node Exporter is installed, verify that metrics are being exported: curl http://localhost:9100/metrics ===== Agent mysqld_exporter install ===== Prepare MySQL user for agent: GRANT REPLICATION CLIENT, PROCESS ON *.* TO 'mysqld_exporter'@'localhost' identified by 'My_Secure_Password'; GRANT SELECT ON performance_schema.* TO 'mysqld_exporter'@'localhost'; FLUSH PRIVILEGES; Create a mysql password file ''/opt/prom-exporters/.mysqld_exporter.cnf'': [client] user=mysqld_exporter password=My_Secure_Password Change file owner and permission: chown prometheus. /opt/prom-exporters/.mysqld_exporter.cnf chmod 600 /opt/prom-exporters/.mysqld_exporter.cnf Install mysqld_exporter: wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.12.1/mysqld_exporter-0.12.1.linux-amd64.tar.gz tar zxvf mysqld_exporter-0.12.1.linux-amd64.tar.gz cd mysqld_exporter-0.12.1.linux-amd64 chown prometheus. mysqld_exporter mv mysqld_exporter /opt/prom-exporters/ cd .. rm -rf mysqld_exporter-0.12.1.linux-amd64* Create a systemd service file ''/etc/systemd/system/mysqld_exporter.service'': [Unit] Description=Prometheus MySQL Exporter After=network.target User=prometheus Group=prometheus [Service] Type=simple Restart=always ExecStart=/opt/prom-exporters/mysqld_exporter \ --config.my-cnf /opt/prom-exporters/.mysqld_exporter.cnf \ --collect.global_status \ --collect.info_schema.innodb_metrics \ --collect.auto_increment.columns \ --collect.info_schema.processlist \ --collect.binlog_size \ --collect.info_schema.tablestats \ --collect.global_variables \ --collect.info_schema.query_response_time \ --collect.info_schema.userstats \ --collect.info_schema.tables \ --collect.perf_schema.tablelocks \ --collect.perf_schema.file_events \ --collect.perf_schema.eventswaits \ --collect.perf_schema.indexiowaits \ --collect.perf_schema.tableiowaits \ --collect.slave_status \ --web.listen-address=0.0.0.0:9104 [Install] WantedBy=multi-user.target Reload systemd configuration and configure mysqld_exporter to auto-start on system boot: sudo systemctl daemon-reload sudo systemctl enable mysqld_exporter sudo systemctl start mysqld_exporter Once the MySQLd Exporter is installed, verify that metrics are being exported: curl http://localhost:9104/metrics ===== Useful links ===== * About monitoring: https://vc.ru/dev/62715-kak-my-stroili-monitoring-na-prometheus-clickhouse-i-elk