====== Prometheus monitoring ======
* Grafana prometheus dashboard: https://grafana.com/grafana/dashboards/1860
* Grafana prometheus percona mysqld dashboard: https://grafana.com/grafana/dashboards/7362
* Prometheus rules collection: https://awesome-prometheus-alerts.grep.to/rules.html
===== Agent node_exporter install =====
sudo useradd node_exporter -s /sbin/nologin
mkdir -p /opt/prom-exporters
wget https://github.com/prometheus/node_exporter/releases/download/v*/node_exporter-*.*-amd64.tar.gz
tar xvfz node_exporter-*.*-amd64.tar.gz
sudo mv node_exporter-*.*-amd64/node_exporter /opt/prom-exporters/
rm -rf node_exporter-*.*-amd64
Create a systemd service file ''/etc/systemd/system/node_exporter.service'':
[Unit]
Description=Node Exporter
[Service]
User=node_exporter
EnvironmentFile=/etc/sysconfig/node_exporter
ExecStart=/opt/prom-exporters/node_exporter $OPTIONS
[Install]
WantedBy=multi-user.target
Create a sysconfig file:
sudo mkdir -p /etc/sysconfig
sudo echo "OPTIONS=\"--collector.textfile.directory /var/lib/node_exporter/textfile_collector\"" > /etc/sysconfig/node_exporter
Reload systemd configuration and configure node_exporter to auto-start on system boot:
sudo systemctl daemon-reload
sudo systemctl enable node_exporter
sudo systemctl start node_exporter
Once the Node Exporter is installed, verify that metrics are being exported:
curl http://localhost:9100/metrics
===== Agent mysqld_exporter install =====
Prepare MySQL user for agent:
GRANT REPLICATION CLIENT, PROCESS ON *.* TO 'mysqld_exporter'@'localhost' identified by 'My_Secure_Password';
GRANT SELECT ON performance_schema.* TO 'mysqld_exporter'@'localhost';
FLUSH PRIVILEGES;
Create a mysql password file ''/opt/prom-exporters/.mysqld_exporter.cnf'':
[client]
user=mysqld_exporter
password=My_Secure_Password
Change file owner and permission:
chown prometheus. /opt/prom-exporters/.mysqld_exporter.cnf
chmod 600 /opt/prom-exporters/.mysqld_exporter.cnf
Install mysqld_exporter:
wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.12.1/mysqld_exporter-0.12.1.linux-amd64.tar.gz
tar zxvf mysqld_exporter-0.12.1.linux-amd64.tar.gz
cd mysqld_exporter-0.12.1.linux-amd64
chown prometheus. mysqld_exporter
mv mysqld_exporter /opt/prom-exporters/
cd ..
rm -rf mysqld_exporter-0.12.1.linux-amd64*
Create a systemd service file ''/etc/systemd/system/mysqld_exporter.service'':
[Unit]
Description=Prometheus MySQL Exporter
After=network.target
User=prometheus
Group=prometheus
[Service]
Type=simple
Restart=always
ExecStart=/opt/prom-exporters/mysqld_exporter \
--config.my-cnf /opt/prom-exporters/.mysqld_exporter.cnf \
--collect.global_status \
--collect.info_schema.innodb_metrics \
--collect.auto_increment.columns \
--collect.info_schema.processlist \
--collect.binlog_size \
--collect.info_schema.tablestats \
--collect.global_variables \
--collect.info_schema.query_response_time \
--collect.info_schema.userstats \
--collect.info_schema.tables \
--collect.perf_schema.tablelocks \
--collect.perf_schema.file_events \
--collect.perf_schema.eventswaits \
--collect.perf_schema.indexiowaits \
--collect.perf_schema.tableiowaits \
--collect.slave_status \
--web.listen-address=0.0.0.0:9104
[Install]
WantedBy=multi-user.target
Reload systemd configuration and configure mysqld_exporter to auto-start on system boot:
sudo systemctl daemon-reload
sudo systemctl enable mysqld_exporter
sudo systemctl start mysqld_exporter
Once the MySQLd Exporter is installed, verify that metrics are being exported:
curl http://localhost:9104/metrics
===== Useful links =====
* About monitoring: https://vc.ru/dev/62715-kak-my-stroili-monitoring-na-prometheus-clickhouse-i-elk