Grafana is a time series analytics tool to measure and visualize a whole array of metrics. The most common usage is in datacenter and enterprise environments, but it can also be used for industrial sensors, home automation, weather and deployment control. An open source software, Grafana is currently the leading tool for sysadmins and devops that want to integrate an extremely versatile analytics tool into their development and production environment. With alerts, plugins, dashboards and sharing options between teams, you can even integrate Grafana with 3rd party tools where you can output your data, such as Slack, Webhook or emails for everyone to access the data they wish to check.
Using Grafana, we can create data and output to showcase important corporate metrics, such as KPI, SLA, uptime and much more. Grafana is best used with other tools such as Prometheus and InfluxDB.
Prometheus’s default interface is pretty simple. But thanks to the integration of Prometheus with Grafana, we can get a more pleasant and interesting graphical interface.
To install the local Grafana, on a server where we have previously installed Prometheus, we have several possibilities: either download the latest version and install it manually, or use a Grafana repo (I prefer the last option):
vi /etc/yum.repos.d/grafana.repo
We add the following:
[Grafana]
name=grafana
baseurl=https://packagecloud.io/grafana/stable/el/7/$basearch
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https: //packagecloud.io/gpg.key https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana
sslverify=1
sslcacert=/etc/PKI/TLS/certs/as-bundle.crt
yum update
yum install graph
systemctl daemon-reload
systemctl enable --now graph-server
systemctl status graph-server
Configurating Grafana
The configuration file for Grafana is /etc/grafana/grafana.ini – here, for example, we can configure SMTP details to send alerts (in the SMTP section fill out: enabled = true).
• we navigate to http://IP_server_prometheus:3000
• default credentials are admin – admin; it is recommended to create a new user with a strong password and delete the admin user;
• creating Prometheus data source:
1. click the Grafana logo
2. click Data Source
3. click Add new
4. selecting Prometheus as a type
5. setting the appropriate Prometheus url (for example, http://IP_server_prometheus:9090)
6. adjust desired data source settings (for example, direct access or proxy)
7. click Add to save
Set up alerts in Grafana
Both Prometheus and Grafana have the ability to configure alerts. Alerts in Grafana are very easy to configure. Generally, to set alerts for a dashboard you’ve created, you have to follow the steps below (for alerts to work, you need to configure the SMTP section in /etc/grafana/grafana.ini):
• go to Alerting / Notification channels
• click Add Channel
• name the alert and provide your email address (here you can also send a test email to make sure alerts are sent)
• go back to the Node Exporter Server Metrics dashboard
• click on the first panel to select it, then click Edit from the menu that opens above the panel
• click the Alert tab, then click Create Alert
• configure the alert using the following conditions: WHEN avg () OF query (A,5m, now) IS BELOW 0.2
• select the Notification tab on the right
• at Send to select the notification channel created above
• write a message that is simple and it describes very well the alert (eg, high CPU usage value)
• save the dashboard (or CTRL + S)