Tag Archives: Zabbix

Transforming IT Infrastructure Visibility at Doğan Trend Automotive

Post Syndicated from Michael Kammer original https://blog.zabbix.com/transforming-it-infrastructure-visibility-at-dogan-trend-automotive/30715/

Established in 2020 to consolidate Doğan Group’s automotive and mobility companies and brands under a single entity, Doğan Trend Automotive is a prominent player in their industry. Representing a diverse portfolio ranging from automobiles, motorcycles, and marine engines to electric commercial vehicles, Doğan Trend also delivers innovative solutions to customers through its e-commerce platforms, such as suvmarket.com and vespastoreturkey.com.

The challenge

Doğan Trend’s IT ecosystem spans data centers, remote locations, and multiple units, necessitating seamless operations as well as an efficient monitoring and alert system. The existing infrastructure posed challenges in monitoring, making it difficult to detect potential issues in a timely manner, thus increasing operational risks.

The solution

To address Doğan Trend’s needs, our associates at ASNSKY implemented a Zabbix-based monitoring system. Key highlights of the project included:

  • Centralized dashboards: Custom dashboards were designed for data centers and remote locations, enabling the unified monitoring of IT locations and components from a single interface.
  • A dynamic alert system: Alerts prioritized based on predefined conditions allowed for the swift and effective resolution of critical issues.
  • Seamless operations: Early detection of potential issues prevented operational disruptions and ensured continuity.

Throughout the integration process, ASNSKY’s team collaborated closely with Doğan Trend’s IT department, addressing the specific requirements of different units and providing training for effective system use.

The results

Implementing the new monitoring system rapidly delivered the following results:

  • Enhanced visibility: Real-time monitoring of all IT locations and components made potential issues easy to spot.
  • Proactive issue management: Early detection of critical issues reduced operational downtime.
  • Increased efficiency: The centralized monitoring system drastically improved the responsiveness and effectiveness of Doğan Trend’s IT team.

“This project with the ASNSKY team made our IT infrastructure more transparent and manageable. With Zabbix’s flexible and effective monitoring capabilities, we gained active control over our critical operations. We thank the ASNSKY team for this successful collaboration.” – Burak Altunalan, IT System Management Specialist at Doğan Trend Automotive

In conclusion

Doğan Trend plans to further take advantage of Zabbix’s flexibility to strengthen their resilience and operational efficiency. Their association with ASNSKY marks a significant step toward achieving these objectives.

To learn more about what Zabbix can do for retail customers in every sector, get in touch with us. 

About ASNSKY

ASNSKY enhances its customers’ competitiveness by integrating the power of enterprise-grade open-source-solutions in security and infrastructure with a professional service approach and high quality standards.

Backed by deep industry expertise and a team of seasoned professionals, ASNSKY stands as a trusted partner in your digital transformation journey.

The post Transforming IT Infrastructure Visibility at Doğan Trend Automotive appeared first on Zabbix Blog.

How to Install Zabbix on Windows with a Linux Subsystem

Post Syndicated from Alexander Petrov-Gavrilov original https://blog.zabbix.com/how-to-install-zabbix-on-windows-with-a-linux-subsystem/30311/

It’s a very well known fact that Zabbix can only be installed on Linux. But what if you are in a Windows environment and getting a Linux machine is not so simple or even possible? This can obstruct the implementation of Zabbix, or at least significantly delay it. Not only that, building a POC outside of the future environment makes data procurement a lot more complicated. Is there a way to work around this and get Zabbix as close to Windows as we possibly can?

WSL

WSL/WSL 2 is a fast and easy solution for installing and using Zabbix in a smaller Windows-dominant environment, be that a POC or a small company office. WSL 2 runs a real Linux kernel in a lightweight VM while being optimized for Windows. This means a faster start, lower resource consumption, and the ability to share files with Windows directly, meaning you can use Windows File explorer to find and manage the VM files.

WSL 2 also allows you to use Linux CLI while working with Windows (i.e. running vim from a Windows terminal and editing Windows files directly). At this point, you may be asking yourself, “Why not Hyper-V and VirtualBox?” Those are definitely options too, but they are quite heavy on system resources. In addition, boot times are a bit longer and sharing files between a host and a guest OS is clunkier.

Maybe Docker Desktop then? It’s an absolutely valid option, but that would require a bit of Docker knowledge and you would still be using WSL, technically speaking. So, with that said, WSL is definitely the fastest and most reliable way to sprung a Zabbix instance in a Windows-focused environment.

We will use WSL 2, but as a note WSL 1 is also available. Here are the differences:

  • WSL 2 is usually the better performer overall, especially for dev environments. It also has better Linux compatibility.
  • WSL 1 Linux files aren’t isolated, which can make them more accessible. In WSL 2, Linux runs in a virtual disk (ext4), so Linux and Windows files are more separate. Integration is still pretty good, however.
  • WSL 2 has better Linux compatibility – systemd, iptables, etc.
  • WSL 1 shared the same IP as Windows, WSL 2 is a VM – some networking required.
  • With WSL 1 you can see Linux running processes in Task Manager. WSL 2 will have processes isolated.

Installing Zabbix using WSL

Install WSL

Open PowerShell as an Administrator and run:

PS C:\Windows\system32> wsl --install

If you’ve already have WSL 1 installed, update it:

PS C:\Windows\system32> wsl --update

You can also set WSL 2 as default:

PS C:\Windows\system32> wsl --set-default-version 2
WSL installation
WSL installation

 

Install/Get preferred Linux Distribution using either Microsoft Store (i.e. Ubuntu, Debian, Oracle Linux) or just download directly. I will be using Oracle Linux 9.4.

Microsoft store WSL images
Microsoft store WSL images

 

You can also download the RootFS tarball from the preferred distribution portal, but then the process will be a bit different. Create a folder using PowerShell:

PS C:\Windows\system32> mkdir C:\WSL\OracleLinux9

Copy the .tar.xz file to this folder, then run:

PS C:\Windows\system32> wsl --import OracleLinux9 C:\WSL\OracleLinux9 .\oraclelinux9-rootfs.tar.xz --version 2

After the image is installed or imported, start Oracle Linux using PowerShell:

PS C:\Windows\system32> oraclelinux94

When installation is finished, there is a prompt to create a default UNIX user account and password for the said user, as the username does not need to match your Windows username. I’ll set it to “zabbix” of course, but you can set it to any other.

PS C:\Windows\system32> Enter new UNIX username: 
PS C:\Windows\system32> zabbix
PS C:\Windows\system32> New password: <your-password>
PS C:\Windows\system32> passwd: all authentication tokens updated successfully.
PS C:\Windows\system32> Installation successful!

Now OracleLinux is ready for use!

Prepare the system

You will be immediately logged in to the new environment. If logged out, to log in again just execute in PowerShel:

PS C:\Windows\system32> oraclelinux94

Being logged in, first double check that your selected OS is indeed installed by executing in the PowerShell, which will now serve as your VM CLI access point:

[zabbix@PC-NAME ~]$ cat /etc/os-release
NAME="Oracle Linux Server"
VERSION="9.4"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="9.4"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Oracle Linux Server 9.4"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:9:4:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL=https://github.com/oracle/oracle-linux

Confirmation received, make sure all OS updates are installed:

[zabbix@PC-NAME ~]$ sudo dnf update -y

When the update process is finished, you will need to decide whether you would like to use systemd or not (this may increase booting time). I will enable systemd. To do this, edit the wsl.conf on the Linux subsystem:

vi /etc/wsl.conf

Add to the newly created file:

[boot]
systemd=true

Reboot the images (this command will reboot all of them):

PS C:\Windows\system32> wsl.exe --shutdown

Start back your Linux distribution:

PS C:\Windows\system32> oraclelinux94

Install Zabbix database

We will need to prepare the database engine. Again, any preferred database engine can be used, in this case I install and configure MariaDB:

[zabbix@PC-NAME ~]$ sudo dnf install -y mariadb-server mariadb
[zabbix@PC-NAME ~]$ sudo systemctl enable --now mariadb

Confirm MariaDB is running:

[zabbix@PC-NAME ~]$ Systemctl status mariadb

mariadb.service - MariaDB 10.5 database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: disabled)
     Active: active (running) since Tue 2025-04-29 12:39:54 EEST; 3min 55s ago
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/
   Main PID: 235 (mariadbd)
     Status: "Taking your SQL requests now..."
      Tasks: 9 (limit: 26213)
     Memory: 109.6M
     CGroup: /system.slice/mariadb.service
             └─235 /usr/libexec/mariadbd --basedir=/usr

After confirmation, secure it a bit by creating a root password and selecting the options in bold:

[zabbix@PC-NAME ~]$ sudo mysql_secure_installation 

Enter current password for root (enter for none):
OK, successfully used password, moving on...

Setting the root password or using the unix_socket ensures that nobody
can log into the MariaDB root user without the proper authorisation.

You already have your root account protected, so you can safely answer 'n'.

Switch to unix_socket authentication [Y/n] n
 ... skipping.

You already have your root account protected, so you can safely answer 'n'.

Change the root password? [Y/n] Y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
 ... Success!

Remove anonymous users? [Y/n] Y
 ... Success!

Disallow root login remotely? [Y/n] Y
 ... Success!

Remove test database and access to it? [Y/n] Y


Reload privilege tables now? [Y/n] Y

 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

Now to create the Zabbix database. Log in to MariaDB:

[zabbix@PC-NAME ~]$ sudo mysql -u root -p 
[zabbix@PC-NAME ~]$ Enter password: <enter your password, won’t be visible>

Follow the steps from the Zabbix installation page:

MariaDB [(none)]> create database zabbix character set utf8mb4 collate utf8mb4_bin;
MariaDB [(none)]> create user zabbix@localhost identified by '<custom-password>';
MariaDB [(none)]> grant all privileges on zabbix.* to zabbix@localhost;
MariaDB [(none)]> set global log_bin_trust_function_creators = 1;

MariaDB [(none)]> quit;

Installing Zabbix

Install the Zabbix repository:

[zabbix@PC-NAME ~]$ sudo dnf install https://repo.zabbix.com/zabbix/7.0/centos/9/x86_64/zabbix-release-latest-7.0.el9.noarch.rpm
[zabbix@PC-NAME ~]$ dnf clean all

Proceed to install the Zabbix server, frontend, and agent:

[zabbix@PC-NAME ~]$ sudo dnf -y install zabbix-server-mysql zabbix-web-mysql zabbix-apache-conf zabbix-sql-scripts zabbix-selinux-policy zabbix-agent
...
[zabbix@PC-NAME ~]$zabbix-agent-7.0.12-release1.el9.x86_64 zabbix-apache-conf-7.0.12-release1.el9.noarch zabbix-selinux-policy-7.0.12-release1.el9.x86_64  zabbix-server-mysql-7.0.12-release1.el9.x86_64 zabbix-sql-scripts-7.0.12-release1.el9.noarch
 zabbix-web-7.0.12-release1.el9.noarch zabbix-web-deps-7.0.12-release1.el9.noarch zabbix-web-mysql-7.0.12-release1.el9.noarch

Complete!

Now import the initial database schema:

[zabbix@PC-NAME ~]$ zcat /usr/share/zabbix-sql-scripts/mysql/server.sql.gz | mysql -u zabbix -p zabbix
Enter password: <enter your DB user password and wait until you will see the next line appear>
[root@ZBX-5CD3221K14 zabbix]#

Disable the log_bin_trust_function_creators option after import has finished:

# mysql -uroot -p
password
MariaDB [(none)]>  set global log_bin_trust_function_creators = 0;
MariaDB [(none)]>  quit;

Add your Zabbix user database password to the Zabbix server configuration file:

[zabbix@PC-NAME ~]$ vi /etc/zabbix/zabbix_server.conf
### Option: DBPassword
#       Database password.
#       Comment this line if no password is used.
#
# Mandatory: no
# Default:
DBPassword=<your-DB-user-password>

Start the Zabbix server and frontend and add them to autorun:

[zabbix@PC-NAME ~]$  systemctl restart zabbix-server zabbix-agent httpd php-fpm
[zabbix@PC-NAME ~]$  systemctl enable zabbix-server zabbix-agent httpd php-fpm
Created symlink /etc/systemd/system/multi-user.target.wants/zabbix-server.service → /usr/lib/systemd/system/zabbix-server.service.
Created symlink /etc/systemd/system/multi-user.target.wants/zabbix-agent.service → /usr/lib/systemd/system/zabbix-agent.service.
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/php-fpm.service → /usr/lib/systemd/system/php-fpm.service.

Installation of the backend is now finished, but we still need the frontend.

Exposing and installing the Zabbix frontend for WSL

Since WSL2 does not expose services to localhost by default, you need to determine the WSL IP:

[zabbix@PC-NAME ~]$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:47:32:c6 brd ff:ff:ff:ff:ff:ff
    inet 172.29.128.155/20 brd 172.29.143.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:fe47:32c6/64 scope link
       valid_lft forever preferred_lft forever

Look for an IP like 172.x.x.x, then using your browser go to:

http://<WSL_IP>/zabbix

In this example, that would be: 

http://172.29.128.155/zabbix

You can also port forward WSL to localhost with netsh in PowerShell:

PS C:\Windows\system32> netsh interface portproxy add v4tov4 listenport=8080 listenaddress=127.0.0.1 connectport=80 connectaddress=<WSL_IP>

Then you will be able to access Zabbix from http://localhost:8080/zabbix. Now, just finish the standard frontend setup and Zabbix is ready to use!

WSL advantages

Some extra advantages you get with this approach include clearer resource usage visibility:

WSL Task manager

 

Direct access to the Linux subsystem files through File explorer with your favorite Windows tools:

Linux subsystem file explorer
Linux subsystem file explorer

 

As you can see, docker is here as well. System and configuration files are also visible and editable:

File explorer Zabbix config files
File explorer Zabbix config files

 

Now you can proceed with building your Zabbix or Zabbix POC, (almost) without needing to leave your regular Windows environment!

The post How to Install Zabbix on Windows with a Linux Subsystem appeared first on Zabbix Blog.

Zabbix and a Federal Government Agency

Post Syndicated from Michael Kammer original https://blog.zabbix.com/zabbix-and-a-federal-government-agency/30708/

Our Premium Partners at the ATS Group work with a large federal government agency in the United States. They primarily provide storage and compute-as-a-service for the agency, which relies on them to stay up and running at all times.

The challenge

The agency’s primary goal was to simplify their capacity and performance monitoring without extra costs. They had very strict regulatory and SLO oversight requirements that had to be met, especially when it came to capacity and performance.

There was no commercially available software that could accomplish everything they needed directly out of the box, but they still required a solution that was powerful and flexible enough to monitor almost anything.

The solution

Because the agency has several different data centers of different sizes, they use a distributed proxy set up, intense SLA reporting, a ServiceNow integration, a variety of internal integrations, and a monitoring solution provided by Zabbix that includes a predictive alerting setup.

The agency has plenty of software in the mix, but it primarily relies on storage, VMWare, and Kubernetes. They also have multiple satellite offices and data centers, so that in the event of a data center failure, another can come online with minimal downtime in between.

On top of that, they have over 30 metrics and more than a trillion data points across 10 major technologies that they need to measure, primarily from a regulatory perspective. Thousands of granular metrics needed to have solutions and reporting designed for them in Zabbix, including (for example) CPU cores and frequency, processor-to-core usage metrics, and virtualization ratios from hosts to virtual machines.

Their Kubernetes-based Openshift environment also needs to be monitored to exact specifications. Deployment took place via Helm Chart, with Zabbix components being installed as Kubernetes resources, node-level resources, and applications being monitored, while data was aggregated and sent to the Zabbix server.

Metrics are collected via the Kubernetes API and kube-state metrics, and the solution uses Prometheus-exported metrics or direct HTTP endpoint calls. When it comes to configuration, proxies and hosts are created in Zabbix to represent Kubernetes nodes and clusters, while templates and macros are configured to point to the Kubernetes API and kube-state-metrics endpoints.

The results

Thanks to Zabbix, the federal government agency in question has a solution that provides centralized monitoring of Kubernetes alongside other IT resources, supports application-specific metrics without requiring Prometheus endpoints, and offers plenty of flexibility to customize and scale.

In addition, Zabbix’s predictive alerting capabilities identify abnormalities in operational data and predictively alert the agency about anything that could potentially impact an application or service, which lets them meet SLAs, optimize user experience, and increase productivity.

In conclusion

Zabbix’s flexibility and ease of customization make it ideal for customers who need a single source of truth that can be relied on in even the most stringent regulatory environments.

To learn more about what Zabbix can do for customers in the public sector, visit us here.

The post Zabbix and a Federal Government Agency appeared first on Zabbix Blog.

Zabbix at the Netherlands Ministry of Infrastructure and Water Management

Post Syndicated from Michael Kammer original https://blog.zabbix.com/zabbix-at-the-netherlands-ministry-of-infrastructure-and-water-management/30681/

The Ministry of Infrastructure and Water Management is the Dutch ministry responsible for transport, aviation, housing policy, public works, spatial planning, land management, and water resource management. Created in 2010 following the merger of the Ministry of Transport and Water Management and the Ministry of Housing, Spatial Planning, and Environment, the ministry works to create an efficient network of roads, railways, waterways, and airways, effective water management to protect against flooding, and improved air and water quality.

The challenge:

The ministry needed a monitoring solution that could handle not only infrastructure monitoring, but also IoT devices responsible for monitoring water levels, water quality, temperature, and other data. The infrastructure components that needed to be monitored included Red Hat Satellite and Capsule servers, Red Hat Virtual Data Centers, Red Hat Identity management, Ansible automation platforms, and a wide range of custom IoT devices.

The solution:

The Red Hat Satellite and Capsule monitoring consists of one satellite, 6 server, and 15 satellite capsules for different environments, with approximately 2000 Linux machines connected to the satellite capsules. The machines retrieve their packages from the capsules and the capsules act as proxies that fetch data from the satellite servers. The capsules also manage the content packages and subscriptions for the machines.

For Red Hat satellite and Capsule monitoring, Zabbix performs capsule discovery via Low Level Discovery, which uses Http requests, which in turn collect data via the REST API. Each capsule’s content sync status is monitored and if the content sync fails, new packages are not installed. Connectivity between capsules and the satellite is also monitored by performing port checks, because capsules need to be able to connect to the satellite in order for the content to be synced.

Zabbix also discovers and monitors satellite repositories, checking both when the last sync was performed and the current sync status. Software subscriptions are also discovered and monitored and alerts are sent, with the severity of the alerts raised at the point when a subscription has only 30 days remaining.

Red Hat Virtual Data Center licences and identity management also benefit from the added flexibility that Zabbix brings to the table. Virtual DC licences must be present on ESX hosts, so situations where an ESX host with an active license has no VMS on it (or has VMS migrated to it) must be avoided, because that would mean that a license is being essentially wasted. Whenever a Zabbix trigger detects a problem, Ansible automatically attaches or detaches a licence to or from the ESX host, depending on the type of problem detected.

When it comes to Red Hat identity management, Zabbix discovers and monitors processes on the identity management platform (including identity management service status) thanks to the ability to extend Zabbix agent with user parameters.

Meanwhile, Ansible Automation Platform monitoring consists of monitoring for controllers. The Ansible Automation Platform API is used to discover the controllers, and each controller is checked to see if any jobs are running, their last seen time, their capacity, and their status. Sometimes controllers are disabled for maintenance and then re-enabled, so alerts are sent out for controllers that have been disabled for a longer time.

Ansible Automation Platform monitoring also includes monitoring decommission machines, which are assigned to a group instead of being immediately deleted. Zabbix monitors the grace period for the decommission machines and alerts users if the grace period is over, generating a warning if an Ansible host is disabled for seven days and then escalating it if the machine has been disabled for more than 14 days.

Zabbix also discovers and monitors configuration management jobs, and if a job fails it will attempt to restart it. If the issue is still not resolved, it gets escalated to the appropriate individual. These Ansible checks are primarily done via Http agents, from Zabbix servers or proxies.

Finally, in addition to infrastructure monitoring, Zabbix also monitors the health of IoT devices responsible for water levels, water quality, temperature, and other data. These devices are running Raspberry Pi modules and Zabbix Agent 2 is used to monitor the device status. Zabbix Agent 2 with a local agent database is used in cases where the agent is unable to send the metrics on these devices. Should a network outage happen, Zabbix stores the backlog data in the local agent database.

The results:

Trusting their monitoring to Zabbix has greatly improved processes at the ministry, saving time and money by making it easy to notice and fix issues before affected departments themselves were aware of them. In addition, having the latest historical data at their fingertips has been invaluable to the ministry’s technical teams during troubleshooting or when dealing with performance issues, saving everyone involved a great deal of time.

In conclusion

Zabbix’s flexible nature and its ability to integrate with popular platforms as well as custom devices made it the perfect “one-stop shop” for the ministry’s needs, consolidating all of their monitoring in a single pane of glass and giving them complete visibility into every layer of their infrastructure – while also integrating smoothly with their existing systems.

To learn more about what Zabbix can do for customers in the public sector, contact us.

The post Zabbix at the Netherlands Ministry of Infrastructure and Water Management appeared first on Zabbix Blog.

Podman Container Monitoring with Prometheus Exporter, part 2

Post Syndicated from Janis Eidaks original https://blog.zabbix.com/podman-container-monitoring-with-prometheus-exporter-part-2/30538/

In the first part of this post, we explored how to get data with HTTP agent from the Prometheus Podman exporter and use the same item data for the Podman pods Discovery rule as well as item and trigger prototypes. In part 2 of the same series, we’ll learn how to discover and monitor Podman containers.

Creating a template discovery rule

I will create another discovery rule for container discovery. This discovery rule is also based on the same item [Podman info] in the template – Podman containers by HTTP and Prometheus (you can check part one of this series to find out how to configure it). The parameters of the discovery rule are shown below. This discovery rule will allow us to discover the pod name and ID.

Template: Podman containers by HTTP and Prometheus

▲ Discovery rule
  ▪ Name:                   Container discovery
  ▪ Type                    Dependent item
  ▪ Key:                    training.containers.discovery
  ▪ Master item             Podman containers by HTTP and Prometheus: Podman info
  ▪ Delete lost resources  After 10d
  ▪ Disable lost resources Immediately
♯ Preprocessing
  ▪ Prometheus to JSON     podman_container_info
♦ LLD Macros
  ▪ {#CONTAINER.ID}        $.labels.id
  ▪ {#CONTAINER.NAME}      $.labels.name
Fig 1. Discovery rule: Container discovery
Fig 2. Discovery rule: Container discovery preprocessing tab
Fig 3. Discovery rule: Container discovery LLD macros tab

Next, different dependent item prototypes are created in this container discovery rule. As the Prometheus Podman exporter provides a lot of different metrics about the containers, I will create multiple such items: state, health, creation date, input/output network traffic information, and so on. So, check out what metrics can be acquired and use what is relevant for you.

You can also add a description of each item prototype. I am interested only in metrics with the discovered container ID macros, and I am not interested in what values are for the other fields, such as pod_id, pod_name, so I use ~”.*”, which matches any value. I will show the item prototype configuration screenshots of one of the item prototypes.

These item prototypes are similar to each other, with some minor differences, such as Prometheus patterns, or in some cases, with a different master item (item prototype as master item).

Fig 4. Discovery rule preprocessing step: Prometheus to JSON with pattern podman_container_info
Fig 5. Discovery rule LLD macros: assigning relevant JSONPATH to LLD macros

Creating a template discovery rule: Item prototypes

After the containers have been discovered, we have to create item prototypes. These prototypes will also be dependent item prototypes and will use the same item as the discovery rule: Podman info. Prometheus Podman exported returns a lot more metrics for the containers than it did for the pods.

You can get container metrics such as container health, state, creation date, disk read/write, memory usage, network usage, and more. In this blog post, I have added most of them, so check what metrics are relevant to your monitoring needs and start monitoring.

Fig 6. Low-level discovery rule and item prototypes based on the same item.

The screenshots of the item prototype is shown below.

Fig 7. Container state item prototype tab
Fig 8. Container state item prototype tag tab
Fig 9. Container state item prototype preprocessing tab

Remember, you can also test these item prototypes in the preprocessing step – just copy the Prometheus exporter data and set the relevant macro to value you want to check.

The configuration parameters of the item prototypes are shown below. There are a lot of metrics you can monitor, but remember to monitor what is relevant and necessary for you.

Template: Podman containers by HTTP and Prometheus; Discovery rule: Container discovery

● Item prototype #1
  ▪ Name: 		Container health: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.health[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (float)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 
♦ Tags (name:value) 	
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:health		
♯ Preprocessing
  ▪ Prometheus pattern 	podman_container_health{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #2
  ▪ Name: 		Container state: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.state[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (float)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		
♦ Tags (name:value)  			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:state		
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_state{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #3
  ▪ Name: 		Created at: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.created[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		unixtime
♦ Tags (name:value) 		
  ▪ Container:{#CONTAINER.NAME}	
  ▪Metric:created		
♯ Preprocessing
  ▪ Prometheus pattern 	podman_container_created_seconds{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #4
  ▪ Name: 		Disk read per second: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.disk.read[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		B
♦ Tags (name:value) 	
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:disk_read		
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_block_output_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value
  ▪ Change per second

● Item prototype #5
  ▪ Name: 		Disk write per second: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.disk.write[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		B
♦ Tags (name:value) 	
  ▪ Container:{#CONTAINER.NAME}	 
  ▪ Metric:disk_write		
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_block_input_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value
  ▪ Change per second

● Item prototype #6
  ▪ Name: 		Exit code: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.exit_code[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (float)
    ▪ Master item	Podman containers by HTTP and Prometheus: Podman info
▪ Units: 			
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:exit_code	
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_exit_code{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #7
  ▪ Name: 		Image tags: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.image.tags[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Character
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 			
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:tag
♯ Preprocessing
▪ Prometheus pattern podman_container_info{id="{#CONTAINER.ID}",image=~".*",name=~".*",pod_id=~".*",pod_name=~".*",ports=~".*"} label image
  ▪ Regular expression	\.*(\/.\w.*)	\1

● Item prototype #8
  ▪ Name: 		Memory usage: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.mem[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		B
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:mem		
♯ Preprocessing
  ▪ Prometheus pattern podman_container_mem_usage_bytes{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #9
  ▪ Name: 		Network input dropped: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.in.drop[{#CONTAINER.NAME}]
  ▪ Type of inf: Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		packets
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_in_drop		
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_net_input_dropped_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #10
  ▪ Name: 		Network input errors: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.in.errors[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_in_err		
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_net_input_errors_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #11
  ▪ Name: 		Network input total: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.in.total[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		B
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_in_tot
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_net_input_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #12
  ▪ Name: 		Network input per second: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.in.change[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (float)
  ▪ Master item		prototype - Network input total: [{#CONTAINER.NAME}] 
  ▪ Units: 		Bps
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_in_change
♯ Preprocessing
  ▪ Change per second

● Item prototype #13
  ▪ Name: 		Network output dropped: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.out.drop[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_out_drop	
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_net_output_dropped_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #14
  ▪ Name: 		Network output errors: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.out.errors[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_out_err	
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_net_output_errors_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #15
  ▪ Name: 		Network output total: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.out.total[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		B
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_out_tot	
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_net_output_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #16
  ▪ Name: 		Network output per second: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.net.out.change[{#CONTAINER.NAME}]
  ▪ Type of inf: 	Numeric (float)
  ▪ Master item		prototype - Network output total: [{#CONTAINER.NAME}]
  ▪ Units: 		Bps
♦ Tags 			 
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:net_out_change
♯ Preprocessing
  ▪ Name			Change per second

● Item prototype #17
  ▪ Name: 		Rootfs size: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.rootfs.size[{#CONTAINER.NAME}]
  ▪ Type of inf: Numeric (unsigned)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		B
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:rootfs
♯ Preprocessing
  ▪ Prometheus pattern	podman_container_rootfs_size_bytes{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

● Item prototype #18
  ▪ Name: 		Total system CPU time: [{#CONTAINER.NAME}]
  ▪ Type 		Dependent item
  ▪ Key: 		container.cpu.time
  ▪ Type of inf: 	Numeric (float)
  ▪ Master item		Podman containers by HTTP and Prometheus: Podman info
  ▪ Units: 		s
♦ Tags 			
  ▪ Container:{#CONTAINER.NAME}	
  ▪ Metric:sys_time
♯ Preprocessing
  ▪ Prometheus pattern: podman_container_cpu_system_seconds_total{id="{#CONTAINER.ID}",pod_id=~".*",pod_name=~".*"} value

Creating a template discovery rule: Trigger prototype

I have created a user macro {$CONTAINER.RUNNING.STATE} on the template with a value of 2, which corresponds to the containers running state. After that, create a trigger prototype to check if the container is in different state other than running.

Template: Podman containers by HTTP and Prometheus; Discovery rule: Container discovery

◘ Trigger prototypes
  ▪ Name:               Container [{#CONTAINER.NAME}] state has changed from running
  ▪ Severity:           Warning
  ▪ Expression:         last(/Podman containers by HTTP and Prometheus/container.state[{#CONTAINER.NAME}])<>{$CONTAINER.RUNNING.STATE}
  ▪ PROBLEM event generation mode: Single
  ▪ OK event closes: All problems

So, once all of this is done, and some container status changes from running and or pod status also changes from running, you will get a problem event.

Fig 10. Generated problem events when the podman pod and container change states.

Technically, I could also create a trigger for container health; however, as all of the received container values for me are -1 (meaning unknown) it makes little sense to make a trigger that will fire right away. You can also add additional item/trigger prototypes in the template. If everything is set up as expected, you should see something like the screenshot below after the LLD rule execution.

Fig 11. Example of the mysql-server container and zabbix pod item values.

Summary

Now, you can monitor both Podman pods and containers using both blog posts of this series. We used the same template item for both the container LLD and item prototypes from the first part of this post.

The post Podman Container Monitoring with Prometheus Exporter, part 2 appeared first on Zabbix Blog.

Podman Container Monitoring with Prometheus Exporter, part 1

Post Syndicated from Janis Eidaks original https://blog.zabbix.com/podman-container-monitoring-with-prometheus-exporter-part-1/30513/

In part one of this blog post, I will show you how to monitor Podman pods using HTTP agent item to retrieve data from the Prometheus Podman exporter. Let’s get started!

Installing and checking Prometheus Podman exporter

First, you will need to install and enable the Prometheus Podman exporter (my OS is CentOS Stream release 9). Then, check that the service is active and running.

# dnf install -y prometheus-podman-exporter

# systemctl enable prometheus-podman-exporter –now

# systemctl status prometheus-podman-exporter

You can check that you are getting the data from the exporter with either the curl command from the machine/VM where the Prometheus podman exporter is installed and started:

# curl http://localhost:9882/metrics
Fig 1. Output of Prometheus podman exporter in CLI

Or through the browser (replace abc with the machine’s IP/DNS ): abc:9882/metrics.

Fig 2. Output of Prometheus Podman exporter in browser

A line starting with # is a comment and contains an explanation regarding the metric; in this case, podman_container_block_input_total will return data in bytes.  In Figure 2, after the comments, you can see several podman_container_block_input_total metrics, one for each container, with different container IDs, pod IDs, and pod names listed in each metric. The metric’s value is displayed on the right side after curly brackets.

Creating a template and template items

Next, I will create a template Podman containers by HTTP and Prometheus where I will put all of the entities (everything will be created on the template). In the template, I will create an item Podman info, which will gather all of the necessary data at defined intervals. This approach will be convenient from a data collection standpoint as the same item data will be used for LLD and item prototypes. During testing, you can set “History” to store data for some time, and when everything is working as expected, then set “History” not to keep any data. This item will be used for the Low-Level Discovery rule and the item prototype.

The item Podman info parameters are as follows:

Template: Podman containers by HTTP and Prometheus

○ Item
  ▪ Name:         Podman info
  ▪ Type          HTTP agent
  ▪ Key:          podman.info
  ▪ Type of inf   text
  ▪ URL           http://{HOST.CONN}:9882/metrics
  ▪ Request type  GET
  ▪ Update int.   5m
  ▪ Req status c. 200
  ▪ History       Do not store
◊ Tags
  ▪ Podman:raw

 

Fig 3. Template item for data gathering

At this moment, this item will contain just raw data, without any preprocessing steps applied. The IP address will be taken from any host interface added to the host. You will get an error message if the host has no interface.

Fig 4. Error on the host with the linked template without any interface

If you do not want to add an interface to the host, you can define a user macro on the template level and use that user macro in the items URL. After adding the template to the host, just modify the user macro value on the host to correct IP/DNS name.

Fig 5. User macro on template
Fig 6. Template item for data gathering with user macro instead of built in macro from host interface

I can also create an item to determine the number of containers created. I can count specific Prometheus pattern occurrences in the master item to determine this. For this, I will use the podman_container_state parameter. Likewise, I could use different parameters, such as podman_container_info, and count the occurrences of such a pattern. The parameters of the item container count:

Template: Podman containers by HTTP and Prometheus

○ Item
  ▪ Name:         Container count
  ▪ Type          Dependent item
  ▪ Key:          container.count
  ▪ Type of inf   Numeric (unsigned)
  ▪ Master item   Podman containers by HTTP and Prometheus: Podman info
◊ Tags
  ▪ Containers:total
♯ Preprocessing
  ▪ Prometheus pattern     podman_container_state     count
Fig 7. Template item preprocessing step for counting the total number of containers

Creating a Discovery rule in template

Next, the LLD rule will be created to discover Podman pods. It will be a dependent LLD rule based on a Podman info item with a preprocessing step to convert the Prometheus pattern data to JSON format. The caveat is that the LLD discovery will be executed as frequently as the data is received for the item. If there are a lot of hosts with such a template, there will be a lot of LLD processes executed, which can put a strain on your Zabbix instance.

To rectify this issue, I will add a preprocessing step: discard unchanged with heartbeat (as there are no dynamic parameters in the extracted pattern, otherwise we would need to filter out dynamically changing information). For LLD discovery, the recommended interval is around 1h. Additionally, LLD macros will be created from selected JSNOPath variables. The parameters of the LLD rule are shown below.

Template: Podman containers by HTTP and Prometheus

▲ Discovery rule
  ▪ Name:                   POD discovery
  ▪ Type                    Dependent item
  ▪ Key:                    training.pod.discovery
  ▪ Master item             Podman containers by HTTP and Prometheus: Podman info
  ▪ Delete lost resources  After 10d
  ▪ Disable lost resources Immediately
♯ Preprocessing
  ▪ Prometheus to JSON     podman_pod_info
  ▪ Discard unchanged with heartbeat 1h
♦ LLD Macros
  ▪ {#POD.ID}              $.labels.id
  ▪ {#POD.NAME}            $.labels.name
Fig 8. Discovery rule: Pod discovery
Fig 9. Discovery rule: Pod discovery preprocessing tab
Fig 10. Discovery rule: Pod discovery LLD macros tab

The block diagram below will show how the data is transformed. First, a preprocessing step is applied to the data to convert the Prometheus pattern to JSON format, as all data for LLD must be supplied in JSON format.

In the example below, the matching queried pattern is returned in JSON format after this preprocessing step.

Fig 11. Discovery rule preprocessing step: Prometheus to JSON with pattern podman_pod_info

After the preprocessing step, we can assign specific JSONPATH values to LLD macros.

Fig 12. Discovery rule LLD macros: assigning relevant JSONPATH to LLD macros

Creating a template Discovery rule: item prototypes

Now that we have discovered the macros we are interested in, the discovered macros can be used for further prototype (ITEM/HOST/TRIGGER) creation. In this example, I am using the same master item for LLD discovery and the dependent item prototypes, because it is convenient for me, and all the information is available in one item. But usually, there are scenarios where you have to use one item’s data for discovery and the data of another item for populating the prototype values.

In this case, I am interested in the pod ID, when the pod was created, the number of containers in the pod, and the state of the pod. Therefore, I will create the item prototypes and use the LLD macro in the name, key, and preprocessing step. Zabbix will cycle through the discovered LLD macro values and create the items based on the prototype by replacing the LLD macro with discovered values. Although you can set matching item prototype names (which will be confusing), you still have to use the LLD macro in the item key so that different item keys are generated – otherwise, you will get an error regarding duplicate keys. The item prototype parameters are given below.

Fig 13. Low-level discovery rule and item prototypes based on the same item.
Template: Podman containers by HTTP and Prometheus; Discovery rule: POD discovery

○ Item prototype #1
  ▪ Name:         POD ID: [{#POD.NAME}]
  ▪ Type          Dependent item
  ▪ Key:          pod.id[{#POD.NAME}]
  ▪ Type of inf   Character
  ▪ Master item   Podman containers by HTTP and Prometheus: Podman info
♦ Tags
  ▪ Metric:ID
  ▪ Pod:{#CONTAINER.NAME}
♯ Preprocessing
  ▪ Prometheus pattern     podman_pod_containers{id="{#POD.ID}"}          label    id

○ Item prototype #2
  ▪ Name:         POD state: {#POD.NAME}
  ▪ Type          Dependent item
  ▪ Key:          pod.state[{#POD.NAME}]
  ▪ Type of inf   Numeric (float)
  ▪ Master item   Podman containers by HTTP and Prometheus: Podman info
  ▪ Value mapping POD state
♦ Tags
  ▪ Metric:state
  ▪ Pod:{#CONTAINER.NAME}
♯ Preprocessing
  ▪ Prometheus pattern     podman_pod_state{id="{#POD.ID}"}      value

○ Item prototype #3
  ▪ Name:         POD created at: [{#POD.NAME}]
  ▪ Type          Dependent item
  ▪ Key:          pod.created[{#POD.NAME}]
  ▪ Type of inf   Numeric (unsigned)
  ▪ Units         unixtime
  ▪ Master item   Podman containers by HTTP and Prometheus: Podman info
♦ Tags
  ▪ Metric:created
  ▪ Pod:{#CONTAINER.NAME}
♯ Preprocessing
  ▪ Prometheus pattern     podman_pod_created_seconds{id="{#POD.ID}"}     value

○ Item prototype #4
  ▪ Name:         POD container count: [{#POD.NAME}]
  ▪ Type          Dependent item
  ▪ Key:          pod.count[{#POD.ID}]
  ▪ Type of inf   Numeric (unsigned)
  ▪ Master item   Podman containers by HTTP and Prometheus: Podman info
♦ Tags
  ▪ Metric:count
  ▪ Pod:{#CONTAINER.NAME}

On the template, I have also created a value map for deciphering the numerical pod state codes to text strings for better clarity.

Fig 14. Value mapping for the POD state item

Here are some screenshots of the POD state item prototype, shown below.

Fig 15. POD state item prototype: item prototype tab
Fig 16. POD state item prototype: tag tab
Fig 17. POD state item prototype: preprocessing tab

Creating a template Discovery rule: trigger prototype

We can also create a trigger prototype to generate an alert if there is something wrong with the pod. I have created a user macro {$POD.RUNNING.STATE} on the template with a value of 4, which corresponds to the running state.

Template: Podman containers by HTTP and Prometheus; Discovery rule: POD discovery

◘ Trigger prototypes:
  ▪ Name:               POD [{#POD.NAME}] state has changed from running
  ▪ Severity:           Warning
  ▪ Expression: last(/Podman containers by HTTP and Prometheus/pod.state[{#POD.NAME}])<>{$POD.RUNNING.STATE}
  ▪ PROBLEM event generation mode: Single
  ▪ OK event closes: All problems
Fig 18. Trigger prototype based on POD state item value

Once you link the template to the host and execute the LLD rule, you should start seeing the Podman pods ( if you have them), similar to the screenshot below.

Fig 19. Latest data for the host with the linked template

Summary

This blog post shows how to get data with HTTP agent from Prometheus Podman exporter and use the same item data for the Discovery rule as well as item and trigger prototypes. Check out part 2 of this series to find out how to discover and monitor Podman containers.

The post Podman Container Monitoring with Prometheus Exporter, part 1 appeared first on Zabbix Blog.

Next-Level Alert Analysis with DeepSeek and Zabbix

Post Syndicated from Zhe Cheng original https://blog.zabbix.com/next-level-alert-analysis-with-deepseek-and-zabbix/30424/

As IT infrastructures grow increasingly complex, efficiently analyzing monitoring data and accelerating incident response have become critical challenges for operations teams. This post explores a few innovative applications of DeepSeek when integrated with Zabbix.

Requirements:

– Zabbix server 7.0 or higher
– DeepSeek API (Alternatively, other AI APIs can be used if needed)

1. Scenario One: One-Click Intelligent Alert Analysis

By integrating DeepSeek Analytics into the Zabbix frontend, users can conduct intelligent alert analysis with just one click. This integration facilitates the swift generation of comprehensive fault analyses and solution suggestions, markedly decreasing the MTTR (Mean Time to Resolution). Consequently, it streamlines the troubleshooting process, alleviates the workload on IT personnel, ensures system stability, and conserves both time and resources.

1.1  On the Zabbix home page, navigate to “Alerts” > “Scripts”, and click on the “Create script” button.

1.2  Configuration script:

  • Name: Can be customized
  • Scope: Select “Manual event action”
  • Menu path: Customize menu paths for quick access
  • Type: Select “Script”
  • Execute on: Select “Zabbix proxy or server”

1.3 Enter the following command in the command bar:

/etc/zabbix/scripts/send_alert_to_ai.sh "{TRIGGER.NAME}" "{TRIGGER.SUBJECT}"  "{HOST.NAME}" "{HOST.IP}" "{EVENT.TIME}" "{TRIGGER.SEVERITY}"

1.4  Create an API call script on zabbix-server.

1.4.1 Modify the Zabbix Server Configuration File and Enable Global Scripts:

Open the Zabbix server configuration file for editing:

vi /etc/zabbix/zabbix_server.conf

Set the EnableGlobalScripts option to 1:

EnableGlobalScripts=1

Save the changes and exit the editor. Then, restart the Zabbix server service to apply the changes:

systemctl restart zabbix-server

1.4.2 Create an API Call Script.

Create a directory for custom scripts if it does not already exist:

mkdir -p /etc/zabbix/scripts && cd /etc/zabbix/scripts

Note: If the frontend prompts that the script file cannot be found, try moving the script to the directory used by the Nginx agent. Create a new script file named send_alert_to_ai.sh:

vi send_alert_to_ai.sh

Add the following content to the script, replacing DeepSeek KEY with your actual API key. Make sure you adjust the API call method if using a different AI service:

#!/bin/bash

# DeepSeek API configuration
API_URL="https://api.deepseek.com/chat/completions"
API_KEY="xxxxxxxxxxxxxxxxxxxx"

# Obtain the parameters to be passed as alarm information
TRIGGER_NAME="$1"
ALERT_SUBJECT="$2"
HOSTNAME="$3"
HOST_IP="$4"
EVENT_TIME="$5"
TRIGGER_SEVERITY="$6"

# Build a more concise JSON format for alarm information
alert_info=$(cat <<EOF
{
"model": "deepseek-chat",
"messages": [
{"role": "system", "content": "You are an assistant focused on responding quickly to system alarms。"},
{"role": "user", "content": "The following alarm information is received:\n\n: $TRIGGER_NAME\n: $ALERT_SUBJECT\n: $HOSTNAME\n: $HOST_IP\n: $EVENT_TIME\n: $TRIGGER_SEVERITY\n\nPlease tell me the cause of the alarm and the handling measures in a short and professional language with a word limit of 300 words。"}
],
"stream": false
}
EOF
)

# Send the POST request and capture the response and HTTP status code
response=$(curl -s -w "\n%{http_code}" -X POST "$API_URL" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $API_KEY" \
-d "$alert_info")

# Separate HTTP status codes from response bodies
http_code=$(echo "$response" | tail -n1)
response_body=$(echo "$response" | sed '$d')

# Parse and extract the content field
if [ "$http_code" -eq 200 ]; then
# Parse JSON using the jq tool
if ! command -v jq &> /dev/null; then
echo "jq could not be found, please install it first."
exit 1
fi

# Extract the content field and format the output
content=$(echo "$response_body" | jq -r '.choices[0].message.content')
echo -e "Analysis result:\n$content"
else
echo "failure: HTTP status code $http_code, respond: $response_body"
fi

Make the script executable:

chmod +x send_alert_to_ai.sh

Note: The script provided invokes the official DeepSeek API. Replace DeepSeek KEY with your actual API_KEY. If you are using another AI service, please confirm the appropriate API invocation method.

Important Notes:

Note: The script relies on jq to process and parse JSON data for tasks such as filtering, mapping, aggregating, and formatting. If jq is not installed on your system, follow these instructions to install it.

For Debian/Ubuntu Systems:

apt-get update
apt-get install jq

For CentOS/RHEL Systems:

yum install epel-release
yum install jq

1.5 Actual Effect Display:

1.6  Optional Optimization Items.

1.6.1 Adjust Output Box Size for Better Browsing.

After executing the script, you may find that the output box is too small and inconvenient to browse. To optimize this, you can modify the front-end CSS file as follows.

Back up the existing CSS File:

cd /usr/share/nginx/html/assets/styles/
cp blue-theme.css blue-theme.css.bak

Edit the CSS File:

vi /usr/share/nginx/html/assets/styles/blue-theme.css

Add Custom Styles at the End of the File.

Add the following CSS rules to adjust the size and behavior of the output box:

#execution-output {
height: 500px; /* Adjust to your desired height */
width: 540px; /* Optional: Adjust the width as required */
overflow-y: auto; /* Displays scrollbar when content exceeds the set height */
}

Save and exit the editor. At this point, clear the browser cache and reload the page to see the changes take effect.

1.6.2 How to Optimize Slow Output Response after Executing the One-Click Analysis Script.

During actual testing, it was estimated that returning a 300-word result takes approximately 20 to 30 seconds. While you can improve the response speed by adjusting the preset prompt words in the script, this approach may reduce the richness of the analysis content. Therefore, it is recommended to balance speed and content depth by adjusting the number of replies in the script’s prompt words according to your actual needs.

Actual effect display:

2. Scenario Two: Zabbix Documentation Knowledge Base Assistant

In today’s fast-paced IT environment, managing and retrieving information efficiently is crucial. To address this need, we’ve developed the Zabbix KB Assistant, an intelligent knowledge base solution built on MaxKB—an open-source Q&A system leveraging large language models.

This assistant streamlines access to Zabbix’s extensive documentation, making it easier than ever for users to find the information they require.
MaxKB stands out for its seamless integration capabilities, allowing for quick uploads of documents and automatic crawling of online content.

Its flexibility means it can be effortlessly embedded into third-party systems, including our very own Zabbix platform. The project is available at the GitHub repository.

The development process of Zabbix KB Assistant involved configuring MaxKB to recognize and parse the official Zabbix documentation. By utilizing this URL, we ensured that the latest updates and comprehensive guides are always accessible within our assistant. After setting up the core model configurations, we created a dedicated knowledge base tailored to Zabbix’s rich content.

With the knowledge base in place, we proceeded to integrate Zabbix KB Assistant into the Zabbix frontend. This step was essential for providing instant access to users navigating the Zabbix interface. By embedding a floating window mode, users can interact with the assistant without leaving their current page—a feature that significantly enhances user experience.

Actual effect display:

3. Scenario Three: DingTalk Alert Enhancement

By integrating DeepSeek’s deep analysis capabilities, DingTalk can automatically analyze alarm information upon receiving alerts. This integration provides precise fault diagnosis and solutions, aiding IT operations and maintenance personnel in quickly identifying and resolving issues. Consequently, this improves the efficiency of system maintenance and reduces downtime.

3.1 Create a Bot and Configure Security Settings.

First, create a new bot within the DingTalk group and ensure that the keyword “Alarm” is properly configured in the security settings. Next, retrieve the webhook URL for this bot and keep it safe for later use.

3.2 Install Python3 and Necessary Libraries.

Ensure that Python3 along with the required libraries are installed on your system. Depending on your operating system, follow these instructions.

For Ubuntu/Debian systems:

sudo apt update
sudo apt install python3 python3-pip
pip3 install requests

For CentOS/RHEL systems:

sudo dnf install python3
pip3 install requests

3.3 Below is an example script (deepseekdingding.py) located at /usr/lib/zabbix/alertscripts/.

Replace the placeholder webhook URL and DeepSeek API key in the script with your actual values:

#!/usr/bin/env python3
#coding:utf-8
import requests
import sys
import json

class DingTalkBot(object):
    # Send an alarm
    def send_news_message(self, webhook_url, subject, content, ai_response):
        url = webhook_url
        data = {
            "msgtype": "markdown",
            "markdown": {
                "title": subject,
                "text": f"{subject}\n{content}\n\n【DeepSeek analysis】:\n\n{ai_response}"  
            }
        }
        headers = {'Content-Type': 'application/json'}
        response = requests.post(url, headers=headers, data=json.dumps(data))
        return response

if __name__ == '__main__':
    WEBHOOK_URL = 'https://oapi.dingtalk.com/robot/send?access_token=224c1ff0c6df60a809b3c5b69b8448486b780d292e9d395ac8fbf84980214e30'  # Webhook
    API_URL = 'https://api.deepseek.com/chat/completions'
    API_KEY = "xxxxxxxxxxxxxxxxxxxx"  # DeepSeek API

    if len(sys.argv) < 3:
        print("Error: Not enough arguments provided.")
        sys.exit(1)

    subject = str(sys.argv[1])  
    content = str(sys.argv[2])  

    print(f"Received subject: {subject}")
    print(f"Received content: {content}")

    try:
        headers = {
            'Authorization': f'Bearer {API_KEY}',
            'Content-Type': 'application/json',
        }
        payload = {
            "model": "deepseek-chat",  # DeepSeek
            "messages": [
                {"role": "user", "content": f"If you are a professional IT operation and maintenance expert, please tell me the cause of these alarms and handling suggestions in a concise and professional language with a word limit of 100 words{content}"}
            ]
        }
        ai_response = requests.post(API_URL, headers=headers, json=payload)
        ai_response.raise_for_status()  
        ai_response_content = ai_response.json().get('choices', [{}])[0].get('message', {}).get('content', '')
    except Exception as e:
        ai_response_content = "\nThe interface call timed out or an error occurred. Please check the configuration and try again"

    bot = DingTalkBot()
    response = bot.send_news_message(WEBHOOK_URL, subject, content, ai_response_content)

    if response.status_code == 200:
        print("successfully")
    else:
        print(f"failed: {response.text}")

3.5 On the Zabbix home page, go to Alerts – Media types – Create Media type and then enter the following information:

  • Name: aiAlarm-Dingtalk
  • Type: script
  • Script name: deepseekdingding.py
  • Script parameter: {ALERT.MESSAGE} {ALERT.SUBJECT}

3.6 Create an alarm action.

Go to Alarm – Action – Trigger actions – Create action and set the name to Alarm -deepseek. Select this parameter as required:

Edit the action options as follows:

Send to media type aiAlarm-Dingtalk
Topic fault alarm: {EVENT.NAME}
message
【Zabbix Alarm Notification 】

Alarm group: {TRIGGER.HOSTGROUP.NAME}

Alarm host: {HOSTNAME1}

Alarm time: {EVENT.DATE} {EVENT.TIME}

Alert level: {TRIGGER.SEVERITY}

Problem information: {TRIGGER.NAME}

Confirm the update.

3.7 Configure notification rights for users.

The following item is added to the “User-User-Alarm” media dialog box. Once added, click Update.

Actual effect display:

4. Scenario Four: One-Click System Service Deep Analysis

Our solution integrates DeepSeek analysis to offer a one-click intelligent inspection tool that automates the collection of service configurations, logs, and status from within your system. This information is then sent via API to DeepSeek for comprehensive analysis.

Our approach begins by extracting relevant configuration data, recent log entries, and current service statuses. These pieces of information are combined with predefined prompts and submitted to DeepSeek through its API. For instance, a prompt might look like this:

“Here are the current logs for XXX service:\n\n${recent_logs}\n\nService. Status is as follows:\n${service_status}\n. Please analyze the following four aspects based on this information and provide a concise report within 500 words: service status analysis, configuration review, historical issue examination, and troubleshooting recommendations.”

DeepSeek processes this input to perform a detailed breakdown across these four areas, delivering structured feedback and actionable insights.
This integration offers deep system analysis and precise optimization suggestions, enabling swift responses to system changes or anomalies. It aids administrators in promptly identifying and addressing issues.

In addition, it’s easily integrated into existing monitoring systems, allowing adjustments to the depth and scope of analysis as needed. The solution boasts high scalability and flexibility, catering to evolving business requirements.

Actual effect display :

The post Next-Level Alert Analysis with DeepSeek and Zabbix appeared first on Zabbix Blog.

Let Zabbix be your Lucky Lady for Lotto Numbers

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/let-zabbix-be-your-lucky-lady-for-lotto-numbers/30251/

Many years ago, maybe around 2017 or 2018, one of my ex-colleagues (Hi, Kevin!) said that I would probably even use Zabbix to come up with the winning lotto numbers. Just to strike back, I did exactly that with a small “easter egg” in Zabbix containing the lotto numbers – a quick bash script feeding the Zabbix item.

Let’s return to that topic, but use a Zabbix Script item type instead. Also, let’s take a look at few other details that help with monitoring.

Let’s create a host and a new template

To begin with, I created a new template and a new host. Here’s the host, nothing more needed than a name and my fancy template:

The template has only one item:

For the script, here’s what ChatGPT came up with. JavaScript is not my strongest skill, so for a fun little experiment this AI vibe coding should be good enough.

// Generate an array of numbers from 1 to 40
var numbers = [];
for (var i = 1; i <= 40; i++) {
   numbers.push(i);
}

// Shuffle the array using the Fisher-Yates algorithm
for (var i = numbers.length - 1; i > 0; i--) {
   var j = Math.floor(Math.random() * (i + 1));
   var temp = numbers[i];
   numbers[i] = numbers[j];
   numbers[j] = temp;
}

// Select the first 7 numbers from the shuffled array
var lotto = numbers.slice(0, 7);

// Optionally sort the selected numbers
lotto.sort(function(a, b) { return a - b; });

// Return the lottery numbers as a space-separated string
return lotto.join(" ");

So, that’s it! Well, almost – to stop Zabbix from coming up with new numbers all the time, here’s a very nice feature of Zabbix.

Custom intervals

If you set your Update interval to 0, you can use Custom intervals. This way, the new numbers will only be generated once per week every Monday at 8:00 to kick off your work week (assuming that your country has the lotto only once per week, of course).

Naturally, in the actual business world, this kind of exact scheduled monitoring can be extremely helpful as well. If you have something you don’t need to check all the time but only during business days and hours, or only on weekends, or only once per day, this is a handy way of doing it.

Does it work?

You know the answer — of course it does! Now when I search for “lotto”, click on “Latest data” and force the check to happen immediately by clicking on “Execute now”, this happens.

Time to dashboard it

I could peek at the values from Latest data, but that would be boring. With a dashboard, it’s a bit more entertaining…

I hope this post gave you some new ideas or maybe even introduced you to Script item type. If you win some major money with this trick, don’t forget to buy me a coffee!

 

The post Let Zabbix be your Lucky Lady for Lotto Numbers appeared first on Zabbix Blog.

Optimizing Incident Management with Zabbix and PagerDuty

Post Syndicated from Zabbix LatAm original https://blog.zabbix.com/optimizing-incident-management-with-zabbix-and-pagerduty/30114/

When monitoring environments, we sometimes need to rely on third-party tools to better manage functionality and optimize responses to alerts. Let’s explore how to integrate Zabbix with PagerDuty, a real-time incident management solution designed to improve the reliability of digital services, including best practices and configuration details.

What is PagerDuty?

PagerDuty is a real-time incident management platform designed to help IT teams react quickly to critical events. The tool helps organizations automate and manage incident response through a system of alerts, escalation, and coordination between teams. When a problem is detected in the system, PagerDuty notifies the responsible individuals and ensures that corrective action is taken quickly. This reduces downtime and improves operational efficiency. Integration with monitoring tools such as Zabbix makes it easy to identify issues before they impact users.

Some of PagerDuty’s key features include:

• Integration with monitoring tools (such as Zabbix)
• Notifications in multiple channels (email, SMS, calls)
• Automatic escalation of incidents to ensure agile responses
• Event analysis to improve the detection of recurring problems

How to integrate PagerDuty with Zabbix

In PagerDuty, go to “Services” and click on “Service Directory.” Create a new service.

Give it a proper name and description.

Accept the escalation terms and click “Next.”

On the next screen, select “Intelligent” and the “Auto-pause incident notifications” option, then click “Next.”

The next step is to add the Zabbix Webhook service, which will allow integration with Zabbix, and then click “Next.”

In Services > Service Directory, select the name of the service. In the “Integrations” tab, copy the integration token that is generated.

It is important to note that the PagerDuty webhook only shows the option of Zabbix versions 5.0 to 5.2, but it works correctly in later versions such as Zabbix 7.2, which was tested without any issues.

On Zabbix Server, go to Alerts > Media types > PagerDuty. Enter the integration token, the Zabbix URL, and select “Update.”

Send a test message to confirm that the integration is working correctly.

In the PagerDuty application, verify that the test alert was received.

To send notifications, you need to grant permissions to a user in Zabbix. Go to Users > Create User. In the “Media” tab, select PagerDuty as the notification method. Set the severity of the alerts you want to receive.

Subsequently, set up a Trigger Action in Alerts > Actions > Trigger Actions to define what types of alerts will be received (either by item or trigger) according to the needs of your team.

Best practices for integrating Zabbix and PagerDuty

Customize notifications: Set rules to send only truly critical alerts, avoiding unnecessary notifications.
Optimize escalations: Set up escalation rules so that alerts reach the right people at the right time.
Monitor key metrics: Measure incident response times and adjust workflows as needed.
Automate incident responses: Use PagerDuty’s capabilities to perform automated tasks in response to specific events.
Notify about service failures: Use PagerDuty to start running recovery scripts, send notifications to the responsible teams, or even escalate the problem to a higher level if there is no solution in a stipulated length of time.

Conclusion

Zabbix’s integration with PagerDuty allows you to monitor the status of critical services in real time, even outside of working hours. This facilitates rapid incident response and improves your IT team’s ability to react.

This combination not only optimizes incident management but also helps minimize downtime, improve operational efficiency, and ensure the reliability of monitored systems.

With proper configuration and best practices, integrating Zabbix with PagerDuty can become essential for the proactive management of your technological infrastructure.

 

 

 

 

 

The post Optimizing Incident Management with Zabbix and PagerDuty appeared first on Zabbix Blog.

Build a Culture of Monitoring and Get Buy-In with Zabbix

Post Syndicated from Michael Kammer original https://blog.zabbix.com/build-a-culture-of-monitoring-and-get-buy-in-with-zabbix/30085/

In today’s fast-paced, interconnected IT world, simply waiting for something to fail before fixing it isn’t good enough. A proactive approach to monitoring, which aims to identify and address potential issues before they escalate into major disruptions, is a necessity rather than a luxury.

Here at Zabbix, we’ve got plenty of reason to believe that we offer the most flexible monitoring solution available on the market today. However, choosing the best monitoring tool for your organization’s needs is only half the battle – you also need to get buy-in from team members who may not understand the need for monitoring, may be fearful of and resistant to change, and may not be familiar with the technologies behind monitoring.

In this post, we’ll take a look at a few strategies you can use to help win over lukewarm or hesitant colleagues and build a culture of monitoring. We’ll also explore how choosing Zabbix for your monitoring needs can make each strategy a bit easier to implement.

Strategy 1: Explain the “why”

One of the first questions that you can anticipate during any change initiative is simply, “what for?” The ethos of “don’t fix what isn’t broken” runs strong in the tech community, and unless you go above and beyond to explain why monitoring matters, your team will remain skeptical.

Zabbix can help you make your case by providing you with the evidence you need to bolster your case. We’ve got plenty of testimonials available from tech communities worldwide (including PeerSpot, Gartner, and Capterra), and no matter what field you’re in or how big your team is, we’ve most likely got a case study or two available that shows how monitoring with Zabbix was a game changer for a company like yours.

All of this should help you explain the rationale for the change in an open and transparent way. When it comes to monitoring, sharing details on costs, expected benefits, and what will happen if no change is made will build understanding around why monitoring is necessary and why monitoring with Zabbix is the right answer for your team’s needs.

Strategy 2: Show your team what’s in it for them

One of the most effective ways to get employee buy-in for monitoring is by highlighting the benefits it will bring to individual employees. Show how monitoring can simplify their tasks, improve efficiency, and enhance their work experience, and give them concrete examples of how the technology can make their jobs easier or help them to deliver better results.

We recently had a large managed services provider (MSP) use our monitoring solution as a true “force multiplier”, allowing them to monitor their systems, automate tasks based on real-time events, and provide immediate responses to issues without manual intervention. Thanks to Zabbix, their engineers report higher job satisfaction thanks to no longer having to be “on call” at all hours to solve simple issues, while management has seen productivity skyrocket thanks to their team’s newfound ability to find potential issues before they become real problems.

Strategy 3: Turn important stakeholders into monitoring champions

Determine who monitoring will impact and who needs to be kept informed. This might be team leaders, IT staff, end users, and/or an executive sponsor. Getting input from these groups early on will help you anticipate needs and concerns, and you’ll also want to identify influential employees who are enthusiastic about monitoring and get them to help you promote it.

A great way to help them do so is by encouraging them to attend one (or more) Zabbix events – we’ve got free meetings, online meetups, regional conferences, or even our yearly Summit in Latvia. No matter where you happen to be located, there’s a pretty good chance that we’ll soon be bringing your key people a chance to network with like-minded professionals from multiple industries, expand their knowledge, get answers to their questions, and explore how Zabbix can work for them.

Strategy 4: Provide adequate training

Equipping employees with the skills and knowledge they need to get the most out of a monitoring system means gaining a solid understanding of their current capabilities and then finding out which gaps you most urgently need to fill. Chances are, you’ll need to provide guidance, documentation, hands-on demonstrations, and access to experts – and this is another area where Zabbix has you covered.

Zabbix Certified trainings are designed to help your people learn Zabbix inside and out, giving them the practical knowledge they’ll need to increase their productivity and performance. When you explore our training options, you’ll find a wide variety of courses, everything from one-day sessions that cover the basics to week-long sessions that guarantee users the ability to tackle any Zabbix challenge on their own.

In addition, we’ve got plenty of other free resources available to teams and individuals looking to upskill, including our famously active forum, blog, webinars, and newsletter.

Conclusion

Building a culture of monitoring requires commitment from every level of an organization. By choosing Zabbix as the guide to your monitoring journey and following the strategies outlined in this article, you and your team can successfully implement and maintain a robust monitoring strategy that will help you achieve your organization’s IT goals.

To learn more about what Zabbix can do for you, visit our website.

The post Build a Culture of Monitoring and Get Buy-In with Zabbix appeared first on Zabbix Blog.

Enhancing Visualizations in Zabbix with the ECharts Module

Post Syndicated from Matheus da Silva Andrade original https://blog.zabbix.com/enhancing-visualizations-in-zabbix-with-the-echarts-module/30199/

One of the great advantages of Zabbix is its extensible and modular architecture. This allows the platform to be enhanced with third-party modules, significantly expanding its functionalities without compromising the stability of the core system. The ECharts-Zabbix module is an excellent example of this flexibility in action.

What is the ECharts-Zabbix module?

ECharts-Zabbix is a module that adds customizable widgets to Zabbix, using the ECharts library to create interactive and dynamic visualizations of your monitoring data. This module complements Zabbix’s standard visual capabilities, enabling richer and more informative graphical representation of complex monitoring environments.

What are the key features available with ECharts in Zabbix?

By integrating ECharts and Zabbix, you gain access to:

  • Multiple chart types (line, bar, pie, gauge, scatter, heatmap, and more)
  • Complete customization of colors, styles, legends, and tooltips
  • Fluid animations for a better user experience
  • Compatibility with Zabbix light and dark themes
  • Direct integration with data without the need for external tools
  • Responsive visualizations that adapt to different screen sizes
  • Helper functions for data formatting and dynamic color generation

Installation and configuration

Installing modules in Zabbix is easy thanks to the platform’s flexibility:

  1. Download the module from the official repository
  2. Extract the files to the modules folder of your Zabbix server
  3. In the Zabbix frontend, go to Administration > General > Modules
  4. Find the ECharts-Zabbix module in the list and click “Enable”
  5. The widget will be available for use in Zabbix dashboards and screens

Practical use cases

Server performance monitoring with Gauge charts

Gauge charts are ideal for visualizing metrics such as CPU, memory, and disk usage. The flexibility of Zabbix combined with ECharts allows you to create impressive visual panels that clearly show the current state of these metrics:

```javascript

const field = context.panel.data.series[0].fields[0];

const value = field.value;

const gaugeData = [{

  value: value,

  name: field.name,

  title: {

    offsetCenter: ['0%', '30%']

  },

  detail: {

    offsetCenter: ['0%', '60%']

  }

}];

return {

  backgroundColor: 'transparent',

  series: [{

    type: 'gauge',

    startAngle: 90,

    endAngle: -270,

    center: ['50%', '50%'],

    radius: '90%',

    pointer: {

      show: false

    },

    progress: {

      show: true,

      overlap: false,

      roundCap: true,

      clip: false,

      itemStyle: {

        borderWidth: 0

      }

    },

    axisLine: {

      lineStyle: {

        width: 20,

        color: [[1, 'rgba(255,255,255,0.1)']]

      }

    },

    splitLine: {

      show: false

    },

    axisTick: {

      show: false

    },

    axisLabel: {

      show: false

    },

    data: gaugeData,

    title: {

      fontSize: 14,

      fontWeight: 'normal'

    },

    detail: {

      width: 80,

      height: 20,

      fontSize: 14,

      fontWeight: 'normal',

      borderWidth: 0

    }

  }]

};

```

Liquid fill chart example

This chart type is great for visualizing percentage-based metrics, like disk usage or SLA compliance, in a visually appealing way:

```javascript

if (!context.panel.data.series || !context.panel.data.series[0] || !context.panel.data.series[0].fields) {

    console.error('Dados não disponíveis no formato esperado');

    return {};

}

const field = context.panel.data.series[0].fields[0];

return {

    backgroundColor: 'transparent',

    series: [{

        type: 'liquidFill',

        data: [field.value / 100],

        radius: '80%',

        color: ['#91cc75'],

        backgroundStyle: {

            color: 'rgba(255, 255, 255, 0.1)'

        },

        label: {

            formatter: function() {

                return field.name + '\n' + field.value.toFixed(2) + field.units;

            },

            fontSize: 28,

            color: 'black'

        },

        outline: {

            show: false

        }

    }]

};

```

 

Below are some other visualization examples available on our github:

Colors and gradients

You can use simple hexadecimal colors or create sophisticated gradients:

```javascript

// Linear gradient

new echarts.graphic.LinearGradient(0, 0, 0, 1, [

  { offset: 0, color: '#83bff6' },

  { offset: 1, color: '#188df0' }

])

```

Number formatting

Format your numerical data as needed:

```javascript

// 2 decimal places

formatter: function(value) {

  return value.toFixed(2) + field.units;

}

// Using context helper

formatter: function(value) {

  return context.helpers.formatNumber(value, 2) + field.units;

}

```

Element positioning

Precisely control where elements are displayed:

```javascript

// Centered

offsetCenter: [0, '70%']

// Custom grid

grid: {

  top: '5%',

  left: '3%',

  right: '4%',

  bottom: '3%',

  containLabel: true

}

```

The Zabbix module ecosystem

Zabbix has a growing ecosystem of modules and integrations, developed by both the community and specialized companies like Monzphere, which contributes the ECharts-Zabbix module. This development dynamic demonstrates how Zabbix has evolved to become a truly extensible platform.

To learn more about the ECharts-Zabbix module and other solutions for Zabbix, you can visit our official GitHub repository or Monzphere’s website.

Conclusion

Zabbix’s modular architecture is one of its greatest differentiators, allowing the platform to grow and adapt to the specific needs of each monitoring environment. The ECharts-Zabbix module is an excellent example of how this flexibility can be leveraged to transform the data visualization experience in Zabbix.

For modern monitoring environments where clear and effective data visualization is essential, the combination of Zabbix with specialized modules represents a complete and adaptable solution. Try expanding your Zabbix with the ECharts module and discover how it can transform your monitoring dashboards!

The post Enhancing Visualizations in Zabbix with the ECharts Module appeared first on Zabbix Blog.

Deploying Zabbix Components with Docker and Docker Compose

Post Syndicated from Janis Eidaks original https://blog.zabbix.com/deploying-zabbix-components-with-docker-and-docker-compose/30025/

Installing Zabbix from packages can feel overwhelming, due to the availability of different configuration options. The detailed and comprehensive documentation certainly helps to check the purpose of these multiple options, what values can be set in their fields, and if one is required for your planned deployment. There are quite a few official Zabbix blog posts about Zabbix in containers, and this post is aimed at showcasing how additional Zabbix components can be easily set up in a docker environment, along with docker run and docker compose examples.

For those who would prefer to use Zabbix in a containerized environment such as Docker, or who want to try out Zabbix quickly, this guide is for you (you can also check out the other Zabbix Docker blog posts). You can also mix and match Zabbix components installed from packages or built from source with those running in containers.

Please follow the official guide on how to set up the docker here.

To better understand the Zabbix architecture for those who are trying out Zabbix for the first time, I will give you an overview that should make it much easier to follow and understand Zabbix.

Zabbix consists of 3 main components (the bare minimum to get started):

  • Zabbix Server – responsible for everything related to data collection, trigger evaluation, event generation, and alerting.
  • Zabbix Frontend – responsible for the configuration (modifying or changing the configuration of the monitoring targets) and visualization (dashboards, graphs, tables, and widgets).
  • Database – this is where the Zabbix configuration and monitoring history data are stored.

You can monitor your targets with the bare minimum setup; however, more comprehensive and complete monitoring can be achieved by using the C-based Zabbix-agent or GO-based Zabbix-agent2 in combination with templates, user parameters, and more. To set up the minimum necessary Zabbix components, you can use this example in the guide.

There are also official guides available on the Zabbix documentation page (for both: the docker run and docker compose) or the Docker/Github.

As of this writing , these official Zabbix docker components are available from the docker hub page:

  • Zabbix Server (with MySQL/PostgreSQL database)
  • Zabbix Proxy (with MySQL/SQLite3 database)
  • Zabbix Frontend (Apache/Nginx with MySQL/PostgreSQL DB)
  • Zabbix Agent (TLS encryption)
  • Zabbix Agent2 (TLS encryption)
  • Zabbix Java Gateway
  • Zabbix SNMP traps
  • Zabbix Web Service

Tags are used to select which OS container an image will be based on, as well as which Zabbix component version you wish to employ. If you only specify tag value – latest, you will get the latest Zabbix version based on the Alpine Linux. The images based on Linux Alpine are more lightweight than the other distros.

When something does not work as expected or fails, check the container error logs! This will be useful for debugging purposes and will help to narrow down the cause of an issue. Additionally, when debugging you can also specify additional options, such as specific lines of log, timestamp since or until, or following the log file content.

# docker logs --tail 50 container_name_or_id
    --details        Show extra details provided to logs
-f, --follow         Follow log output
    --since string   Show logs since timestamp (e.g. "2013-01-02T13:23:37Z") or relative (e.g. "42m" for 42 minutes)
-n, --tail string    Number of lines to show from the end of the logs (default "all")
-t, --timestamps     Show timestamps
    --until string   Show logs before a timestamp (e.g. "2013-01-02T13:23:37Z") or relative (e.g. "42m" for 42 minutes)

In some rare cases, when there is a container issue (everything else is correct, worked before, etc.), restarting the docker service can sometimes solve the issue.

So, what is different if you have only used Zabbix installed from packages?  The examples below illustrate the differences in configuration options based on different Zabbix deployment methods: a) package-based/compiled installation, b) docker run command, and c) docker compose file example. First of all, you will have to specify environment variables in the docker run command or docker compose file. The list of available environment variables for each docker image is available in both docker hub and Github.

A). Package-based config

# vi /etc/zabbix/zabbix_server.conf
...
DBName=zabbix
DBUser=zabbix_usr
DBPassword=zabbix_pwd
...

B).Docker run config

docker run --name zbxsrv -t \
...
-e MYSQL_DATABASE=zabbix\
-e MYSQL_USER=zabbix_usr \
-e MYSQL_PASSWORD=zabbix_pwd\
...

C). Docker compose config

# vi /../...yaml
...
  environment:
   MYSQL_DATABASE=zabbix
   MYSQL_USER=zabbix_usr
   MYSQL_PASSWORD=zabbix_pwd

The environment variables are represented as key-value pairs, e.g., VAR=VAL. The values can optionally be unquoted or double-quoted. If some environment variable value contains special characters, you will need to escape them. To properly escape them, check out the docker documentation page.

You can create custom, user-defined networks to connect multiple containers to the same network. On such networks, containers can resolve each other by name or alias. If needed, you can assign a specific IP address to a container (if the address is already used, you will get an error).

# docker network create --subnet 172.20.0.0/16 --ip-range 172.20.240.0/20 zabbix-net

Docker run

In this section, we have an example of docker run commands for two Zabbix components: Zabbix proxy and Java gateway. When using custom, user-defined networks, you can use container names for communication between containers instead of using IP addresses. Here, instead of defining the IP address for Zabbix Java gateway, the container name is used. You can set a static IP address for your container or let docker do it for you, but confirm if the change of the IP address will not cause issues in case your container gets a different IP address. This can become an issue if you use an IP address in some configuration fields instead of a container name.

A lot of parameters are specified using environment variables with the option -e. Also, 3 different ports are exposed on your host machine. To keep the SQLite3 database file upon container deletion, the container directory containing database file is mounted to host directory (the proxy DB is usually used as a buffer storage before sending data to Zabbix server and usually is not used to store data beyond the moment when the data is sent).

docker run --name zabbix-proxy-active-01 \
-e ZBX_HOSTNAME="Zabbix-proxy-active-01" \
-e ZBX_SERVER_HOST=46.101.140.98 \
-e ZBX_PROXYMODE="0" \
-e ZBX_JAVAGATEWAY_ENABLE=true \
-e ZBX_JAVAGATEWAY=zabbix-java-gateway-proxy \
-e ZBX_JAVAGATEWAYPORT=10052 \
-e ZBX_STARTJAVAPOLLERS=5 \
--network=zabbix-net \
-e ZBX_LISTENPORT=10101  \
-p 10101:10101 \
-p 10050:10050 \
-p 10051:10051 \
-v /var/lib/zabbix/db_data:/var/lib/zabbix/db_data \
--restart unless-stopped \
--init -d zabbix/zabbix-proxy-sqlite3:alpine-7.2.4
docker run --name zabbix-java-gateway-proxy \
--network=zabbix-net \
--restart unless-stopped \
-d zabbix/zabbix-java-gateway:alpine-7.2.4

You can start each of these Zabbix components using the docker run command, however, any change to the container configuration will require you to stop the container, delete it, and execute the docker run command again. You also have another option – you could create a docker compose file and write the necessary configuration in yaml format. When you need to add some changes to the container configuration, run the docker compose down command to remove containers, edit the docker compose file, and run docker compose up command to start them up again with the new configuration:

  • docker compose -f ./docker_compose_v3_proxy.yaml down
  • docker compose -f ./docker_compose_v3_proxy.yaml up -d

If you have not mounted volume or directory to container for the data you want to keep, you can copy the data from the container to your host. Otherwise, that data will be gone if you delete the container or use the docker compose down command. So, it is important to set up the persistent storage/volume for the data that needs retaining, so you don’t lose important data from the container when container configuration is changed. You also need to expose the ports for the necessary services for the appropriate components (if they are set up on on separate hosts): zabbix-server, zabbix-proxy, zabbix-agent/zabbix-agent2 (default ports: 10050 for Zabbix agent passive mode, 10051 for Zabbix-agent active mode, some different port for proxy, 10052 for Java gateway).

Here we have the same docker run options written to docker compose file, including the environment variables, mounted directories and exposed ports. You can specify as many services as needed and start them just with docker compose command.

docker_compose_v3_proxy.yaml

services:
  zabbix-proxy-active-01:
    image: "${PROXY_SQLITE3_IMAGE}:${ALPINE_IMAGE_TAG}"
    environment:
      ZBX_HOSTNAME: Zabbix-proxy-active-01
      ZBX_SERVER_HOST: ${ZBX_SERVER_HOST}
      ZBX_PROXYMODE: 0
      ZBX_LISTENPORT: 10101
      ZBX_JAVAGATEWAY_ENABLE: true
      ZBX_JAVAGATEWAY: zabbix-java-gateway-proxy
      ZBX_JAVAGATEWAYPORT: 10052
      ZBX_STARTJAVAPOLLERS: 5
    volumes:
      - /var/lib/zabbix/db_data:/var/lib/zabbix/db_data:rw
    networks:
      - backend
    ports:
      - 10101:10101
      - 10050:10050
      - 10051:10051
    restart: unless-stopped

  zabbix-java-gateway-proxy:
    image: "${JAVA_GW_IMAGE}:${ALPINE_IMAGE_TAG}"
    networks:
      - backend
    restart: unless-stopped

networks:
  backend:
    name: zabbix-net
    external: true

.env

PROXY_SQLITE3_IMAGE=zabbix/zabbix-proxy-sqlite3
JAVA_GW_IMAGE = zabbix/zabbix-java-gateway
ALPINE_IMAGE_TAG=alpine-7.2.4
ZBX_SERVER_HOST=46.101.140.98

You can also use official Zabbix-supplied docker compose files, try them out, and modify them as needed.

You can read more about the official docker compose files here.

Containerized Zabbix components allow us to use test different scenarios within the docker:

  • Creating HA Zabbix-server nodes
  • Creating multiple proxies
  • Creating multiple agents
  • Adding more Java gateways
  • Creating multiple frontends
  • Easily configure Browser monitoring
  • Configure SNMP traps
  • Easily make scheduled reports

Deploying multiple redundant Zabbix servers

To enable HA Zabbix server mode, modify both the Zabbix-server container and Zabbix-frontend container configuration environment variables.

For the HA Zabbix server mode, add 2 environment variables:

  • ZBX_HANODENAME
  • ZBX_NODEADDRESS

All of the containers are set with the user-defined network, therefore I will use the container name in the ZBX_HANODENAME option instead of the static address, as it will be resolved by docker. If you need to use a different listen port for the trapper, you need to define it using the environment variable ZBX_LISTENPORT. You can omit the port in variable ZBX_HANODENAME, as the ZBX_LISTENPORT (default is 10051) will be applied automatically.

Here is the docker run example for the Zabbix-server HA mode.

docker run --name zabbix-server-mysql-ha1 -t \
-e DB_SERVER_HOST="mysql-server" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="zabbix_pwd" \
-e MYSQL_ROOT_PASSWORD="root_pwd" \
-e ZBX_HANODENAME="zabbix-server-HA1" \
-e ZBX_NODEADDRESS="zabbix-server-mysql-ha1" \
--network=zabbix-net \
-p 10151:10051 \
--restart unless-stopped \
-d zabbix/zabbix-server-mysql:alpine-7.2.4
docker run --name zabbix-server-mysql-ha2 -t \
-e DB_SERVER_HOST="mysql-server" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="zabbix_pwd" \
-e MYSQL_ROOT_PASSWORD="root_pwd" \
-e ZBX_HANODENAME="zabbix-server-HA2" \
-e ZBX_NODEADDRESS="zabbix-server-mysql-ha2" \
--network=zabbix-net \
-p 10251:10051 \
--restart unless-stopped \
-d zabbix/zabbix-server-mysql:alpine-7.2.4

From the frontend container, remove these two environment variables:

  • ZBX_SERVER_HOST
  • ZBX_SERVER_PORT
docker run --name zabbix-web-nginx-mysql -t \
-e ZBX_SERVER_HOST="zabbix-server-mysql" \
-e ZBX_SERVER_PORT=10051
-e DB_SERVER_HOST="mysql-server" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="zabbix_pwd" \
-e MYSQL_ROOT_PASSWORD="root_pwd" \
--network=zabbix-net \
-p 80:8080 \
--restart unless-stopped \
-d zabbix/zabbix-web-nginx-mysql:alpine-7.2.4

Once both container configurations are modified, you should be able to see the currently added HA server nodes and their states without issues.

Fig. 1. Containers of HA Zabbix server containers

Fig. 2. Dashboard – system information

You can also execute commands on the container:

# docker exec -it container_name_or_id sh -c "zabbix_server -R ha_status"

Fig. 3. Executing command on container

Containers of HA Zabbix server containers

I’t’s possible to allocate an interactive pseudo-TTY shell, by adding option -ti and specifying shell after the container name or id.

# docker exec -ti container_name_or_id /bin/bash

Fig. 4. Executing command from within container

You can also start multiple proxies at once in docker. This can help to offload preprocessing to the proxy, gather data from the targets behind the firewall, and send collected data back to the Zabbix server, only requiring one port.

Fig. 5.Overall block diagram of Zabbix monitoring opportunities

Deploying multiple Zabbix proxies

First, you must choose the proxy mode and set the environment variable ZBX_PROXYMODE.

For active mode proxy, please define the server host address for a single server or addresses separated by a semicolon in the case of HA Zabbix server configuration (example shown below).

docker run --name zabbix-proxy-active-01 \
-e ZBX_HOSTNAME="Zabbix-proxy-active-01" \
-e ZBX_SERVER_HOST="zabbix-server-mysql-ha1;zabbix-server-mysql-ha2;zabbix-server-mysql-ha3" \
-e ZBX_PROXYMODE="0" \
--network=zabbix-net \
-e ZBX_LISTENPORT=10101  \
-p 10101:10101 \
-v /var/lib/zabbix/db_data:/var/lib/zabbix/db_data \
--restart unless-stopped \
--init -d zabbix/zabbix-proxy-sqlite3:alpine-7.2.4

For passive mode proxy, define the server host address for a single server or addresses separated by a comma in the case of HA Zabbix server configuration (example shown below).

docker run --name zabbix-proxy-passive-01 \
-e ZBX_HOSTNAME="Zabbix-proxy-passive-01" \
-e ZBX_SERVER_HOST="zabbix-server-mysql-ha1,zabbix-server-mysql-ha2,zabbix-server-mysql-ha3" \
-e ZBX_PROXYMODE="1" \
--network=zabbix-net \
-e ZBX_LISTENPORT=10102 \
-p 10102:10102 \
-v /var/lib/zabbix/db_data:/var/lib/zabbix/db_data \
--restart unless-stopped \
--init -d zabbix/zabbix-proxy-sqlite3:alpine-7.2.4

docker_compose_v3_proxies.yaml

services:
  zabbix-proxy-active-01:
    image: "${PROXY_SQLITE3_IMAGE}:${ALPINE_IMAGE_TAG}"
    environment:
      ZBX_HOSTNAME: zabbix-proxy-active-01
      ZBX_SERVER_HOST: zabbix-server-mysql-ha1;zabbix-server-mysql-ha2;zabbix-server-mysql-ha3
      ZBX_PROXYMODE: 0
      ZBX_LISTENPORT: 10101
    volumes:
      - /var/lib/zabbix/db_data:/var/lib/zabbix/db_data:rw
    networks:
      - backend
    ports:
      - 10101:10101
    restart: unless-stopped

  zabbix-proxy-passive-01:
    image: "${PROXY_SQLITE3_IMAGE}:${ALPINE_IMAGE_TAG}"
    environment:
      ZBX_HOSTNAME: zabbix-proxy-passive-01
      ZBX_SERVER_HOST: zabbix-server-mysql-ha1,zabbix-server-mysql-ha2,zabbix-server-mysql-ha3
      ZBX_PROXYMODE: 1
      ZBX_LISTENPORT: 10102
    volumes:
      - /var/lib/zabbix/db_data:/var/lib/zabbix/db_data:rw
    networks:
      - backend
    ports:
      - 10102:10102
    restart: unless-stopped
networks:
  backend:
    name: zabbix-net
    external: true

.env

PROXY_SQLITE3_IMAGE=zabbix/zabbix-proxy-sqlite3
JAVA_GW_IMAGE = zabbix/zabbix-java-gateway
ALPINE_IMAGE_TAG=alpine-7.2.4
ZBX_SERVER_HOST=46.101.140.98

The proxy name in the frontend must be the same as the value set in proxy environment variable ZBX_HOSTNAME! Also, in frontend for active proxies, you don’t need to add the proxy address.

Next, you can set hosts to be monitored by Zabbix-proxies, but make sure to update the agent configuration, so agents accept connections from proxy.

Fig. 6. Hosts monitored by proxy

Fig. 7.List of proxies and hosts monitored by them

Configuring Proxy groups

You can create as many proxy containers as necessary in Docker, and you can also create proxy groups for load balancing (it is based on the number of hosts per proxy).

First, create a proxy group in the frontend:

  • Set proxy group name
  • Select failover period
  • Minimum number of proxies

Fig. 8.Creating a new proxy group

Next, add proxies to the proxy group, and specify the address for active agents and port for the active agents.

Fig. 9. Adding proxy to proxy group

Do not forget to change Zabbix agent configuration for hosts now monitored through the proxy group (add proxy groups IPs/DNS to Server and ServerActive options).

Fig. 10. Creating a new host and monitoring it through proxy group

You can see additional information regarding the proxies in the Frontend section: Administration/ Proxies.

Fig. 11. List of all configured proxies and those belonging to proxy group

Adding more Java gateways

Zabbix server or proxy can communicate with only one Zabbix java gateway, however, you are not limited tin how many Zabbix proxies you create together with Zabbix Java Gateway. You can make an unlimited number of pairs, consisting of Zabbix proxy with Zabbix Java Gateway.

For the containerized Zabbix server, you will need to add these 4 environment variables:

ZBX_JAVAGATEWAY_ENABLE=true
ZBX_JAVAGATEWAY=zabbix-java-gateway-server
ZBX_JAVAGATEWAYPORT=10052
ZBX_STARTJAVAPOLLERS=5

And start the Java gateway for the zabbix-server in docker:

docker run --name zabbix-java-gateway-server -t \
--network=zabbix-net \
--restart unless-stopped \
-d zabbix/zabbix-java-gateway:alpine-7.2.4

Or if you want to add java gateway to the Zabbix proxy, then add these 4 environment variables to Zabbix proxy in docker:

ZBX_JAVAGATEWAY_ENABLE=true
ZBX_JAVAGATEWAY=zabbix-java-gateway-proxy
ZBX_JAVAGATEWAYPORT=10052
ZBX_STARTJAVAPOLLERS=5

And start the java gateway as a container:

docker run --name zabbix-java-gateway-proxy -t \
--network=zabbix-net \
--restart unless-stopped \
-d zabbix/zabbix-java-gateway:alpine-7.2.4

And here we have a host, monitored by zabbix-agent2 through zabbix-proxy-active-02

Fig. 12. Host monitored by proxy with configured Java gateway

Upgrading docker proxies with SQLite3 database

If you have older Zabbix components already running in docker and you have upgraded the server, you will also need to upgrade the proxies.

If you have a container created from the proxy zabbix-proxy-sqlite3 image and want to upgrade it, you will lose the existing data stored in the SQLite3 database. For most users, the database functions as a buffer to temporarily keep the data until it’s sent to Zabbix server and the loss of the proxy database file data is of no consequence.

Once you have updated the image for the container, the proxy will detect the existing old database version on startup. If the directory is mounted to database file, it will delete the database file and create a new one. This will impact those who keep data after sending it to Zabbix server and use the data from the proxy database for other purposes.

Fig. 13. Database upgrade for proxy container with SQLite3 database

Upgrading docker proxies with MySQL database

To upgrade the MySQL database for proxy, log in in the MySQL database, set the log_bin_trust_function_creators flag to 1. Change the proxy image version to a newer one and start the container.

mysql> set global log_bin_trust_function_creators = 1;

If you have not set the flag, you will receive an error of database upgrade.

Fig. 14. Failed database upgrade for proxy with MySQL database

Replace the previous version of the proxy image with the new one, check the log file, and check the docker logs to see when the database schema upgrade has finished. After the upgrade, set the flag back to 0.

mysql> set global log_bin_trust_function_creators = 0;

The upgrade has been successful, and the proxy service has started after that.

Fig. 15. Successful database upgrade for proxy with MySQL database

An official docker image for the proxy with Postgresql database support is not available due to the extensive number of existing images and different versions.

Deploying multiple frontends

You can launch as many frontends as you need if you are experiencing a sudden surge in Zabbix users. Just specify which port to assign for it and you are good to go (don’t forget to also open the port in the firewall).

docker run --name zabbix-web-nginx-mysql1 -t \
-e DB_SERVER_HOST="mysql-server" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="zabbix_pwd" \
-e MYSQL_ROOT_PASSWORD="root_pwd" \
--network=zabbix-net \
-p 80:8080 \
--restart unless-stopped \
-d zabbix/zabbix-web-nginx-mysql:alpine-7.2.4

Fig. 16. One started Zabbix frontend container in docker

docker run --name zabbix-web-nginx-mysql2 -t \
-e DB_SERVER_HOST="mysql-server" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="zabbix_pwd" \
-e MYSQL_ROOT_PASSWORD="root_pwd" \
--network=zabbix-net \
-p 81:8080 \
--restart unless-stopped \
-d zabbix/zabbix-web-nginx-mysql:alpine-7.2.4

Fig. 17. Two started Zabbix frontend containers in docker

docker run --name zabbix-web-nginx-mysql3 -t \
-e DB_SERVER_HOST="mysql-server" \
-e MYSQL_DATABASE="zabbix" \
-e MYSQL_USER="zabbix" \
-e MYSQL_PASSWORD="zabbix_pwd" \
-e MYSQL_ROOT_PASSWORD="root_pwd" \
--network=zabbix-net \
-p 82:8080 \
--restart unless-stopped \
-d zabbix/zabbix-web-nginx-mysql:alpine-7.2.4

Fig. 18. Three started Zabbix frontend containers in docker

Fig. 19. Multiple frontends accessed through different ports

Browser monitoring

Browser monitoring setup has never been easier! Just add two parameters to zabbix-server container config:

ZBX_WEBDRIVERURL=selenium:4444
ZBX_STARTBROWSERPOLLERS=2

And start the web driver in the docker (with a standalone chrome browser):

docker run --name selenium -t\
--network=zabbix-net \
--restart unless-stopped \
-p 4444:4444 \
--shm-size="1g" \
-d selenium/standalone-chrome:latest

Next step: create a new host, add the template, specify which page to monitor with Macro values, and it’s DONE!!!!

Fig. 20. Creating host for monitoring website

Fig. 21. Screenshot of the monitored website

SNMP traps

For the snmptraps to work, the same directory must be shared among the zabbix-server and zabbix-snmptrap container. On the Zabbix-server side, you need to explicitly set snmp environment variable ZBX_ENABLE_SNMP_TRAPS to true and mount directory /var/lib/zabbix/snmptraps.

You also need to add the same volume to the snmptrap container.

And run the snmptraps container (make sure there is no permission issue for the directory)

docker run --name zabbix-snmptraps -t \
-v /var/lib/zabbix/snmptraps:/var/lib/zabbix/snmptraps:rw \
--network=zabbix-net \
-p 162:1162/udp \
--restart unless-stopped \
-d zabbix/zabbix-snmptraps:alpine-7.2-latest

Fig. 22. Received SNMP trap message

Scheduled reports

You can also easily configure scheduled reports by adding 2 additional environment variables to the Zabbix-server. In my case, both of these containers are in the same custom user network, therefore I will use the container name zabbix-web-service in the ZBX_WEBSERVICEURL option.

ZBX_STARTREPORTWRITERS=5
ZBX_WEBSERVICEURL=http://zabbix-web-service:10053/report

Start the Zabbix-web service, specify also these 2 parameters (you can skip those if defaults are used). You can also allow any incoming connections by setting ZBX_ALLOWEDIP=0.0.0.0/0. We discourage this, however.

ZBX_ALLOWEDIP=zabbix-server-mysql
ZBX_LISTENPORT=10053

Before testing scheduled reports, make sure you have enabled and configured the email media type.

Fig. 23. Configured and enabled media type

It is also encouraged to test it and check that you have received the test email.

Fig. 24. Successful media type test response

Fig. 25. Received test response on the selected media type.

Next, configure the user media where the scheduled report will be sent.

Fig. 26.Media type defined for the user

Last, but not least, set the frontend URL in the section Administration/General/Other section. In my case, I set the container name of the frontend and specify the port.

for Apache: http://<server_ip_or_name>/zabbix
for Nginx: http://<server_ip_or_name>

Fig. 27. Configured frontend address for the Frontend URL option

Next, create a scheduled report based on the dashboard of your choice.

Fig. 28.Configuring scheduled report

Check that you have received the test report in your mail.

Fig. 29.Successful scheduled report test.

Fig. 30. Received scheduled report test in the email

Now you know how to set up scheduled reports!

Docker container monitoring

You can also monitor Docker containers with a containerized Zabbix instance*

* Disclaimer: If docker service is not running, Zabbix monitoring will also not function and you will not receive notifications and alerts.

You can also monitor your docker instance with the Zabbix agent 2, however, you will be required to install Zabbix-agent 2 on the host either as a package or build it from the source.

You will also need to give user zabbix access to the docker.sock file. Just add user zabbix to group docker:

# usermod -aG docker zabbix

Otherwise, you will get an error message in items:

Cannot fetch data: Get "http://1.28/info": dial unix /var/run/docker.sock: connect: permission denied.

Go back to the frontend and create a Host for monitoring the docker containers:

  • Link template: Docker by Zabbix agent 2
  • Add host to host group
  • Specify host address or dns name, set the correct connect to option, and specify the agent port (if a default port is used, then set 10050).

Fig. 31. Configuring the host for monitoring the docker container

Now, if some issue happens to other containers, Zabbix will monitor them. But to be notified of an issue, don’t forget to enable and configure the media, user media, media templates, and trigger actions, so that you receive alerts.

Fig. 32.Latest data for the docker host

Thank you for reading – I hope you’ve found this article helpful and informative!

 

The post Deploying Zabbix Components with Docker and Docker Compose appeared first on Zabbix Blog.

Monitor Your Wi-Fi Signal Strength with Zabbix

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/monitor-your-wi-fi-signal-strength-with-zabbix/29835/

Can you monitor the signal strength of different Wi-Fi devices that are connected to your (home) router with Zabbix? Of course you can! This is a really quick post that also shows how ChatGPT or any LLM can boost your productivity when doing this kind of thing.

I have an ASUS RT-AX68U router running on Asuswrt-Merlin firmware. On its web interface, it can show you all kinds of details about your network and the devices on it. This is nice, but it would be even nicer to add some of that to Zabbix. One interesting idea for me would be to monitor the signal strength of my Wi-FI devices around the house, so let’s do that and start monitoring RSSI!

What’s RSSI?

Here’s a reply by ChatGPT:

In Wi-Fi (and RF in general), RSSI (Received Signal Strength Indicator) is typically measured in negative dBm values:

• The closer the value is to 0 dBm, the stronger (better) the signal.

• The more negative the value, the weaker the signal.

Broadly speaking, here is a rough guideline:

• -30 dBm: Extremely strong signal (almost too strong – rare in normal conditions).

• -50 dBm: Excellent signal.

• -60 dBm: Very good signal, plenty strong for most uses.

• -70 dBm: Adequate; connectivity is usually reliable but might slow at times.

• -80 dBm: Marginal; still connected but performance may degrade.

• -90 dBm or lower: Very weak; likely to drop connection or have very poor speeds.

Monitoring implementation

If you are a regular reader, you should know by now that I’m not a fan of letting Zabbix agent or any other agent run commands directly for gathering metrics unless I really need the metrics that second. Rather, I’ll use cron jobs or any other background way of creating text files which then will be parsed by Zabbix.

That said, my ASUS now runs a shell script every minute, which then writes a text file /tmp/rssi.txt, which is read by Zabbix agent.

The shell script

Thank you ChatGPT for the following: The script uses wl -i assoclist command to list the connected devices with their MAC addresses and signal strength, and converts those MAC addresses to hostnames to be human-readable.

#!/bin/sh

# Interfaces for 2.4 and 5 GHz (adjust if your router uses different names)
IFACES="eth5 eth6"

LEASES_FILE="/var/lib/misc/dnsmasq.leases"
rm -f /tmp/rssi.txt

echo "Hostname:RSSI" >/tmp/rssi.txt
for iface in $IFACES
do
# List all MACs associated on this interface
for MAC in $(wl -i "$iface" assoclist 2>/dev/null | awk '{print $2}')
do
# Get RSSI
RSSI=$(wl -i "$iface" rssi "$MAC" 2>/dev/null)

# Look up IP and hostname in dnsmasq leases (if present)
# The leases file format is: <epoch> <MAC> <IP> <hostname> <clientid>
IP=$(grep -i "$MAC" "$LEASES_FILE" | awk '{print $3}')
HOSTNAME=$(grep -i "$MAC" "$LEASES_FILE" | awk '{print $4}')

# If the device is static or not found in dnsmasq leases, IP/HOSTNAME might be empty
# so handle that gracefully
[ -z "$IP" ] && IP="Unknown"
[ -z "$HOSTNAME" ] && HOSTNAME="Unknown"

#echo "MAC $MAC:"
#echo " RSSI: $RSSI dBm"
#echo " IP: $IP"
#echo " Hostname: $HOSTNAME"
echo "$HOSTNAME:$RSSI" >>/tmp/rssi.txt
done
done

It outputs this, with self-explanatory results.

Hostname:RSSI
Watch:-43
058743599:-66
Samsung:-62
SonosZP:-45
BroadLink_OEM-T1-89-d3-bb:-57
Kitchen:-52
Mac:-39
iPhone:-40
*:-60
MacBookPro:-43

Adding it to Zabbix

First, I added a new template, for which I then added a new master item reading the /tmp/rssi.txt file.

Because ChatGPT script did make the output in CSV format with : as delimiter, we can use Zabbix item preprocessing to convert that CSV to JSON. The JSON output looks like this.

[{"Hostname":"Watch","RSSI":"-45"},{"Hostname":"058743599","RSSI":"-70"},{"Hostname":"Samsung","RSSI":"-60"},{"Hostname":"SonosZP","RSSI":"-44"},{"Hostname":"BroadLink_OEM-T1-89-d3-bb","RSSI":"-61"},{"Hostname":"Kitchen","RSSI":"-53"},{"Hostname":"Mac","RSSI":"-37"},{"Hostname":"iPhone","RSSI":"-39"},{"Hostname":"*","RSSI":"-56"},{"Hostname":"MacBookPro","RSSI":"-41"}]

With this, we can then use Zabbix low-level discovery to automatically create the items.

Discovery rule

Now that we have our master item, let’s add the discovery rule, which can go through the JSON. The discovery rule uses my previous item as a dependent item, from which it can parse everything in one go.

Discovery item prototype

In item prototype, let’s make it again use the raw list as a dependent item and go from there.

Then in preprocessing, it picks the RSSI value for whatever device LLD was going through by using a JSONPath query…

…or as text:

$[?(@.Hostname=='{#WIFICLIENT}’)].RSSI.first()

That’s pretty much it!

We now have the data coming in once per minute:

Here’s a little dashboard, too. It shows you the traditional bar that’s available on the Top hosts/items widget, and also the new Sparkline that’s on Zabbix 7.2.

Bonus: Location estimation

After a little bit of walking around and observing the devices, I added some value mapping to make Zabbix estimate where the devices would be located. It’s not so useful for static objects, but when I move around with my Apple Watch and iPhone, I could make an attempt to monitor my location at home, too.

After this fine-tuning, my dashboard now looks like this:

Thanks for reading, and have fun conducting your own experiments!

The post Monitor Your Wi-Fi Signal Strength with Zabbix appeared first on Zabbix Blog.

Monitoring Pure Storage FlashArray with Zabbix

Post Syndicated from Aleksandr Iantsen original https://blog.zabbix.com/monitoring-pure-storage-flasharray-with-zabbix/29752/

Monitoring data storage systems is the key to keeping modern IT systems running smoothly. With the rapid growth of data and the need for instant access, using high-performance solutions like Pure Storage FlashArray is not just an advantage – it’s a necessity. However, even the most advanced systems require careful oversight regarding their performance and health. Good monitoring helps find problems early and makes it possible to use resources more efficiently. In this article, we will explore how to set up monitoring for the Pure Storage FlashArray storage system with Zabbix using our new templates.

Pure Storage FlashArray offers two API versions: REST API 1.X and REST API 2.X. To ensure compatibility and comprehensive coverage for the maximum number of devices, two templates have been developed for these API versions. This allows users to effectively monitor their Pure Storage FlashArray storage systems regardless of which API version they are utilizing, making sure that they can take full advantage of the monitoring capabilities and performance metrics provided by each version. By accommodating both API versions, organizations can achieve a more flexible and comprehensive monitoring setup tailored to meet their specific infrastructure needs.

Preparing Pure Storage FlashArray for monitoring with Zabbix

In all of these examples, the Purity for FlashArray (Purity//FA) graphical user interface (GUI) will be used, so keep in mind that some of the UI elements or navigation menus can potentially change in the future.

User creation

First of all, you need to set up a user in GUI that Zabbix will use to access the REST API and gather data. To do so, navigate to 'Settings' -> 'Users and Policies' -> 'Users' from the left-side menu. On this page, pay attention to the ‘Users’ block. In the upper right corner of this block, you will see three dots. Click on them to open a context menu. In this menu, select the 'Create User...' option. Here, create a new user by filling in the fields.

 

API Key creation

Unlike Pure Storage FlashArray v2 by HTTP, Pure Storage FlashArray v1 by HTTP supports authentication using a username and password instead of a token. This feature is left for backward compatibility with older versions of devices and firmware. However, it is strongly recommended to use token authentication if there are no technical limitations.

If you do plan to use username and password authentication in the Pure Storage FlashArray v1 by HTTP template, you can skip this step and move on to the next one.

Once you have created the user, the next step is to generate an API token. To do this, find the newly created user in the 'Users' block on the 'Settings' -> 'Users and Policies' page. On the right side of the user’s entry, locate the three dots and click on them to open the menu. From this menu, select 'Create API Token...'. Follow the prompts to generate the API token, which Zabbix will use to authenticate requests. The 'Expires In' field can be left empty.

 

 

After clicking the Create button,  the GUI will show you details about the API key. Save this information somewhere safe for now, as we will need to use this data later in Zabbix. After saving, you can close this pop-up.

Preparing Zabbix

Create a host

Open your Zabbix web interface, then navigate to the ‘Configuration' -> 'Hosts‘ page and create a new host. In this step, you need to specify a host name of your choice, so choose one of the Pure Storage FlashArray v1 by HTTP or Pure Storage FlashArray v2 by HTTP templates and assign the host to a group. The choice of template depends on the version of the Pure Storage FlashArray RESTful API that is supported by your devices.

 

Before clicking the Add button, you need to configure macros. Open the Macros tab and choose both Inherited and host macros. You’ll find a lot of macros there, but only a few of them need to be changed to start using the template. Let’s take a look at these macros.

Macro list in the Pure Storage FlashArray v1 by HTTP template:

Macro Default value Description
{$PURE.FLASHARRAY.API.URL} Web interface URL.
{$PURE.FLASHARRAY.API.TOKEN} API token.
{$PURE.FLASHARRAY.API.USERNAME} Web interface username.
{$PURE.FLASHARRAY.API.PASSWORD} Web interface password.
{$PURE.FLASHARRAY.API.VERSION} 1.19 API version.

For the Pure Storage FlashArray v1 by HTTP template, it is mandatory to specify the {$PURE.FLASHARRAY.API.URL} macro, as well as either the {$PURE.FLASHARRAY.API.TOKEN} or {$PURE.FLASHARRAY.API.USERNAME} and {$PURE.FLASHARRAY.API.PASSWORD}. It is highly recommended to use a token for authentication.

Macro list in the Pure Storage FlashArray v2 by HTTP template:

Macro Default value Description
{$PURE.FLASHARRAY.API.URL} Web interface URL.
{$PURE.FLASHARRAY.API.TOKEN} API token.
{$PURE.FLASHARRAY.API.VERSION} 2.36 API version.

For the Pure Storage FlashArray v2 by HTTP template, it is mandatory to specify just the {$PURE.FLASHARRAY.API.URL} and {$PURE.FLASHARRAY.API.TOKEN} macros to start using the template.

You can change the value for the {$PURE.FLASHARRAY.API.VERSION} macro if your device does not support this version of the API.

After specifying at least the mandatory macro values, your Macros tab should look something like this:

 

After clicking the Add button, this host will be added to Zabbix.

Data collection

After following the above steps, you should notice the newly created triggers and items after a short time if the macro values are correct.

In case there are any problems with the template’s data collection, you will find errors in the last history data of items with a name ending with item errors. Also, the corresponding triggers should be fired if there are any problems with the collection of any data.

After that, you should see newly discovered items in the Items view (for example).

 

On top of that, each host will have its own dashboard created automatically that will provide you with a good overview of resource utilization.

 

 

Use macros for low-level discovery filtering

In official Zabbix templates, you might find macros that end with MATCHES and NOT_MATCHES. These are used for low-level discovery rules (LLDs), to help you filter resources that should or should not be discovered. These values use regular expressions. Therefore, you can use wildcard symbols for pattern matching.

Usage of these macros can be found in the Filters tab, under discovery rules.

The typical default value for MATCHES is .* and for NOT_MATCHES – CHANGE_IF_NEEDED. This means that any kind of value will be discovered if it is not equal to CHANGE_IF_NEEDED. For example, in Network interface discovery, filters are used to check the interface name:

  • Macro {$PURE.FLASHARRAY.NETIF.LLD.FILTER.NAME.MATCHES} has a value of .*;
  • Macro {$PURE.FLASHARRAY.NETIF.LLD.FILTER.NAME.NOT_MATCHES} has a value of CHANGE_IF_NEEDED.

You can set the value of macro {$PURE.FLASHARRAY.NETIF.LLD.FILTER.NAME.NOT_MATCHES} to filevip, which will cause an interface named filevip to not be discovered.

Now that you have an idea how these filters work, you can adjust them based on your requirements.

HTTP proxy usage

If needed, you can specify an HTTP proxy for the template to use by changing the value of the{$PURE.FLASHARRAY.HTTP_PROXY} user macro. Every request will use this proxy.

Afterword

To wrap things up, setting up monitoring for Pure Storage FlashArray devices in Zabbix is an important step that guarantees the smooth operation of your infrastructure. I hope that our new templates will help you manage and monitor your devices more effectively.

This short article has been created to provide you with the necessary knowledge and tools to set up a monitoring system that meets your specific needs. By enabling efficient monitoring, you will be better equipped to respond to changes in system performance and maintain optimal operation. I believe this material will be valuable in helping you achieve these goals! 

The post Monitoring Pure Storage FlashArray with Zabbix appeared first on Zabbix Blog.

The ATS Group and a Large MSP

Post Syndicated from Michael Kammer original https://blog.zabbix.com/the-ats-group-and-a-large-msp/29857/

One of the most critical clients of our Premium Partners at the ATS Group is a large MSP that acts as a service and administration platform for their own clients, providing them with hardware, software, engineers, support staff, metrics, and reporting.

The challenge

The MSP needed a stable, high-performance platform monitoring solution that would cover all the services they provided. They didn’t have the capabilities or budget to run multiple monitoring solutions – a single, flexible solution that could track every service was paramount, as was the ability to react to anomalies before they became serious problems.

After an initial trial with a different monitoring solution that was notable for poor service, a lack of integrations, no community, and almost no documentation, they took a closer look at Zabbix, thanks in large part to our focus on preventative action and automation.

The solution

Because of their focus on performance-based monitoring, the client went with a “hot-cold” architecture and an integration with Ansible EDA, which stands for Event-Driven Ansible. It turned out to be a true “force multiplier”, as using Zabbix, Ansible, and EDA together allowed the MSP to monitor their systems, automate tasks based on real-time events, and provide immediate responses to issues without manual intervention.

The integration was designed to sort issues by whether or not they were able to be automated. If an issue arose that required human intervention, alerts could be sent to ServiceNow via multiple channels. If human intervention was unnecessary, the issue was rerouted to Event-Driven Ansible, which runs automation on all monitored hosts.

For example, with the joint Zabbix/Ansible solution, a slash admin backstage management system filling up at 2AM because of an overflowing log file for some script is no longer an urgent issue. If there are multiple gigabytes of room in the volume group, Zabbix can tell Ansible it’s a problem. Ansible can then increase the file system by 25% and send a message letting the engineers know in the morning that they took action on their behalf.

The results

With essentially no software costs and an automation integration that can find issues and fix them independently, the MSP was able to rapidly achieve a much higher service-to-spend ratio than they’d ever imagined possible.

There has been a noted increase in employee satisfaction as well – thanks to automation, engineers no longer have to be “on call” at all hours to solve simple issues, while C-level executives have seen productivity skyrocket thanks to the joint solution’s ability to find potential issues before they become real problems.

In conclusion

At Zabbix, we work hard to stay on the forefront of automation. That means constantly improving our own product while also staying on top of new technologies like Event-Driven Ansible in order to better integrate with them. To learn more about what Zabbix can do for MSPs, visit us here.

The post The ATS Group and a Large MSP appeared first on Zabbix Blog.

The First Steps Toward Monitoring with Zabbix and SNMP

Post Syndicated from Zabbix LatAm original https://blog.zabbix.com/the-first-steps-toward-monitoring-with-zabbix-and-snmp/29784/

In this article, we’ll explore how to use Zabbix to monitor a MikroTik device via SNMP, using specific templates that allow you to visualize the status of interfaces and their performance. Read on to understand how to use network monitoring to ensure the correct operation and performance of devices in an infrastructure employing the SNMP protocol.

Verifying SNMP communication

Before you begin, make sure that SNMP communication is configured correctly on your MikroTik device. Also, set up an appropriate SNMP community for your equipment.

Create a host in Zabbix

Once SNMP is configured, go to Data Collection > Hosts > Create Host.

Here you will need to enter the basic details of the device, such as the name, IP, and the group it belongs to. If you are working with multiple MikroTik devices, organize the hosts into groups according to their characteristics.

Apply a template

Zabbix offers a wide variety of default templates that fit different device models. By selecting the appropriate template for your MikroTik device, you will be able to view all its resources efficiently.

Configure SNMP macros

In the Macros section, specify the SNMP community you previously configured on your MikroTik.

Then, click “Update” to save the changes. This configuration will allow Zabbix to access the device data.

If you are monitoring multiple devices using the same SNMP community, it is best to configure a global macro in the path Administration > Macros.

This will allow you to efficiently manage a network of devices without having to configure them individually.

 

Visualization and monitoring

After completing the above steps, you will be able to start viewing device information directly in Zabbix, including:

Overall device performance:

Connected interfaces:

Items

Zabbix can capture a new interface automatically at defined intervals. This makes it easy to monitor a new interface without the need to include it manually, thanks to the Network Interfaces Discovery functionality.

To analyze the status, we can go to Data Collection > Hosts, find our MikroTik device, and select Items.

In this section, we can observe interface 2 of our client, which appears as a dependent item. This means that there is a master item that collects data through MIBs, which are network information databases. These items in the description section provide much more detailed and technical information about their functionality.

Configuration of specific items for interfaces

If we want to create a specific item, we must access Data Collection > Hosts > Create Item.

We must also assign a name that identifies SNMP Agent, specify the key that identifies the parameter to be monitored, analyze the corresponding MIB to capture the OID, and define the metrics capture interval according to our monitoring needs.

To validate the OID using snmpwalk, it returns the information of the MIB IF-MIB::ifOperStatus.2, which represents the interface status.

Configuration of custom triggers

To configure a trigger to alert us about the status of the interface, we go to Data Collection > Hosts > Triggers > Create Trigger.

Then, we assign a descriptive name to the trigger (either manually or by using macros), define the event that will trigger the alert, set the appropriate severity, and create a logical expression that determines the status.

State 2 → The interface is down.

Status 1 → The interface is operational.

To correctly interpret SNMP values in Zabbix, we go to Data Collection > Templates > MikroTik RB4011iGS+RM by SNMP > Value Mapping.

From here, we can observe the values returned by SNMP and configure our triggers based on them.

Finally, we can test our configuration in Monitoring > Problems, where we can see the triggers running.

Advantages of using SNMP with Zabbix

Using Zabbix as a monitoring tool not only facilitates network management, but also allows you to monitor third-party applications that use the SNMP protocol.

Its flexibility, together with the wide range of templates and configurations, make it the best choice for optimizing resources and ensuring stable performance in your infrastructure.

 

The post The First Steps Toward Monitoring with Zabbix and SNMP appeared first on Zabbix Blog.

Using Frontend Scripts to the Max with Rick van der Ploeg

Post Syndicated from Michael Kammer original https://blog.zabbix.com/using-frontend-scripts-to-the-max-with-rick-van-der-ploeg/29668/

Zabbix Conference Benelux 2025 is right around the corner, which means that it’s time for another interview with a Conference presenter. This week, we’re talking to Rick van der Ploeg, a Zabbix consultant and expert at Opensource ICT Solutions. We asked him about the road that led him to Zabbix and how frontend scripts can open the door to enhanced Zabbix functionality.

Please tell us a bit about yourself and the journey that led you to Zabbix.

I’m a Zabbix consultant at OICTS. During my time as a DevOps engineer for an MSP, my team was tasked with implementing a monitoring solution for our customers’ network equipment. We successfully integrated Zabbix and never looked back. Since then, I have gained extensive experience with Zabbix, and I recently even became a certified Zabbix Expert.

How long have you been using Zabbix? What kind of Zabbix-related tasks do you take care of on a daily basis?

I’ve been working with Zabbix for over five years now. The first version I used was 4.4, and a lot has changed since then! As a Zabbix consultant, my main role is advising on and implementing Zabbix solutions for our customers. This means anything from building custom templates to installing and managing complete monitoring environments.

Can you drop a couple of hints about what we can expect from your conference presentation?

I’ll be talking about different ways to utilize frontend scripts in Zabbix, and I’ll also showcase some relevant examples of how scripts can enhance Zabbix’s functionality.

How did you decide on the topic of Zabbix and frontend scripts? Was there a specific catalyst or event that led you to this topic?

What I love about frontend scripts is their ability to extend Zabbix’s functionality to almost anything you can think of. Over the years, I’ve developed several solutions that have significantly increased the value and efficiency of the Zabbix installations I’ve worked on.
I’ve noticed that many people are still unaware of the power and possibilities that frontend scripts offer, so I figured this would be a great topic to talk about.

What makes Zabbix especially well-suited for frontend scripts?

For me, the best thing about frontend scripts in Zabbix is that you can use data from Zabbix as input for your scripts. This flexibility allows you to create simple yet powerful scripts that can be easily used by any type of Zabbix user.

The post Using Frontend Scripts to the Max with Rick van der Ploeg appeared first on Zabbix Blog.

Change is the Only Constant: 2024 in Review

Post Syndicated from Michael Kammer original https://blog.zabbix.com/change-is-the-only-constant-2024-in-review/29528/

Time waits for no one, and as impossible as it may seem, one of the most consequential years in the history of Zabbix is already in the books. You can be forgiven for feeling at times like we were trying to cram a decade of events into a single year, which is why we’ve prepared this handy look back at the highlights of 2024 – just in case there’s anything you were too busy to notice as it happened!

Re-imagining the product

One of the year’s unquestioned highlights was the long-awaited release of Zabbix Cloud on October 1. As our first new product release since the creation of Zabbix, Zabbix Cloud is designed to provide the Zabbix features our community members know and love, but with easier deployment and management as well as automatic upgrades and easier scalability.

To celebrate the release itself, we held a release party at our Riga HQ with members of our global community who were in town for Zabbix Summit 2024, which kicked off a few days later. The release definitely got our community talking – the level of interest in this new “portable” version of Zabbix exceeded even our highest expectations.

Our team also released Zabbix 7.0 LTS on June 4, packed full of improvements and upgrades that our users have been asking for, including upgraded performance and scalability, new ways to visualize data, faster network discovery speed, and more. We followed that in December with the release of Zabbix 7.2, which added improved monitoring features and workflows as well as a host of new templates.

It can’t be stressed enough – all these new products, innovations, and updates were the result of feedback and suggestions from our global community. We listen, we learn, and we take your ideas to heart!

Security first

In 2024, Zabbix transitioned to the ISO/IEC 27001:2022 certification, with an extended scope to cover Zabbix Cloud. This milestone also includes compliance with ISO/IEC 27017:2015, further enhancing cloud-specific security controls.

Meanwhile, the HackerOne bug bounty program continues to be a success. 2024 brought us 33 valid submissions, and we paid $35,000 in bounties. Out of those, we have already fixed and published 24 vulnerabilities for source code.

The Zabbix CVE (Common Vulnerabilities and Exposures) program and processes are also continuing to mature. Recently, an audit was performed against our organization’s CVE submission for 40 submissions in the NIST NVD (National Vulnerability Database). Not only did we pass the audit and gain contributor status, but Zabbix is currently the only CVE Numbering Authorities (CNA) in Latvia.

Making our presence felt

Our community members piled up the frequent flier miles as we traveled the globe to keep in touch with our ever-expanding community and win over new converts. The events we took part in this year included the following:

• 31 meetings (with first-time visits to many destinations in North America)
• 1 forum in Mexico City
• 18 meetups (both online and in a variety of global locations)
• 4 conferences (Benelux, China, Japan, and Latin America)
• Far too many exhibitions, trade fairs, and expos to list conveniently
• One extremely successful Zabbix Summit in Riga

“The long hours and even longer flights really paid off, as this year was our most successful yet in terms of new business. The events we held in North America were especially helpful in terms of breaking new ground. We understood that we have an amazing Zabbix community in the United States and a much bigger market to work with.” – Ronalds Sulcs, Zabbix Head of Sales

A year of continuous growth

2024 saw us add new team members in every location we operate in, while also recruiting remote workers from nearly every corner of the globe. All told, the team grew by 30 people, which means we are now almost 200 strong! Meanwhile, the Partners team was also operating at full throttle, adding 12 new highly-skilled Certified Partners and 16 Resellers in locations from Australia to Morocco. All partners and resellers were chosen for their unique blend of experience and expertise, and we’re confident that they’ll provide best-in-class knowledge where it’s needed most.

The fact that we’ve managed to build on our status as an employer of choice across three continents while adding an ever-increasing number of quality partners in every corner of the globe speaks to the hard work and competence of our colleagues as well as the quality of the products and services we provide. Congratulations to everyone who did their part to make sure we continue to add talent and expertise!

Giving back

2024 was an exceptionally successful year in terms of bringing our products and services to the world, but we’re proud of the fact that we also managed to export our values of openness, transparency, and a desire to give back to the communities we live and work in. This took multiple forms throughout the year:

• The LatAm team worked with the DEDICA Foundation – Foundation for Digital Development and Open Knowledge to create the Zabbix Innova Challenge. The challenge is designed to encourage creativity and stimulate technological development in local communities through a Hackathon and other activities, with the goal of introducing Zabbix to a new generation of tech talent.
• In December, we made a generous donation to Pārtikas banka “Paēdušai Latvijai” (Food Bank “For a Full Latvia”) in support of their mission of getting nutritious food to communities in need. We also gave to support Bērnu slimnīcas fonds, Latvijas Bērnu fonds, Autisma atbalsta punkts, and Ziedot.lv – all with the goal of supporting children’s health, while our employees in the Riga office prepared gifts for senior citizens at the Rīga Social Care Center Mežciems as part of the “Eņģeļa pasts (Angel’s Mail)” charity project, Santa’s Workshop.
• The end of the year also saw us contribute €50,000 to Dod pieci!, the Latvian charity marathon organized by Latvijas Radio, Latvijas Televīzija, LSM.lv, and Ziedot.lv. The marathon helps make life-saving cancer treatment more affordable.

It’s been our experience that making a difference and donating to good causes reinforces a shared commitment to the company as well as to each other. We’d like to thank and congratulate everyone who took part in our outreach efforts over the past year!

Getting noticed

The news about what we got up to in 2024 seemed to be everywhere, as tech journals, newspapers, and global organizations showered us with positive publicity. At Interop Tokyo 2024, the Zabbix Japan team picked up the prestigious “Best of Show” Grand Prize in the Management and Monitoring category for the Zabbix 7.0 LTS release. The award is granted by a jury made up of some of the world’s most knowledgeable IT and monitoring experts, so recognition was truly an honor.

In Latin America, Milenio published a profile of our CEO and Founder Alexei Vladishev that brought the Zabbix story to thousands of new readers, while Mexico’s Encuentro Vidal marked the occasion of Zabbix Conference Mexico in November with a look at how Zabbix is helping countries in the region on their journey to digital transformation.

Globo published a well-written and informative piece that explored how the Brazilian city of Extrema has been investing in new technologies (with Zabbix prominent among them) in order to better serve the population of the city and make its administration more efficient and transparent.

Carrying our momentum into 2025

As 2025 gets underway, remember to stay on top of Zabbix news by following us on social media, reading our blog, and checking our forum.

“2024 was an eventful year that was full of excitement, growth, and change. It was the year we made Zabbix Cloud a reality, and a true milestone in the growth of our company and our community. I’m sure that everyone in the Zabbix family is excited to see what our 20th Anniversary year of 2025 will bring!” – Alexei Vladishev, Zabbix Founder and CEO

 

The post Change is the Only Constant: 2024 in Review appeared first on Zabbix Blog.

File Integrity Monitoring with Zabbix

Post Syndicated from Paulo R. Deolindo Jr. original https://blog.zabbix.com/file-integrity-monitoring-with-zabbix/29460/

We have often seen Zabbix used as a simple tool for monitoring network assets as well as Information and Communication Technology (ICT) infrastructure. While this concept is not incorrect, it is equally important to understand that with the advancement of Zabbix versions, more and more functionalities have been made available for other types of monitoring, enabling advanced data analysis and stunning visualizations through new and modern widgets in the frontend layer.

In this short blog post, we will explore some of the existing yet under-discussed features of Zabbix that contribute to the maturity of the cybersecurity discipline within organizations — a topic that is becoming increasingly critical in the corporate environment.

FIM – File Integrity Monitoring

FIM is a very common concept among information security tools, specifically in tools like SIEM/XDR (Security Information Event Management/Extended Detection and Response). The name is quite suggestive of its usability, but while some tools highlight this feature as one of their main functionalities, it is also available for those who use Zabbix – just not explicitly labeled under this name.
Here, we will approach FIM as a concept rather than just a functionality. This is because we aim to achieve a result, not merely have a menu with a name to claim compliance while using our tool. In fact, the outcome needs to be more important than mere “marketing.”

What should we expect from FIM?

Imagine that your servers have certain directories and/or files so critical that you cannot afford to neglect monitoring them for changes, insertions, or deletions. Additionally, these files may have owners and properties that must not be altered – otherwise, the systems that depend on them might lose the ability to read or execute their functions. This, at a minimum, is what we expect from FIM as a functionality.
To illustrate this a bit further, consider a database service like MariaDB:

# ls -lR /etc/mysql/
/etc/mysql/:
total 24
drwxr-xr-x 2 root root 4096 Jun 25 18:40 conf.d
-rwxr-xr-x 1 root root 1740 Nov 30 2023 debian-start
-rw------- 1 root root 544 Jun 25 18:43 debian.cnf
-rw-r--r-- 1 root root 1126 Nov 30 2023 mariadb.cnf
drwxr-xr-x 2 root root 4096 Sep 30 16:36 mariadb.conf.d
lrwxrwxrwx 1 root root 24 Oct 20 2020 my.cnf -> /etc/alternatives/my.cnf
-rw-r--r-- 1 root root 839 Oct 20 2020 my.cnf.fallback

/etc/mysql/conf.d:
total 8
-rw-r--r-- 1 root root 8 Oct 20 2020 mysql.cnf
-rw-r--r-- 1 root root 55 Oct 20 2020 mysqldump.cnf

/etc/mysql/mariadb.conf.d:
total 40
-rw-r--r-- 1 root root 575 Nov 30 2023 50-client.cnf
-rw-r--r-- 1 root root 231 Nov 30 2023 50-mysql-clients.cnf
-rw-r--r-- 1 root root 927 Nov 30 2023 50-mysqld_safe.cnf
-rw-r--r-- 1 root root 3795 Sep 30 16:36 50-server.cnf
-rw-r--r-- 1 root root 570 Nov 30 2023 60-galera.cnf
-rw-r--r-- 1 root root 76 Nov 8 2023 provider_bzip2.cnf
-rw-r--r-- 1 root root 72 Nov 8 2023 provider_lz4.cnf
-rw-r--r-- 1 root root 74 Nov 8 2023 provider_lzma.cnf
-rw-r--r-- 1 root root 72 Nov 8 2023 provider_lzo.cnf
-rw-r--r-- 1 root root 78 Nov 8 2023 provider_snappy.cnf

All the files, directories, and subdirectories listed above have already been configured, and the system (whatever it may be) is functioning perfectly. However, if someone suddenly decides to alter a configuration in the file /etc/mysql/mariadb.conf.d/50-server.cnf, this could be disastrous for the service. Regardless, the important thing to do is to monitor this scope and notify the relevant stakeholders so that an appropriate analysis can be conducted.

Zabbix can help with that. Let’s see how.

Zabbix and File Integrity Monitoring functions

Consider that the Zabbix agent is installed on the server to be monitored:

vfs.dir.count[/etc/mysql]

With this key, we can count the objects present within the /etc/mysql directory. Subsequently, we can create a trigger to be activated if there is any change related to the initial collection count, such as someone deleting or adding a file or directory in this location.

 

 

 

 

 

vfs.dir.size[/etc/mysql]

With this key, we can determine the total size in bytes used by the directories and configuration files. In the future, we can create a trigger that activates when this size changes, indicating the deletion or addition of a file.

vfs.file.exists[/etc/mysql/mariadb.conf.d/50-server.cnf]

Among several important files, we may have a greater interest in some configuration files, and we can validate their existence by creating a trigger that activates when such a file ceases to exist. This will clearly indicate that something important has disappeared.

In this case, the value “1” represents “OK” for the existence of the file.

vfs.file.cksum[/etc/mysql/mariadb.conf.d/50-server.cnf,sha256]

In addition to verifying the existence of the configuration file we consider important, we need to be informed if anything in it changes. This key handles that by generating a hash in a variety of possible formats, allowing a trigger to be activated in case of a hash change, which would reflect a file modification (unfortunately, we won’t know what exactly was altered).

vfs.file.regmatch[/etc/mysql/mariadb.conf.d/50-server.cnf,^max_connections\s+=\s+(\d+)]

We might have a specific parameter of interest – for example, the maximum number of connections allowed to the database. Monitoring this is important because if the configuration is set to the default value, it means that no “tuning” has been applied to the database. Alternatively, it could mean that someone simply deleted or commented out this line, causing it to be ignored by the system. Therefore, verifying whether the parameter exists and is properly configured is crucial.

In this case, the value “1” indicates that the regular expression was successfully found, meaning that the configuration or parameter we need to exist is indeed present.

vfs.file.regexp[/etc/mysql/mariadb.conf.d/50-server.cnf,^max_connections\s+=\s+(\d+),,,,\1]

Beyond verifying the existence and integrity of the file, it is also possible to determine what was changed within it. However, we would need to specify the configuration of interest using a regular expression. For example, considering that the maximum number of connections allowed by the database system is “x,” we can be alerted by a trigger if it changes to “y,” “z,” or any other value different from “x.” This setup allows us to monitor the parameter of interest with precision. This logic can be applied to any other parameter you consider important. Of course, there is another way to automate this process, but we will not cover that automation here.

In this case, the parameter defining the maximum number of connections is not only present, but we also know the exact number of connections. This way, we will have a history of the applied parameterization in case it is changed at any point.

vfs.file.owner[/etc/mysql/mariadb.conf.d/50-server.cnf]

vfs.file.owner[/etc/mysql/mariadb.conf.d/50-server.cnf,group]

The two keys above allow us to determine the owner of a file and (in the case of a Linux system) the owning group. We can also choose to monitor the user’s name or their UID in the system. Naturally, a trigger can be activated to alert us in case of an ownership change, indicating that someone might be “taking over” an important file in the system.

vfs.file.permissions[/etc/mysql/mariadb.conf.d/50-server.cnf]

The key above allows us to determine a file’s permissions—read, write, read and write, execution, or a special permission bit. Naturally, a trigger can be activated to alert us if there is any permission change in the file.

vfs.file.attr[/etc/mysql/mariadb.conf.d/50-server.cnf]

The key above does not exist by default. It was created with a UserParameter, which is a customization for verifying a command that, in this case, checks the attributes of a specific file. Consider the following command executed directly in your system’s terminal:

# lsattr /etc/mysql/mariadb.conf.d/50-server.cnf
--------------e------- /etc/mysql/mariadb.conf.d/50-server.cnf

What interests us are the attributes:

--------------e-------

If someone who invades the system modifies the attribute of a file (for example) using this command…

# chattr +A /etc/mysql/mariadb.conf.d/50-server.cnf
# lsattr /etc/mysql/mariadb.conf.d/50-server.cnf
-------A------e------- /etc/mysql/mariadb.conf.d/50-server.cnf

…it could mean that someone does not want the system to log when this file was accessed (refer to the chattr command manual). Additionally, any other attribute can be added or removed, which poses a risk to the system because these attributes can alter how files are accessed, stored on disk, and later read. Therefore, we can create a UserParameter as follows:

# cd /etc/zabbix/zabbix_agent2.d/
# echo "UserParameter=vfs.file.attr[*],lsattr \$1 | cut -d\" \" -f1" > attr.conf
# zabbix_agent2 -R userparameter_reload

Finally, we can test the reading of attributes directly from the terminal:

# zabbix_agent2 -t vfs.file.attr[/etc/mysql/mariadb.conf.d/50-server.cnf]
vfs.file.attr[/etc/mysql/mariadb.conf.d/50-server.cnf][s|-------A------e-------]

You can also try this now through the frontend.

When creating the item, don’t forget to create the trigger that should be activated in case there is a change in the attribute of a file, whatever it may be.

Paying attention to file access and modification times

To delve a bit deeper into the concept of FIM, we should ask ourselves if we are monitoring file access and modifications concerning their timestamps. In a way, if we have implemented everything proposed above, the answer is yes.

That said, there is an easier way to keep track of all the things we’ve discussed. It involves using this key:

vfs.dir.get[/etc/mysql]

When creating an item with this key, we will recursively obtain all its objects, such as subdirectories and files. The output format will be a JSON, which allows us to create LLD (Low-level Discovery) rules to automate FIM. Below is a small snippet of the monitoring output:

{
"basename": "mariadb.cnf",
"pathname": "/etc/mysql/mariadb.cnf",
"dirname": "/etc/mysql",
"type": "file",
"user": "root",
"group": "root",
"permissions": "0644",
"uid": 0,
"gid": 0,
"size": 1126,
"time": {
"access": "2024-11-30T23:01:01-0300",
"modify": "2023-11-30T01:42:37-0300",
"change": "2024-06-25T18:41:01-0300"
},
"timestamp": {
"access": 1733018461,
"modify": 1701319357,
"change": 1719351661
}
...

Considering that the output includes all objects from the main directory, this would be the most sensible approach to configure our FIM. However, it is necessary to create the LLD and prototypes. We will not cover this in detail in this article, but this is the path I recommend you follow.

Below is a “blueprint” for an LLD to create automated File Integrity Monitoring:

The “Master item”:

The “Dependent rule”:

The LLD Macro:

The item prototypes:

Below are the components of a trigger prototype (I created just one to symbolize a type of alert for file modification):

Name: Object: {#BASENAME} just changed

Event name: Object: {#BASENAME} just changed. Last hash: {ITEM.VALUE} The previous one: {?last(/MySQLDB/vfs.file.cksum["{#PATHNAME}",sha256],#2)} Object: {#BASENAME} just changed. Last hash: {ITEM.VALUE} The previous one: {?last(/MySQLDB/vfs.file.cksum["{#PATHNAME}",sha256],#2)}

Severity: Warning

Expression: last(/MySQLDB/vfs.file.cksum["{#PATHNAME}",sha256],#1)<>last(/MySQLDB/vfs.file.cksum["{#PATHNAME}",sha256],#2)

And then, some results:

Conclusion

The implementation of a robust File Integrity Monitoring system helps to ensure the security of IT infrastructure. Detecting unauthorized changes in critical files helps prevent attacks, identify security breaches, and ensure the integrity and availability of systems. With Zabbix, we have an effective solution to implement FIM, enabling process automation and the real-time visualization of changes. This monitoring not only reinforces protection against intrusions but also facilitates auditing and compliance with regulatory standards.

The main benefits of integrating File Integrity Monitoring with Zabbix include:

1. Early detection of changes in critical files, enabling quick responses.
2. Enhanced compliance with security regulations and internal policies.
3. Protection against malware and ransomware by identifying changes in essential files.
4. Ease of auditing with automated reports and modification histories.
5. Greater visibility and control over the integrity of data and systems in real time.
6. Operational efficiency through the automation of alerts and reports.
7. Improved proactive security, helping prevent attacks before they become critical.

By using Zabbix, organizations can strengthen their security posture and optimize risk management, ensuring that any unauthorized changes are detected and promptly corrected.

 

 

The post File Integrity Monitoring with Zabbix appeared first on Zabbix Blog.

NetBox as Home CMDB and Integrated with Zabbix

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/netbox-as-home-cmdb-and-integrated-with-zabbix/29324/

Welcome to another episode of What’s up, home? weirdness! Who wouldn’t have their own NetBox at home – and who wouldn’t think of it as a  home CMDB? I’ve just started experimenting with it. For those who do not know, a Configuration Management Database (CMDB) is the source of truth for your inventory of stuff. In data centers, it keeps track of your servers, their cables, and everything else, telling you in which data center and which rack they are.

For me… well, take a look at for yourself. One picture says more than a thousand words of my storytelling.

What is it good for?

Well… in the real business world, it’s good for many things – from knowing about your assets, their serial numbers, purchase dates, hardware configuration, and so much else. I could go as deep as that, but there’s a limit how far even I want to go with these little experiments. Today’s case is merely to demonstrate the flexibility of Zabbix, yet again.

How did I do this?

I quickly threw the data in to NetBox by hand — it looks by a lot of work to do, but in fact, it wasn’t too bad – took me about 45 minutes to do the following:

  • Create a Site called “”What’s up, home?”
  • Create the rooms by adding new locations and making the previously created site as their parent
  • Add some manufacturers
  • Add some device roles
  • Add some device types

After that, adding the devices themselves is a breeze. If you have not used NetBox, this is what adding a new device looks like. Yes yes, in the real business world there would have been many more items for me to fill in, but for this case I only added the mandatory items and even those I could do just by choosing from the drop-down menus. Not a big deal.

…and the Zabbix integration?

Actually, this is something I created many years ago for other purposes, but still seems to work with today’s versions of NetBox. My little template queries NetBox over its API and asks if it has anything that matches with the host name that’s in Zabbix. If it has, then it gets the rack location and other stuff.

How this then works is pretty standard stuff. Retrieve a master item…

…and the dependent items then gather the data, parse some JSONPaths with Zabbix item preprocessing, and at least some of the items also populate bits and pieces in the Zabbix inventory. This is handy in real world, as your alerts can then contain the exact rack location and so forth about your failing devices. Add them as tags or add them as part of the alert text, your imagination is your limit.

Does it work?

Of course it does! Here’s the inventory grouped by manufacturer:

If I click on any of them, I get this:

Of course I can also browse the data through the latest data, for example…

…or I could just create some dashboards for visualizing all this. I have not done that yet, as this is what I did tonight so far and now I’m going to bed. To be continued – maybe! For now, the template only pulls data from NetBox, but I’d like to push data towards it as well, to also tell if a light bulb is powered on or not, for example. Stay tuned!

 

 

The post NetBox as Home CMDB and Integrated with Zabbix appeared first on Zabbix Blog.