Post Syndicated from Dmitry Lambert original https://blog.zabbix.com/why-zabbix-throttling-preprocessing-is-a-key-point-for-high-frequency-monitoring/12364/
Sometimes we need much more than collecting generic data from our servers or network devices. For high-frequency monitoring, we need functionality to offload сore components from the extensive load. Throttling is the exact thing that will allow you to drop repetitive values on a Pre-processing level and collect only changing values.
Contents
I. High-frequency monitoring (0:33)
1. High-frequency monitoring issues (2:25)
2. Throttling (5:55)
Throttling is available since Zabbix 4.2 and is highly effective for high-frequency monitoring.
High-frequency monitoring
We have to set update intervals for all of the items we create in Configuration > Host > Items > Create item.
Setting update interval
The smallest update interval for regular items in Zabbix is one second. If we want to monitor all items, including memory usage, network bandwidth, or CPU load once per second, this can be considered a high-frequency interval. However, in the case of industrial equipment or telemetry data, we’ll most likely need the data more often, for instance, every 1 millisecond.
The easiest way to send data every millisecond is to use Zabbix sender — a small utility to send values to the Zabbix server or the proxy. But first, these values should be gathered.
High-frequency monitoring issues
Selecting an update interval for different items
We have to think about performance, as the more data we have, the more performance issues will arise and the more powerful hardware we’ll have to buy.
If the data grabbed from a host is constantly changing, it makes sense to collect the data every 10 or 100 milliseconds, for instance. This means that we have to process this changing data with the triggers, store it in the database, visualize it in the Latest data, as every time we receive a new value.
There are values that does not have that trend to change very frequently, but without Throttling we would still collect a new value every milisecond and process it with all our triggers and internal processes, even if the value does not change over hours.
Throttling
The greatest way to solve this problem is through throttling.
To illustrate it, in Configuration > Hosts, let’s create a ‘Throttling‘ host and add it to a group.
Creating host
Then we’ll create an item to work as a Zabbix sender item.
Creating Zabbix sender item
NOTE. For a Zabbix sender item, the Type should always be ‘Zabbix trapper’.
Then open the CLI and reload the config cache:
zabbix_server -R config_cache_reload
Now we can send values to the Zabbix sender, specifying IP address of the Zabbix server, hostname, which is case-sensitive, the key, and then the value — 1:
zabbix_sender -z 127.0.0.1 -s Throttling -k youtube -o 1
If we send value “1” several times, they all will be displayed in Monitoring > Latest data.
Displaying the values grabbed from the host
NOTE. It’s possible to filter the Latest data to display only the needed host and set a sufficient range of the last values to be displayed.
Using this method we are spamming the Zabbix server. So, we can add throttling to the settings of our item in the Pre-processing tab in Configuration > Hosts.
NOTE. There are no other parameters to configure besides this Pre-processing step from the throttling menu.
Discard unchanged
Discard unchanged throttling option
With the ‘Discard unchanged‘ throttling option, only new values will be processed by the server, while identical values will be ignored.
Throttling ignores identical values
Discard unchanged with a heartbeat
If we change the pre-processing settings for our item in the Pre-processing tab in Configuration > Hosts to ‘Discard unchanged with a heartbeat‘, we have one additional Parameter to specify — the interval to send the values if they are identical.
Discard unchanged with a heartbeat
So, if we specify 120 seconds, then in Monitoring > Latest data, we’ll get the values once per 120 seconds even if they are identical.
Displaying identical values with an interval
This throttling option is useful when we have nodata() triggers. So, with the Discard unchanged throttling option, the nodata() triggers will fire as identical data will be dropped. If we use Discard unchanged with heartbeat even identical values will be grabbed, so the trigger won’t fire.
In simpler words, the ‘Discard unchanged‘ throttling option will drop all identical values, while ‘Discard unchanged with heartbeat‘ will send even the identical values with the specified interval.
—
Watch the video.