All posts by Janne Pikkarainen

What’s Up, Home? – Observe!

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/whats-up-home-observe/21201/

By day, I monitor a global cyber security company for a living. By night, I monitor my home with Zabbix and Grafana. In this weekly blog series, I’m sharing my weird experiments and new ideas on how to utilize monitoring.

On Easter, we were not at home but doing Easter stuff, that week I did not implement any major functionality to my home monitoring environment. But while I’m brewing new weird features, here are some bits and pieces of what I have learned about my home, and not shown here earlier.

I’m watching you, TV

We have one of those ‘smart’ TVs, just like about every recent TV happens to be. The one we have is a Samsung 2021 model. And, of course, I monitor it.

On the last two-day graph above, value 1.0 means that our TV is awake and responding to ICMP ping. During the annotated short spikes the TV does not have its screen on, but it is just silently awake and doing something with the network — may be checking for firmware updates or sending telemetry?

Anyway, it is definitely doing that many times per day. I will need to snoop more closely on what the heck it is doing.

A longer period of responding to ping indicates that we are actually watching the TV (or me playing PS5).

Garage, or not to garage?

That time, when I was writing this blog post, the spring has finally come, so we were doing some spring cleaning at home; no need for heavy winter jackets to be in our hallway closet anymore and so forth. For some items, my wife wondered what would be the humidity percentage in our garage.

Zabbix & Grafana to the rescue! The graph below shows the humidity levels of our living room and garage.

So, our garage definitely is a more humid place, and for now, some humidity-sensitive items were left inside our house instead of the garage.

Don’t get lost, get a map

This part is very much of a work in progress and is lacking the majority of the IoT devices we have, but I am also building a visual network map of my home environment. The map below uses the traditional Zabbix network map, but if I manage to pull a rabbit or two out of my hat, during the upcoming weeks you will see something Completely Else. Stay tuned!

Next week I will show you a definitely very weird target to monitor if I just manage to figure out how to do it.

There’s an app for that

But what if I am not at home? Sure, for any serious situations like a freezer temperature rapidly rising my Zabbix will e-mail me, but what if I just want to browse around? Using the web interface via iPhone could be done but is definitely not very convenient, so I am using ZBX Viewer app for iPhone instead. It’s handy, it’s free and it works.

I have been working at Forcepoint since 2014 and never get bored of staying up to date about the status of my house. — Janne Pikkarainen

* Please note that this blog post was originally written in April and some events mentioned do not correspond to the actual date at the time of publication.

The post What’s Up, Home? – Observe! appeared first on Zabbix Blog.

What’s Up, Home? – Don’t Forget the Facial Cream

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/whats-up-home-dont-forget-the-facial-cream/21063/

Can you monitor the regular use of facial cream with Zabbix? Of course, you can! Here’s how. This same method could be very useful for monitoring if the elderly remember to take their meds or so.

What the heck?

A little background story. My forehead has a tendency for dry skin, so I should be using facial cream daily. Of course, as a man, I can guarantee you that 100% of the days I remember to use the cream, I apply it, so in practice, this means about 40-50% hit ratio.

As lately I have been adding more monitored targets to my home Zabbix, one night my wife probably thought she was being snarky or funny when she said “One monitor I could happily receive data about would be how often you remember to use your facial cream.”

A monitoring nerd does not take such ideas lightly.

Howdy door sensor, would you like to do some work?

I found a spare magnetic door sensor and a handy box where to store the cream.

You can see where this is going. This totally beautiful prototype of my Facial Cream Smart Storage Box is now deployed to test. If I open or close the box, the door sensor status changes, thus the facial cream mercy countdown timer resets.

How does it work? And does it really work?

Cozify smart IoT hub is keeping an eye on the magnetic door sensor’s last status change. And look, that awesome brown tape does not bother the magnets at all, Cozify reported the status as changed.

Now that I got the Cozify part working, my Zabbix can then receive the last change time as in Unix time.

On my Grafana, there’s now this absolutely gorgeous new panel, converting the Unix time to the “How long ago the last event happened?” indicator.

So the dashboard part is now working. But that is not all we need to do.

Alerting and escalation

Dashboards and monitoring are not useful at all if proper alerts are not being sent out. I now have this new alert trigger action rule in place.

In other words, if I forget to apply the facial cream, I have a one-hour time window to apply it, or otherwise, the alert gets escalated to my wife.

Will this method work? Is my prototype box reliable? I will tell you next time.

I have been working at Forcepoint since 2014 and never get tired of finding out new areas to monitor. — Janne Pikkarainen

The post What’s Up, Home? – Don’t Forget the Facial Cream appeared first on Zabbix Blog.

What’s Up, Home? – Use the Zabbix, Luke

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/whats-up-home-use-the-zabbix-luke/20953/

Welcome to my weekly blog about how I monitor my home with Zabbix. Like Batman, I have a casual day job as a monitoring tech lead, and by night I tinker around with my home Zabbix. (Except that Batman does not do monitoring, or who knows.)

Anyone using Zabbix knows how it can gather data from just about anywhere, and it can send its alerts to just about anything — pager systems like PagerDuty or OpsGenie, ticketing systems like Jira, e-mail, SMS, etc. Integrating with those takes minutes, is officially supported, very well documented, and would not make sense at home. But, what if at home I would like to show any possible alerts in a completely different way?

Zabbix, meet Star Wars

So, if I want to integrate my Zabbix with a screen saver, how would that work? And how long would that take? Is it even possible?

My friend, in the case of the good old xscreensaver you have many, many options. For its text-based screen saver modules, you can feed it a text file, or an URL from where it downloads the text to show on the screen. Making xscreensaver contents dynamic is easy.

For Zabbix, to make it send its alerts as text, you have many options. At least:

  1. Configure an action that runs whatever command to save the alert to a text file; even echo would do
  2. Let your Other System fetch the alerts over Zabbix API
  3. Let your Other System fetch the alerts directly from the Zabbix database
  4. Send out your alerts as e-mails and let your Other System parse those e-mails
  5. Configure a new custom media type to do something
  6. Use Zabbix real-time export functionality
Internals of my xscreensaver showcase

For this exercise, I decided to use the sixth option: Zabbix real-time export functionality.

What’s that, you ask?

It makes Zabbix save history, trends and/or triggers to JSON files, which any 3rd party program can then parse and utilize. Enabling it happens in practice by commenting out three lines in the Zabbix server config file, altering the path where you want the JSON files to be created, deciding the maximum size of the created files, and what kind of events you want to export. Restart the Zabbix server process, done.

My Zabbix is now running on Raspberry Pi 4. Then, I have a FreeBSD laptop for anything nerdy I want to do, and the FreeBSD laptop has xscreensaver for this demonstration.

So, my FreeBSD laptop does rsync the JSON files from Zabbix server every minute, extracts the event host name and trigger name using jq, and saves the output to text file. Surround that with header and footer text files, and you are done.

Now every time I don’t touch my FreeBSD laptop in a while, it turns on its screensaver and shows me the recent Zabbix alerts. Zabbix, meet Star Wars.

For now, the alert format shown on scroller is not perfect, but it works and took three or four lines of bash in total to accomplish. That’s easy, and in total took maybe 15 minutes to implement to its current stage.

Other news about my home monitoring project
  • My facial cream usage monitoring is going great! I have not missed applying my facial cream even once — I mean, I have received alerts from Zabbix, but each and every time I have then proceeded to apply the cream and my wife has received zero alerts so far. Good boy, me! (Read more about this project next week!)
  • As the trains stopping at our station are not always reliable (they can be either very late or canceled), I now have a live map showing the real-time status of the trains we are interested in. This part actually does not have anything to do with Zabbix at the moment, it’s Grafana and its GraphQL plugin querying data from an official train traffic open data system.
  • Zabbix 6.0 gained a new official weather template, so I now have a local weather dashboard as well provided by Zabbix.
  • I made a “home status shown as emojis” dashboard to make monitoring fun(?) and interesting(?) for the whole family — now our home status can be observed from our living room TV easily.

Some screenshots are below:

I have been working at Forcepoint since 2014 and never get bored of showing the alerts in new ways. — Janne Pikkarainen

The post What’s Up, Home? – Use the Zabbix, Luke appeared first on Zabbix Blog.

What’s Up, Home? – Zabbix the Weatherman

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/whats-up-home-zabbix-the-weatherman/20897/

This week, I advanced my project on multiple fronts, so welcome to this little smorgasbord of different topics. In my future posts, I will go deeper into each topic as my project goes forward.

Zabbix the weatherman

Let me begin with a monitoring blooper.

As Zabbix has very well-working forecast/prediction functions for your usual IT capacity trending, I tried what happens if I let it predict the outdoor temperature based on recent temperatures. On my first try, this did not go as I planned.

You see, currently, here in Finland the temperatures change a lot during a 24 hours period: from nightly -10C or below temperatures to maybe +5C to +10C during the day. As I asked Zabbix to predict the weather based only on one hour of data one day ago, this did not go as planned.

OK, clearly the one hour worth of data was too little. What if ask Zabbix to base its forecast on one week worth of data?

The prediction slightly improves — at least it won’t predict a nuclear winter anymore — but only slightly. Zabbix in its little mind has no idea that the weather could get warmer due to the springtime. Or, in case Zabbix was right, I’ll let you know in a week.

Average data for Joe Average

As my monitoring setup collects more data, one thing I can get out of it will be averages. What’s the average temperature? What’s the average for this and that?

Above shows the average data for the last 24 hours, and on my Grafana dashboard the values change dynamically based on the time period I choose on it.

Who wouldn’t need home SLA reports?

Everybody knows how The Suits love their reports. I have this mental image where I think during their mornings they are like

[x] coffee
[x] warm bread
[x] orange juice
[x] classical music
[x] latest reports

And oh dear, their morning is ruined if the [x] is missing from the last entry. Poor Suits.

Anyway, as the recent Zabbix 6.0 brought us revamped Business Services Monitoring, why not use it for home monitoring, too? This part includes very much work in progress, but I will show you the current results.

When I’m finished, each room will be configured as its own Business Service. For now, I only have entered the room names and some other stuff. There is only one room with some actual content, for now, and it’s our bedroom. What happens if I click on it?

I will get to see if the lights and temperature are OK, both from a technical standpoint and for their values. In case the status would not be OK, the root cause column would show me the reason why everything is not OK — though I would not need to click my way this far, the data would be shown on the previous page already.

As for SLAs (Service Level Agreement, for example, if you promise that your service will be available 99.9% of the time, it better be or your customer will be a sad panda and yell at you), those are also a work in progress. Zabbix can be let to generate daily/weekly/whatever SLA reports for any of the configured Business Services. I have yet to build them, but I have one for my home router already.

Come on, it’s sunny, let’s go out, Zabbix!

True story: this morning my wife asked that could I add pollen monitoring to Zabbix. My non-technical wife is getting excited about home monitoring, too! (I think she’s only pretending. Still AWESOME!)
I still need to add pollen monitoring — the data is available as open data — but I initialized The Great Outdoors Monitoring in two other areas.

Where’s my train?

Just before creating this post, I proved to myself that I can show live train data on Grafana. I sure got a screenful, as I have not played around with GraphQL too much, and for now, I got way more trains than I planned to get, and the data contains extra fields I need to filter out with Grafana’s Organise Fields. Still, connection established! Wooooo!

What’s for lunch?

Only added one lunch restaurant for now, but in theory, I will receive an alert whenever the restaurant posts its new weekly lunch menu. Zabbix is configured to be a good netizen though and it will only try to fetch the menu every one hour on Monday morning, no point to poll them all week, so let’s see how this will work.

That’s all for now. See you next week!

I have been working at Forcepoint since 2014 and I am a walking monitoring unit. — Janne Pikkarainen

* Please note, that this blog post was originally written a few months ago, in early Spring, and the temperature records do not correspond to the actual weather at the time of publication.

The post What’s Up, Home? – Zabbix the Weatherman appeared first on Zabbix Blog.

What’s Up, Home? – Thumbs up!

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/whats-up-home-thumbs-up/20677/

In my previous blog post, I wrote about how I monitor my home with Zabbix. This week, I am showing how I utilize Grafana to visualize the data collected by Zabbix and what are my plans to further improve all this.

What’s on TV, honey?

First of all, one of the reasons I am building my home Grafana dashboards is that they can look fantastic. Combine that with the fact that nowadays it is super easy to cast your screen to the living room TV — or even access Grafana by using TV’s built-in web browser –, and you have one heck of a situational awareness screen. Not that it would really be needed at home, but hey, a real-time dashboard easily beats your average soap opera. I am sure my wife would not appreciate the idea that we would stare at Grafana all night long, but that is a different story altogether. I digress.

The other reason why I am building all this? I have monitored all kinds of IT stuff since 2001, and have done some very creative gymnastics with Nagios and Zabbix, so now it’s time to try out monitoring The Real World™. So far I have found out it is very similar to monitoring IT (duh).

Let’s dive into details

Above you can see a glimpse of my overall status Grafana dashboard. That’s actually all I have now, though it scrolls down for a page or two more.

The page provides me some really interesting information from battery levels to light status to firmware status of our devices. I will create some sub-dashboards and a Grafana playlist (slideshow), so our living room Mission Control TV can then show all the nuts and bolts of our home. Actually, we only have one TV and again, I am sure my wife would not appreciate The Grafana TV Show for too long, but one can dream.

Implemented so far:

  • Smart power outlet on/off status
  • Smart light bulbs on/off status
  • Info if our kitchen speaker is playing or not
  • Reachability status of different IoT devices we have around
  • Firmware status (is an upgrade needed or not) of our IoT devices
  • Amount of light (lux) status reported by Philips Hue motion sensors
  • Battery level monitoring of IoT devices; very good info to know especially about the smoke alarm device
  • Temperature monitoring in different rooms and outdoors
  • Humidity monitoring in different rooms and outdoors
  • Tons of details about our home Internet router; operational status of network ports, incoming/outgoing bandwidth, uplink status, errors, uptime, memory, CPU, disk and so on reported over SNMP
Let’s Explore!

For now, for the panels I chose to show a single stat and would like to see the timeline history of the values, I can quickly click on Explore and see my data in a different way. Explore is a very powerful feature of Grafana, so if you are a Grafana user and have not yet realized its potential, try it out!

Still to come

This public blog about monitoring my home kind of forces me to progress with it. So, here’s what is still to come:

  • Create a sensible Zabbix template; I have made some progress on investigating the JSON provided by Cozify, so stay tuned!
  • Buy a Raspberry Pi (that rhymes, yo) and move this setup from two virtual machines running on my ages-old MacBook Pro Retina mid-2012 to it. And, I gotta say, for a ten-year-old machine this MacBook is still fantastic!
  • For a Finn, a catastrophic, show-stopping missing feature is that our sauna is not monitored. AIEEE! Need to fix that.
  • The spring is coming and so is the gardening time. Not that I would understand anything about it, but I’m sure that this is an area my wife would totally approve — I’ll buy some sensors so we get alerted if our flowers and other plants are threatened by excessive heat and dryness.
  • Buy some air quality sensors so I can track the air quality both indoors and outdoors.
  • Extend the monitoring to cover not only our home, but nearby services as well. I already have a Python script that can tell me if our local train is gonna be late or is canceled, but that was for different reasons a long time ago and not even used in Zabbix or Grafana. However, inserting that data into Zabbix is trivial, so I will add that.
  • Add upcoming/active weather alerts to Grafana
  • Grafana is perfectly capable to display for example the lunch menus of the nearby restaurants, so why not?

I have worked at Forcepoint since 2014 and never get bored of visualizing and analyzing data. — Janne Pikkarainen

The post What’s Up, Home? – Thumbs up! appeared first on Zabbix Blog.

Marrying Zabbix and Cozify IoT hub

Post Syndicated from Janne Pikkarainen original https://blog.zabbix.com/marrying-zabbix-and-cozify-iot-hub/20377/

For those of you who know me, this should not come as a surprise: I absolutely love Zabbix. It gives me the ultimate freedom to monitor whatever I need to monitor and is flexible enough to be able to monitor absolutely everything you can imagine. It’s free, it’s open-source, and scales to whatever needs you might have.

What does a monitoring nerd who is a technical lead for monitoring in a global cyber security company do during his downtime? That’s a silly question, mind you. Of course, he monitors his home with a home Zabbix instance.

Temperatures at our home, measured by Cozify, data collected by Zabbix

Hello, Cozify

We have had a Cozify home automation system in our household since 2017. It is a nice central hub that supports IoT devices from a plethora of vendors and a vast selection of device categories, ranging from Philips Hue lights to motion sensors to cameras to fire alarm systems. You can then configure actions on some other device based on actions on one device: for example, turn on a light if a motion sensor detects movement.

Cozify is a very capable device, but where it definitely lacks is monitoring and analytics about what’s going on underneath.

As a monitoring addict, that is something I simply cannot stand.

Let’s build a bridge between Cozify and Zabbix

Someone has built an unofficial Python library for communicating with Cozify API. The library is a bit limited in functionality, the most limiting factor being that it only supports read-only operations. However, for my monitoring purposes, that does not matter, as I anyway need to read data.

For my initial testing purposes, I wrote a couple of small Python scripts to gather temperature and humidity data from our temperature sensors, and one script to monitor the general availability of the different IoT devices we have around. The scripts are run from cron every five minutes, and the results are written to text files that Zabbix reads. Zabbix has master items for temperature, humidity and reachability files, and using the dependent items, it can populate the data for all the 40+ data points I have now using just three polls.

Benefits of such project

Other than the cool geek factor, what’s the benefit of monitoring your home IoT hub? There’s plenty!

  • I get to learn all kinds of patterns about our home status: temperatures, reliability of individual devices, and the amount of time any device has been on/off
  • I get notified immediately if a critical device, like a smoke alarm, does not function properly
  • I get notified if the battery level on any battery-operated IoT device is getting low and can react before a device dies
  • I can follow how quickly the battery is draining on some device
Still for me to do

The current implementation is way too manual. It would be possible to utilize Zabbix low-level discovery to parse the JSON received from Cozify, but if I just dump everything from it, it contains all the possible device categories with different parameters: Philips Hue lights do report everything from their current brightness/color settings to if their firmware has been upgraded, and then the temperature or motion sensors do report back completely different set of data. That makes creating the monitored items automatically in a sane way a bit difficult.

So, I need to think a bit and figure out how to make my Cozify template more automatic.

I also need to set up a home Grafana instance speaking to Zabbix. Zabbix is excellent at collecting the monitoring data and sending out alerts, but Grafana is the perfect partner for Zabbix to do all the analytics and eye candy.

I have 20+ years of sysadmin/monitoring experience. Forcepoint has been my landing spot since 2014, and there I have been a monitoring technical lead since 2016. Everything Linux/FreeBSD, Zabbix, Grafana and open source in general is close to my heart. So close, in fact, that monitoring is also my hobby and I do weird experiments with Zabbix & Grafana at home. — Janne Pikkarainen

The post Marrying Zabbix and Cozify IoT hub appeared first on Zabbix Blog.