All posts by Blogs on Grafana Labs Blog

timeShift(GrafanaBuzz, 1w) Issue 71

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/12/07/timeshiftgrafanabuzz-1w-issue-71/

Welcome to TimeShift

We’re excited to be speaking at and sponsoring KubeCon + CloudNativeCon NA 2018 in Seattle next week and hope we get a chance to hang out. Swing by our booth to check out a new open source project we’ve been working on, and give us some feedback on Grafana and features you’d like to see.

As always, we hope you enjoy this week’s TimeShift, and don’t be shy about telling us how we can make it better. Hope to see you in Seattle!

See an article we missed? Contact us.


Latest Stable Release: Grafana v5.4

Release Highlights

You can learn more about Grafana v5.4 in the release blog post.

Download Grafana v5.4 Now


From the Blogosphere

Grafana v5.4 Stable released!: This release blog post highlights the major enhancements included in Grafana v5.4 and how to use the new features.

Step By Step Monitoring Cassandra With Prometheus And Grafana: Check out this step by step guide on how to monitor a Cassandra cluster with Prometheus and Grafana on a VM.

Cool Features in Grafana that You Might Missed: In this article, Fairuz discusses how he used Grafana’s rendering API to send png snapshots of graph panels to his various services.

How to export alerts from Prometheus to Grafana: David dives into alerting in this article and explores the method he’s found most successful in creating an alerting dashboard in Grafana with Prometheus.

Introduction to Kubernetes Monitoring: This article explores what you should be monitoring in Kubernetes and how to go about it with Rancher, Prometheus, and Grafana.

Monitoring Server Power Usage and Cost with Grafana | and How to create the graphs: Building an effective dashboard can be as much art as science. This article shows you how to monitor the server power usage and cost of your homelab, and how to build your own Grafana dashboard to track it.


GrafanaCon – We’re shaking things up on day 2!

You spoke and we listened. Based on your feedback, we’re adding TSDB focused tracks and hands-on workshops to our second day program to give you a chance to dive into the nitty gritty of the most popular open source monitoring tools from the experts who build and maintain them.

Class will be in session for topics like:




Get your tickets while they last!

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks and in-depth workshops on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

Register Now


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Featured Job

As Grafana continues to grow we’re building our European sales team and are hiring Business Development Representatives based in our Stockholm office. This is a rare opportunity to join an early stage startup and take an instrumental role in helping to build the sales function. Apply now.

View All our Open Positions


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

Thanks Peter! This was a popular feature request, and we’re excited that it made it into this release.


How are we doing?

Thats a wrap for another issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Grafana v5.4 Released

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/12/03/grafana-v5.4-released/

v5.4 Stable released!

Grafana v5.4 brings new features, many enhancements and bug fixes. This article will detail the major new features and enhancements.

Alerting

Grafana v5.4 ships with a new alert rule setting named For which is great for removing false positives. If an alert rule has a configured For and the query violates the configured threshold it will first go from OK to Pending. Going from OK to Pending Grafana will not send any notifications. Once the alert rule has been firing for more than For duration, it will change to Alerting and send alert notifications. Typically, it’s always a good idea to use this setting since it’s often worse to get false positive than wait a few minutes before the alert notification triggers.

In the screenshot you can see an example timeline of an alert using the For setting. At ~16:04 the alert state changes to Pending and after 4 minutes it changes to Alerting which is when alert notifications are sent. Once the series falls back to normal the alert rule goes back to OK. Learn more.

Additionally, there’s now support for disable the sending of OK alert notifications. Learn more.

Google Stackdriver

Grafana v5.3 included built-in support for Google Stackdriver which enables you to visualize your Stackdriver metrics in Grafana.
One important feature missing was support for templating queries. This is now included together with a brand new templating query editor for Stackdriver.

The Stackdriver templating query editor lets you choose from a set of different Query Types. This will in turn reveal additional drop downs to help you
find, filter and select the templating values you’re interested in, see screenshot for details. The templating query editor also supports chaining multiple variables
making it easy to define variables that’s dependent on other variables.

Stackdriver is the first datasource which has support for a custom templating query editor. But starting from Grafana v5.4 it’s now possible for all datasources, including plugin datasources, to
create their very own templating query editor.

Additionally, if Grafana is running on a Google Compute Engine (GCE) virtual machine, it is now possible for Grafana to automatically retrieve default credentials from the metadata server.
This has the advantage of not needing to generate a private key file for the service account and also not having to upload the file to Grafana. Learn more.

Please read Using Google Stackdriver in Grafana for more detailed information on how to get started and use it.

MySQL Query Builder

Grafana v5.4 comes with a new graphical query builder for MySQL. This brings MySQL integration more in line with some of the other datasources and makes it easier for both advanced users and beginners to work with timeseries in MySQL. Learn more about it in the documentation.

Graph Panel Enhancements

Grafana v5.4 adds support for highlighting weekdays and/or certain timespans in the graph panel. This should make it easier to compare for example weekends, business hours and/or off work hours.

Additionally, when rendering series as lines in the graph panel, should there be only one data point available for one series so that a connecting line cannot be established, a point will
automatically be rendered for that data point. This should make it easier to understand what’s going on when only receiving a single data point.

Team Preferences

Grafana v5.4 adds support for customizing home dashboard, timezone and theme for teams, in addition to the existing customization on Organization and user Profile level.

  1. Specifying a preference on User Profile level will override preference on Team and/or Organization level
  2. Specifying a preference on Team level will override preference on Organization level.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

timeShift(GrafanaBuzz, 1w) Issue 70

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/30/timeshiftgrafanabuzz-1w-issue-70/

Welcome to TimeShift

Lots to catchup on this week after taking a break for Thanksgiving, so let’s dive in! This week we share the video from the ‘Logging is coming to Grafana’ talk, an article on how Stack Overflow tackles monitoring, a shout out from AWS re:Invent 2018, and a plugin preview for Timestream, Amazon’s new TSDB for IoT apps.

See an article we missed? Contact us!


Latest Stable Release: Grafana v5.3.4

Updates in this release
  • Alerting: Delete alerts when parent folder was deleted #13322
  • MySQL: Fix $__timeFilter() should respect local time zone #13769
  • Dashboard: Fix datasource selection in panel by enter key #13932
  • Graph: Fix table legend height when positioned below graph and using Internet Explorer 11 #13903
  • Dataproxy: Drop origin and referer http headers #13328 #13949, thx @roidelapluie

This release also includes the file exfiltration vulnerability security fix (added to 5.3.3)

Download Grafana v5.3.4 Now


From the Blogosphere

VIDEO – Logging is Coming to Grafana: In our last issue of TimeShift we shared the slides from David Kaltschmidt’s recent OSMC talk that showed off some of the new Prometheus inspired logging he’s been working on. You can now watch the video of the talk and see it in action.

VIDEO – AWS re:Invent 2018: Chris Dyl, Director of Platform at Epic Games, Speaks at Monday Night Live: We always love to get shout-outs from companies using Grafana, esp at events like AWS re:Invent 2018. In this talk, Chris shows off Epic Games’ epic analytics pipeline infra, which includes Grafana.

Pro Tips: Using Grafana in Quality Assurance: Quality assurance is a priority at Evolution Gaming, the world leader in live casino gaming, and as Andrejs Kalnacs, Lead Software Developer in Test, said during his GrafanaCon EU talk in March, Grafana has been a game changer.

Stack Overflow: How We Do Monitoring – 2018 Edition: A very comprehensive article in a series about Stack Overflow’s architecture. This article goes in-depth into their monitoring stack. In 2016 Kyle Brandt discussed the culture and reality of monitoring at Stack Overflow at GrafanaCon NYC – might also be worth a watch

VIDEO – How to use Grafana with AWS CloudWatch: Step-by-step video guide on how to connect AWS Cloudwatch to Grafana. There’s also a written article to use for reference.

Monitoring docker containers on Windows using Prometheus + Grafana: Microsoft has made installing Docker on the latest version of Windows easier. In the latest version of Docker CE you can now try out kubectl operations via the “Enable Kubernetes” option. This article will show you how to get everything up and running.


Get your GrafanaCon LA Tickets Now!

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

Register Now

Join us at GrafanaCon LA and learn about:




We’ve already confirmed these awesome speakers and will be adding more!

Brendan Burns
Distinguished Engineer & Kubernetes co-founder
Microsoft

Stig Sorensen
HEAD OF TELEMETRY
BLOOMBERG

Sean Hanson
SOFTWARE DEVELOPER
BLOOMBERG

Dave Cadwallader
Site Reliability Architect
DNANEXUS

Ben Kochie
Staff Backend Engineer, Monitor
GitLab

Paul Dix
Co-Founder | CTO
InfluxData


Featured Job

As Grafana continues to grow we’re building our European sales team and are hiring Business Development Representatives based in our Stockholm office. This is a rare opportunity to join an early stage startup and take an instrumental role in helping to build the sales function. Apply now.

View All our Open Positions


Grafana Plugin Update

A new premium plugin, a new polystat panel plugin, plus some updates to Instana and the Akumuli datasouce plugins. To update any of your plugins in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click.

NEW PREMIUM PLUGIN

Amazon Timestream – Amazon announced their new TSDB called Timestream at AWS re:Invent 2018. If you’re a Grafana Enterprise customer, you will have an opportunity to get a preview of the Timestream plugin for Grafana.

Install

NEW PLUGIN

Polystat Panel – This new panel is a multi-stat panel in the hexagonal (honeycomb) style. Each hexagon represents a metric, and colors are used to signal the state making this an effective way to visualize multiple metrics. The composites feature allows you to combine Hexagons/metrics into one hexagon. The panel has lots of options for layout, setting thresholds and styling.

Install

UDPATED PLUGIN

Instana Datasource – Version v1.3.1 adds support for changing legend format to adjust the legend display. Also, it is now possible to customize legend keys in the Graph panel. The new Legend format field lets you combine different variables like the Service label or Entity type with your own custom text.

Install

UPDATED PLUGIN

Akumuli Datasource – This plugin received a general update which includes a number of small bugfixes.

Install


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

Visualizing Time-Series Data with Grafana | Webinar – December 4, 2018, 12:00pm EST:
Matt Toback and Diana Hsieh – Visualization is a key component for generating meaningful insights from time-series data. Grafana, an opensource visualization platform, is a great option for users looking to create dynamic dashboards that can be shared across teams. In recent releases, Grafana has introduced tighter integrations with both TimescaleDB and PostgreSQL. This includes a graphical query builder easier iteration when working with complex SQL queries.

In this webinar, we will discuss how visualization coupled with a flexible full SQL interface can drive powerful insights that inform both technical and business decisions. Additionally, the webinar will include a demo covering basic dashboards that users can use to jumpstart their own internal visualization systems.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

This is awesome! Maybe *cough*NASA*cough* should consider an update. 😉


How are we doing?

Thats a wrap for another issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Pro Tips: Using Grafana in Quality Assurance

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/29/pro-tips-using-grafana-in-quality-assurance/

Andrejs Kalnacs
Andrejs Kalnacs – Evolution Gaming

Quality assurance is a priority at Evolution Gaming, the world leader in live casino gaming, and as Andrejs Kalnacs, Lead Software Developer in Test, said during his GrafanaCon EU talk in March, Grafana has been a game changer.

While Kalnacs and his team have found Grafana to be invaluable in many areas, here are three best practices they’ve learned throughout their journey.

1. Putting test results in context

Kalnacs’ Grafana journey started with one simple thing: Evolution wanted to test how fast the games were loading. “I took some tools, created a solution, and then I got some numbers,” he said. “But numbers are just numbers. Without context, they don’t make much sense.” He consulted with some big data experts who suggested he put the information into InfluxDB and visualize it in Grafana. And voila! “We got some graphs where we can compare different products so our product owners can see how new features are affecting performance,” Kalnacs said. “Grafana became a platform for discussion about the performance of our games.”

2. Live monitoring performance tests

“Sometimes performance tests might be long; like 30 minutes, 1 hour, maybe 4 hours, or maybe even a day,” said Kalnacs. “Getting feedback while these tests are running is quite important.” The Evolution teams rely on the open source Apache JMeter load testing tool extensively, due to its ease of use. With its GUI, you can click and create a test. Plus, for Kalnacs’s purposes, it has native support for exporting data to InfluxDB protocol. “I just took a dashboard from grafana.com/dashboards and added it to an existing test, and in just a few minutes, I’ve already got some live results of tests,” he said. “In JMeter we can watch how our system under test is behaving while the tests are running, which is quite a good feature. If you have long performance tests, then you can monitor them as they’re running and make some conclusions even before they’re finished.”

For more context, Kalnacs added, it’s easy to layer more information on top of the dashboard. “For example, you can add some key performance metrics like CPU load of system under test on top of this graph, and it will give you much more information.”

Another popular performance tool used at Evolution is Gatling, which has native support for exporting to time series databases such as InfluxDB in its Frontline enterprise version.

3. Test execution data visualization

“In our journey to continuous integration, we want to understand how fast our tests are running compared with each other,” Kalnacs said.

He pointed out that if you are using Jenkins, there’s an InfluxDB Jenkins plugin, which supports exporting different data, such as length of test or failure rate, or information about JUnit or Robot framework tests, or static analysis tools like Cobertura or SonarQube, or the Java code coverage library JaCoCo.

In reality, there are so many test execution frameworks in so many different languages, that he suggested using an open source, multi-language test report tool called Allure. “Basically you can collect information from all different tests in one common way, and you don’t need to rely on any continuous integration systems like Jenkins,” he said. “It works in the same way for all these test execution frameworks on any CI server.”

Using Allure allows him “countless opportunities to display data in Grafana,” he said. One example: He can compare the execution length of different products and different tests, and “it may conclude that if tests run 20 minutes and there are single tests that are running 3 minutes, then maybe these 20 minute tests are too long.”

In conclusion, Kalnacs urged everyone to “share knowledge about Grafana with everyone in your company, because it can be used in very different ways, not only in monitoring of production.”

Check out the full video of Andrejs’ talk below and download his presentation slides.

Video: Using Grafana in Quality Assurance

timeShift(GrafanaBuzz, 1w) Issue 69

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/16/timeshiftgrafanabuzz-1w-issue-69/

Welcome to TimeShift

This week we released Grafana v5.3.4 which includes an important security fix, so we recommend you update your instance today. Also in this issue of TimeShift we share the first group of confirmed GrafanaCon LA speakers, and tell you about 5 plugins that have been recently updated.

See an article you think should be shared here? Contact us!


Latest Stable Release: Grafana v5.3.4

Updates in this release
  • Alerting: Delete alerts when parent folder was deleted #13322
  • MySQL: Fix $__timeFilter() should respect local time zone #13769
  • Dashboard: Fix datasource selection in panel by enter key #13932
  • Graph: Fix table legend height when positioned below graph and using Internet Explorer 11 #13903
  • Dataproxy: Drop origin and referer http headers #13328 #13949, thx @roidelapluie

This release also includes the file exfiltration vulnerability security fix (added to 5.3.3)

Download Grafana v5.3.4 Now


From the Blogosphere

Grafana 5.3.3 and 4.6.5 released with important security fix: Grafana 5.3.3 was released this week which includes an important security fix for all Grafana installations between 4.1.0 and 5.3.2. Learn more about the fix in the blog post and update today.

Cloud-Native Monitoring with Prometheus and Grafana: With the advent of containerization, the traditional ways to monitor application have rapily become obsolete. This article waks through the components and configuration of a modern monitoring stack.

Create QA dashboards with Grafana (Part 2): This quick walkthrough gets you familiar with how to add a datasource in Grafana and build your first dashboard. Part one in this series will take you through setting up your environment.

How to set up Grafana in Centos 7: As the title implies, you’ll learn how to get Grafana setup and configured in Centos 7. You’ll also learn how to add a MySQL datasource.


GrafanaCon LA
General Admission On Sale Now!

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

Get Your Ticket Now

Join us at GrafanaCon LA and learn about:




We’ve already confirmed these awesome speakers and will be adding more!

Brendan Burns
Distinguished Engineer & Kubernetes co-founder
Microsoft

Stig Sorensen
HEAD OF TELEMETRY
BLOOMBERG

Sean Hanson
SOFTWARE DEVELOPER
BLOOMBERG

Dave Cadwallader
Site Reliability Architect
DNANEXUS

Ben Kochie
Staff Backend Engineer, Monitor
GitLab

Paul Dix
Co-Founder | CTO
InfluxData


Featured Job

As Grafana continues to grow we’re building our European sales team and are hiring Business Development Representatives based in our Stockholm office. This is a rare opportunity to join an early stage startup and take an instrumental role in helping to build the sales function. Apply now.

View All our Open Positions


Grafana Plugin Update

This week we have 2 App plugin updates to share as well as a number of Panel plugin updates. To update any of your plugins in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click.

UDPATED PLUGIN

Kentik Connect Pro – Version 1.3.1 addresses an issue where Filter operators do not match Group By operators.

Install

UPDATED PLUGIN

GLPI App – Updates in version 1.3.0 include:

  • add Apache / NGINX configuration in the readme
  • add more instructions in the readme to configure API into GLPI
  • add link of a user dashboard in the readme
  • add the patch of GLPI in the readme when use the datasource in mode server
  • fix PHP timezone in the datasource
  • update dependencies (and so fix some vulnerabilities)
  • fix a bug in number elements displayed in the singlestat panel

Install

UPDATED PLUGIN

Parity Report Panel – Minor update to packages for this panel plugin.

Install

UPDATED PLUGIN

SVG Panel – Updates in v0.2.0 of the SVG Panel include:

Install

UPDATED PLUGIN

Boom Table Panel – Boom Table is a table/multi stat plugin with multiple columns for Graphite, InfluDB, Prometheus, Azure Monitor.

  • Icons support in table cells and options to use table as heatmap
  • Options to hide first column and table header
  • Options to hide metrics based on values
  • Tooltips in each cells of the table

Install


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

Visualizing Time-Series Data with Grafana | Webinar – December 4, 2018, 12:00pm EST:
Matt Toback and Diana Hsieh – Visualization is a key component for generating meaningful insights from time-series data. Grafana, an opensource visualization platform, is a great option for users looking to create dynamic dashboards that can be shared across teams. In recent releases, Grafana has introduced tighter integrations with both TimescaleDB and PostgreSQL. This includes a graphical query builder easier iteration when working with complex SQL queries.

In this webinar, we will discuss how visualization coupled with a flexible full SQL interface can drive powerful insights that inform both technical and business decisions. Additionally, the webinar will include a demo covering basic dashboards that users can use to jumpstart their own internal visualization systems.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


How are we doing?

Thats a wrap for another issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Grafana 5.3.3 and 4.6.5 released with important security fix

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/13/grafana-5.3.3-and-4.6.5-released-with-important-security-fix/

Today we are releasing Grafana 5.3.3 and 4.6.5. These patch releases include an important security fix for all Grafana installations between 4.1.0 and 5.3.2

We also release 5.3.4 at the same time containing some fixes and improvements that we have been holding off for a while to release 5.3.3.

Release 5.3.3 only containing a security fix:

Latest stable release in 4.x:

Latest stable release in 5.x:

File Exfiltration vulnerability (CVE-2018-19039)

On the 5th of November at we were contacted about a potential security issue that could allow any users with Editor or Admin permissions in Grafana to read any file that the Grafana process can read from the filesystem. Note, that in order to exploit this you would need to be logged in to the system as a legitimate user with Editor or Admin permissions.

Affected versions

Grafana releases 4.1.0 through 5.3.2 are affected by this vulnerability.

Solutions and mitigations

All installations between 4.1.0 and 5.3.2 that have users that should not have access to the filesystem where Grafana is running must be upgraded as soon as possible. If you can not upgrade, you should set all users to viewers and remove all dashboards that contain text panels.

All instances of Grafana Cloud have already been updated to 5.3.3. Grafana Enterprise customers have been provided with fixed binaries ahead of this disclosure.

CVE ID: CVE-2018-19039

Timeline and postmortem

Here is a detailed timeline starting from when we originally learned of the issue.

5 Nov 2018 16:30 CET
Received details of vulnerability from Sebastian Solnica.
6 Nov 2018 13:00 CET
Confirmed issue.
Started working on a fix for latest stable in a private mirror.
Backported the fix to 4.6.5 in private mirror.
6 Nov 2018 16:00 CET
Received CVE-2018-19039
Started preparing 5.3.3 and 4.6.5 release from private mirror.
6 Nov 2018 17:33 CET
Started rolling out 5.3.3 to Grafana Cloud customers.
Decided on making release public on Tuesday Nov 13 13:00 CET. The date was chosen to give people time to prepare and not run into the weekend. The time was chosen to fall into main work time of the EU and US while still giving Asia a fair chance to react.
7 Nov 2018 22:05 CET
Proactively provided Grafana Enterprise customers with details and download links.
Completed rollout of 5.3.3 to Grafana Cloud.
13 Nov 2018 13:00 CET
Publish of release & this blog post.

Reporting security Issues

If you think you have found a security vulnerability please send a report to [email protected]. This address can be used for all of
Grafana Labs’s open source and commercial products (including but not limited to Grafana, Grafana Cloud, Grafana Enterprise, and grafana.com). We can accept only vulnerability reports at this address. We would prefer if you encrypted your message to us, please use our PGP key. The key fingerprint is

F988 7BEA 027A 049F AE8E 5CAA D125 8932 BE24 C5CA

The key is available from pgp.mit.edu by searching for grafana.

Security Announcements

We maintain a category on the community site named Security Announcements
where we will post a summary, remediation, and mitigation details for any patch containing security fixes. You can also subscribe to email updates to this category if you have a grafana.com account and sign on the community site or via track updates via an RSS feed.

Conclusion

If you run a Grafana between version 4.1.0 and 5.3.2 with users that should not have access to the filesystem where Grafana is running, please upgrade to Grafana 5.3.3 or 4.6.5 as soon as possible.

We would like to thank Sebastian Solnica And NCC Group for reporting this issue.

timeShift(GrafanaBuzz, 1w) Issue 68

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/09/timeshiftgrafanabuzz-1w-issue-68/

Welcome to TimeShift

The team was busy speaking at events in Europe and the US this week, showing off new Grafana features and talking about what’s to come. Check out the presentation on logging in Grafana below; we’ll share the video once it’s available. Also in this week’s issue we have 2 new plugins to share, and your weekly dose of Grafana related blog articles and videos.

See an article you think should be shared here? Contact us!


Latest Stable Release: Grafana v5.3.2

Updates in this release
  • InfluxDB/Graphite/Postgres: Prevent cross site scripting (XSS) in query editor #13667, thx @svenklemm
  • Postgres: Fix template variables error #13692, thx @svenklemm
  • Cloudwatch: Fix service panic because of race conditions #13674, thx @mtanda
  • Cloudwatch: Fix check for invalid percentile statistics #13633, thx @apalaniuk
  • Stackdriver/Cloudwatch: Allow user to change unit in graph panel if cloudwatch/stackdriver datasource response doesn’t include unit #13718, thx @mtanda
  • Stackdriver: stackdriver user-metrics duplicated response when multiple resource types #13691
  • Variables: Fix text box template variable doesn’t work properly without a default value #13666
  • Variables: Fix variable dependency check when using ${var} format #13600
  • Dashboard: Fix kiosk=1 url parameter should put dashboard in kiosk mode #13764
  • LDAP: Fix super admins can also be admins of orgs #13710, thx @adrien-f
  • Provisioning: Fix deleting provisioned dashboard folder should cleanup provisioning meta data #13280
  • Docker: adds curl back into the docker image for utility. #13794

Download Grafana v5.3.2 Now


From the Blogosphere

Logging is coming to Grafana: This week Grafana Labs’ Director, UX – David Kaltschmidt spoke at OSMC in Nuremberg to give a quick overview of Grafana, outline some of the new features since v5, discuss the new explore UI and logging functionality, and tease a few new features in the works.

How to create a high availability Grafana 5.3 environment in AWS OpsWorks: This tutorial will tell you everything you need to know to set up Grafana in AWS OpsWorks.

Microservices monitoring with Envoy service mesh, Prometheus & Grafana: This is the second post in the Observability with Envoy service mesh series. You can read the first post about Distributed Tracing here. This article explains what Envoy service mesh does, and how to get Envoy metrics into Prometheus and visualize the data in Grafana.

[VIDEO] Bonobo, Airflow And Grafana To Visualize Your Business: At PyConDE, Romain Dorgueil shows us how to use a few different tools to build valuable dashboards to monitor business metrics and the steps that will set us up for success: 1. Plan, 2. Implement, 3. Monitor, 4. Iterate.


GrafanaCon LA
General Admission On Sale Now!

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

We’ve already posted the first batch of speakers, and will be adding more shortly, so keep checking grafanacon.org.

Get Your Ticket Now


Featured Job

As Grafana continues to grow we’re building our European sales team and are hiring Business Development Representatives based in our Stockholm office. This is a rare opportunity to join an early stage startup and take an instrumental role in helping to build the sales function. Apply now.

View All our Open Positions


Grafana Plugin Update

This week we have 2 new plugins to share and an update to the Clickhouse datasource plugin. To update any of your plugins in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click.

NEW PLUGIN

Consul DatasourceConsul is a distributed service mesh, and this new datasource integrates Consul KV (key value) data with Grafana. It can be used in Singlestat and Table panels.

Install

NEW PLUGIN

Warp 10 Datasource – The Warp 10 Platform is designed to collect, store and manipulate sensor data (timeseries data combined with location data). The new Warp 10 datasource plugin allows you to query the Warp 10 database using the WarpScript query language and visualize the results in Grafana. It also integrates well with the Worldmap panel plugin.

Install

UPDATED PLUGIN

Clickhouse Datasource – Two new macros that convert query results as “change rate per interval” for Counter-like(growing only) metrics have been added to this latest version.

Install


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

That sounds awesome Dave! We’re excited to have you speaking at GrafanaCon LA and look forward to hearing more about this trampoline experiment!


How are we doing?

Thats a wrap for another issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

timeShift(GrafanaBuzz, 1w) Issue 67

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/02/timeshiftgrafanabuzz-1w-issue-67/

Welcome to TimeShift

This week we highlight Grafana project contributors from Hacktoberfest, share an article we published earlier this week on monitoring devices in your home with Prometheus and Grafana, how SoftwareMill manages their dashboards across environments and more.

Have an article you’d like shared here? Contact us!


Latest Stable Release: Grafana v5.3.2

Updates in this release
  • InfluxDB/Graphite/Postgres: Prevent cross site scripting (XSS) in query editor #13667, thx @svenklemm
  • Postgres: Fix template variables error #13692, thx @svenklemm
  • Cloudwatch: Fix service panic because of race conditions #13674, thx @mtanda
  • Cloudwatch: Fix check for invalid percentile statistics #13633, thx @apalaniuk
  • Stackdriver/Cloudwatch: Allow user to change unit in graph panel if cloudwatch/stackdriver datasource response doesn’t include unit #13718, thx @mtanda
  • Stackdriver: stackdriver user-metrics duplicated response when multiple resource types #13691
  • Variables: Fix text box template variable doesn’t work properly without a default value #13666
  • Variables: Fix variable dependency check when using ${var} format #13600
  • Dashboard: Fix kiosk=1 url parameter should put dashboard in kiosk mode #13764
  • LDAP: Fix super admins can also be admins of orgs #13710, thx @adrien-f
  • Provisioning: Fix deleting provisioned dashboard folder should cleanup provisioning meta data #13280
  • Docker: adds curl back into the docker image for utility. #13794

Download Grafana v5.3.2 Now


Hacktoberfest

Hacktoberfest is a month-long celebration of open source software run by DigitalOcean in partnership with GitHub and Twilio. The rules are simple – make five PRs between October 1–31 to any public Github repo and get an awesome t-shirt.

We’d like to thank the following amazing people for contributing to the Grafana project during Hacktoberfest:

We’ll reach out each of you personally, but as a thank you, we want to send you an official Grafana shirt.


From the Blogosphere

Pro Tips: Using Prometheus and Grafana for Monitoring Power Usage: Erwin de Keijzer, a Linux engineer at the Dutch consulting firm Snow, gave a talk at GrafanaCon EU about how he used Prometheus and Grafana to monitor the power usage… of his washing machine. A great presentation about montioring on a totally different scale.

Managing multiple Grafana instances: This article dives into how SoftwareMill uses Wizzy created by Utkarsh Bhantagar to manage Grafana dashbord storage, management, and syncing over multiple environments.

Raspberry Pi Weather Station: Nessy shows off the SensorTag configuration in a Raspberry Pi 3 Model B weather station he recently built. If you’re interseted in building something similar, you might also be interseted in checking out his article on getting Grafana 5.x running on a Raspberry Pi as well.

Monitoring Spring Boot application using Actuator, Micrometer, Prometheus and Grafana: Dhaval delves into the nuances of monitoring and observability of distributed systems and constructs a system with hypothetical requirements to understand Spring Boot Actuator and its usage in real world monitoring applications. He includes details on the exposed endpoints, available metrics, and how to configure Prometheus and Grafana.


GrafanaCon LA
General Admission Now on Sale

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

We’ve already posted the first batch of speakers, and will be adding more shortly, so keep checking grafanacon.org.

Get Your Ticket Now


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

InfluxDays 2018 | San Francisco – November 7-8, 2018:

Matt Toback: Optimizing the Grafana Platform for Flux – Flux, the new InfluxData Data Scripting Language (formerly IFQL), super-charges queries both for analytics and data science. Matt will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.

In this talk, Matt Toback from Grafana Labs will share the latest updates they have made with their Flux builder in Grafana.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

Glad that you’re liking the beta. Please reach out to @davkals with any feedback you may have.


How are we doing?

Thats a wrap for another issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Pro Tips: Using Prometheus and Grafana for Monitoring Power Usage

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/11/01/pro-tips-using-prometheus-and-grafana-for-monitoring-power-usage/

Erwin de Keijzer
Erwin de Keijzer – Snow

Sure, thousands of technologists around the world are using Prometheus and Grafana to monitor their business systems. But how about putting these technologies to work at home?

Erwin de Keijzer, a Linux engineer at the Dutch consulting firm Snow, gave a talk at GrafanaCon EU about how he used Prometheus and Grafana to monitor the power usage… of his washing machine. “This is a talk that’s a bit different scale than we’ve heard so far” at GrafanaCon, he quipped.

Spin Cycle

Erwin had a problem that might sound frustratingly familiar to many of us: The timer on his washing machine wasn’t accurate. “I come back two minutes after it should be done, and it’s 15 minutes left,” he said. “I come back 10 minutes later, and it’s still 10 minutes left…. It’s almost half an hour after it should have been done.”

So what’s an engineer to do? “I thought about putting a camera in there or buying a new washing machine that has an API,” he said. “But I’m also cheap, so I wanted to do it with stuff I had lying around already. So I thought, let’s get some monitoring on my power usage and determine if I can see from the graphs that come out of that if my washing machine is done.”

Erwin, who lives in the Netherlands, connected his Raspberry Pi Zero to his smart energy meter with a USB2 P1 cable, which enabled him to gather all of the data that the smart meter collected onto his Raspberry Pi. He wrote an exporter in Go using the promhttp library to get the data from the P1 port, store it, and expose the metrics to be scraped by Prometheus.

In his presentation, Erwin showed the graphs from his smart meter. “You can see some peaks. I’m using four kilowatts [here]. That’s probably because when [the washing machine] turned on, that uses three kilowatts,” he said. “There’s some more going on, I’m not sure what it is. So I put Prometheus on the other Intel NUC that I had lying around. So you have the gathering, but the visualization of Prometheus is not stored, so here comes Grafana.”
Power usage
Default Prometheus graph showing the 4Kw peak

Enter Grafana

With Grafana, Erwin made some graphs to show his gas usage versus the temperature. “I was interested if it’s very cold outside, do I use more gas?” he said. (Turns out, there was no correlation.) “I also wanted a method of looking at those graphs I created with Grafana, so in comes Caddy.” Caddy is an open source project that enables a web server with TLS encryption.

So did the graphs show him when his washing machine was done? “The graphs are not linear,” he said, with some disappointment. “There’s not a clear cut-off point of when it was done. So I basically failed at getting notifications when my washing machine was done.”

That said, he did have two interesting graphs about his power usage, and with those graphs, he was able to glean some insights.

Is Your Refrigerator Running?

The first graph showed the current draw and the meter reporting. “With Prometheus it is possible to do a derivative of your graph, so this [graph] is the comparison between what the meter is reporting, and what the current draw is, and as you can see they do align,” he said. “There’s a small difference between them, which also makes sense because the smart meter only reports its values every 20 seconds.”

Current draw vs what the meter reports
Current draw vs what the meter reports

The graph, interestingly, showed some peaks at night, every 60-90 minutes. What was drawing power at night? His refrigerator. “At one point I was up at night and I heard it turn on,” Erwin said. “I was like, ‘Oh, let’s check [the graph] on my phone.’”

The second graph of note showed the off-peak and peak measurements. “I found out that I have two meters, and on the weekends it’s all off-peak,” he said. “I also noticed Christmas is off-peak. Apparently January 1 is off-peak. So I now have insights into when I save money and when I don’t save money.”

Peak vs off-peak usage
Peak vs off-peak usage

Though he still has to get up and go to the laundry room to check if his washing machine is done, Erwin now has a secure, private web interface for exploring his power usage. If you’re interested in setting up your own power usage monitoring, you can find him on Twitter and Github.

Video: Using Prometheus and Grafana for Monitoring Power Usage

timeShift(GrafanaBuzz, 1w) Issue 66

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/10/26/timeshiftgrafanabuzz-1w-issue-66/

Welcome to TimeShift

All Things Open, a conference focused on open source technologies, was held in Raleigh, NC this week and was bursting at the seams with over 4,500 attendees. Grafana Labs’ own Tom Wilkie gave his RED Method: How to instrument your services talk to a standing-room only crowd. We were excited to speak at and sponsor ATO and look forward to next year.

We only have a handful of early bird tickets left for GrafanaCon LA. Be sure to grab your ticket before Oct. 31 when prices go up.

Have an article you’d like shared here? Contact us!


Latest Stable Release: Grafana v5.3.2

Updates in this release
  • InfluxDB/Graphite/Postgres: Prevent cross site scripting (XSS) in query editor #13667, thx @svenklemm
  • Postgres: Fix template variables error #13692, thx @svenklemm
  • Cloudwatch: Fix service panic because of race conditions #13674, thx @mtanda
  • Cloudwatch: Fix check for invalid percentile statistics #13633, thx @apalaniuk
  • Stackdriver/Cloudwatch: Allow user to change unit in graph panel if cloudwatch/stackdriver datasource response doesn’t include unit #13718, thx @mtanda
  • Stackdriver: stackdriver user-metrics duplicated response when multiple resource types #13691
  • Variables: Fix text box template variable doesn’t work properly without a default value #13666
  • Variables: Fix variable dependency check when using ${var} format #13600
  • Dashboard: Fix kiosk=1 url parameter should put dashboard in kiosk mode #13764
  • LDAP: Fix super admins can also be admins of orgs #13710, thx @adrien-f
  • Provisioning: Fix deleting provisioned dashboard folder should cleanup provisioning meta data #13280
  • Docker: adds curl back into the docker image for utility. #13794

Download Grafana v5.3.2 Now


GrafanaCon LA
Early Bird Tickets Now on Sale

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

Get Your Ticket Now


From the Blogosphere

Implementing SLOs Using Prometheus and Grafana: This article discusses general service availability terms like SLAs, SLOs, and SLIs, and shows you how to use Prometheus to collect the data and Grafana to build an overview dashboard. There’s also a shout out to Tom Wilkie’s RED Method concept.

[VIDEO] Devops basics: Monitoring your operations: In this video, Kai Hendry shows how he’s using Grafana for his dashboards and alerts to keep an eye on his CloudWatch metrics.

[Video] Monitor AKS cluster with Container Monitoring Solution using OMS: The Engineers.sg Meetup recently posted a video demo of different ways to monitor AKS clusters.

Deploying Helm / Tiller, Prometheus, AlertManager, Grafana, Elasticsearch On Your kubernetes Cluster – Part 2: In part 2 of this series, Eli walks us through the installation and configuration of Prometheus and Grafana. This walkthrough is part of a broader series of posts on how to configure and monitor a Kubernetes 3 Master Node cluster.


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Grafana Plugin Update

We’re sharing a few plugin updates this week, including updates to one of our premium plugins. To update any of your plugins in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click.

UDPATED PLUGIN

Kentik Connect ProKentik Detect is a SAAS service for network analytics and DDoS detection. The Grafana App recently got a new update with two new features:

  • Support for custom dimensions (user-specific columns) when doing adhoc filtering in Grafana.
  • Support for using Saved Filters from the Kentik portal to do adhoc filtering in Grafana.

Install

UDPATED PLUGIN

Instana Datasource – This update adds some new metrics to the metrics catalog.

Install

UPDATED PREMIUM PLUGIN


Dynatrace Datasource – The recent update of the Dynatrace Datasource plugin adds support for regex filtering of series.

Install


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

InfluxDays 2018 | San Francisco – November 7-8, 2018:

Matt Toback: Optimizing the Grafana Platform for Flux – Flux, the new InfluxData Data Scripting Language (formerly IFQL), super-charges queries both for analytics and data science. Matt will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.

In this talk, Matt Toback from Grafana Labs will share the latest updates they have made with their Flux builder in Grafana.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

timeShift(GrafanaBuzz, 1w) Issue 65

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/10/19/timeshiftgrafanabuzz-1w-issue-65/

Welcome to TimeShift

This week we’ll learn about the new Google Stackdriver core datasource in Grafana, dive into the new Postgres query editor and share some best practices.

GrafanaCon

The GrafanaCon CFP window is closing TODAY. Be sure and submit your talk before it’s too late.
Also, don’t forget to grab an early bird ticket they’re going fast, and you don’t want to miss out on this price!

Have an article you’d like shared here? Contact us!


Latest Stable Release: Grafana v5.3.1

Updates in this release
  • Render: Fix PhantomJS render of graph panel when legend displayed as table to the right #13616
  • Stackdriver: Filter option disappears after removing initial filter #13607
  • Elasticsearch: Fix no limit size in terms aggregation for alerting queries #13172, thx @Yukinoshita-Yukino
  • InfluxDB: Fix for annotation issue that caused text to be shown twice #13553
  • Variables: Fix nesting variables leads to exception and missing refresh #13628
  • Variables: Prometheus: Single letter labels are not supported #13641, thx @olshansky
  • Graph: Fix graph time formatting for Last 24h ranges #13650
  • Playlist: Fix cannot add dashboards with long names to playlist #13464, thx @neufeldtech
  • HTTP API: Fix /api/org/users so that query and limit querystrings works

See everything new in Grafana v5.3.1.

Download Grafana v5.3.1 Now


GrafanaCon LA
Early Bird Tickets Now on Sale

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

Get Your Ticket Now


From the Blogosphere

Introducing Stackdriver as a datasource for Grafana: The Google Cloud team received lots of requests from customers wanting to view Stackdriver data in Grafana, so we worked closely with them to build and release the Stackdriver plugin for Grafana in v5.3. This article gives you some background on Stackdriver and shows how you can get started using this beta plugin. Please give it a try, we’re looking forward to your feedback.

Make time-series exploration easier with the PostgreSQL/TimescaleDB query editor: Grafana v5.3 includes a new visual query editor for the PostgreSQL datasource. The folks from TimescaleDB did a lot of work on this feature (thanks @svenklemm), and put together a great writeup on using the new query builder and some best practices.

Azure Monitor – Combine Azure Monitor And Azure Log Analytics Data In Grafana: In this article, Stefan walks you through the steps to combine Azure Monitor and Azure LogAnalytics Data and embed the graphs in Grafana.

Bit v. Byte Episode 41 – Google & Facebook Hacks, Pixel 3 & Home Hub, Grafana: The release of Grafana 5.3 was mentioned in last week’s Bit v. Byte podcast where Adam discussed some of the new features. Always a good series to stay up to date on the latest tools and techniques.

Graph top N time series in Grafana: Brian Brazil explains how to graph the top N series over a duration using Prometheus in Grafana v5.3.


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

All Things Open 2018 | Raleigh, NC – October 21-23, 2018:

Tom Wilkie: The RED Method – How to Instrument your Services – The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.

In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.

Register Now

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

InfluxDays 2018 | San Francisco – November 7-8, 2018:

Matt Toback: Optimizing the Grafana Platform for Flux – Flux, the new InfluxData Data Scripting Language (formerly IFQL), super-charges queries both for analytics and data science. Matt will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.

In this talk, Matt Toback from Grafana Labs will share the latest updates they have made with their Flux builder in Grafana.

Register Now

Docker London Meetup | London, United Kingdom – November 22, 2018:

Tom Wilkie: Monitoring Docker with Prometheus – Prometheus has become the defacto monitoring system for cloud native applications, with many systems natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Docker monitoring system, including service discovery, node-exporter, cAdvisor and Grafana. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, Cortex, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

Very cool! The worldMap panel is one of my favorites – looks like something they’d use in the NORAD command center.


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Make time-series exploration easier with the PostgreSQL/TimescaleDB query editor

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/10/15/make-time-series-exploration-easier-with-the-postgresql/timescaledb-query-editor/

By Timescale’s Sven Klemm and Diana Hsieh

Grafana, PostgreSQL and Timescale logos

Grafana v5.3 comes with a new visual query editor for the PostgreSQL datasource. The query editor makes it easier for users to explore time-series data by improving the discoverability of data stored in PostgreSQL. Users can use drop-down menus to formulate their queries with valid selections and macros to express time-series specific functionalities, all without a deep knowledge of the database schema or the SQL language. Prior to Grafana v5.3, users had to handwrite SQL queries in order to query data. By combining the usability of the query editor with the power of full SQL, users can generate dynamic dashboards and visualizations in Grafana.

Both the query editor and PostgreSQL datasource are contributions made by TimescaleDB engineers to the open-source Grafana community. In this blog post, we will cover why we decided to build a first-class integration with Grafana for PostgreSQL (and TimescaleDB) and provide an overview of the new capabilities now possible with the query editor.

Building an open-source visualization stack

TimescaleDB is an open-source time-series database powered by PostgreSQL. We leverage the extensibility of PostgreSQL to provide a database that scales for time-series data without sacrificing the flexibility and expressiveness of relational databases. Our users typically range across a wide variety of use cases including IoT, DevOps, and web applications. Through our interactions with our users, we came to realize that users with time-series needs also consistently had visualization needs, regardless of the industry they were in. The nature of these visualizations were often different – some were powering custom visualization dashboards surfaced to end users, while others were hooking visualization tools into their TimescaleDB instances in order to analyze and share operational data internally.

Within the DevOps monitoring use case specifically, TimescaleDB is a compelling option for storing long-term metrics since it can handle complex data schemas and queries. We discovered that DBAs and SREs who chose to store metrics in TimescaleDB quickly found that they could run more interesting analyses using full SQL and that those insights would be valuable across the entire organization (e.g., for resource allocation, capacity planning, internal billing / cost accounting). They wanted to share these insights using dynamic dashboards that different business units could access.

This story is ultimately what inspired us to invest time in adding TimescaleDB compatibility to the existing PostgreSQL datasource, as well as building the PostgreSQL query editor for Grafana. We noticed that Grafana was quickly becoming the de facto visualizer of choice for the monitoring use case. As big supporters of open-source software and being an open-source solution ourselves, we view our contributions to Grafana as a continued commitment to developing a vibrant open-source community.

Using the Grafana Query Editor for PostgreSQL

The PostgreSQL query editor provides an enhanced user experience for Grafana users connecting to PostgreSQL as a datasource. It features a template that users can use to build SQL queries, as well as context aware suggestions for table names and columns. Ultimately, the query editor makes it easier for users who are not intimately familiar with the database schema to explore columns and values.

In Grafana v5.3, you’ll notice the new query editor when you edit an existing datasource or create a new PostgreSQL datasource. Upon building a new graph, the query editor will jumpstart the editor with a valid example query (if one exists). You can then leverage this example and edit it to fit your needs.

A good place to get started with the query editor is to build a simple query that graphs a time series. In the below image, the query editor has selected the cpu_utilization time-series from the devops table. The $__timeFilter macro generates queries that fill in the dates from the time selector on the top right, thus allowing dynamic visualizations based on time ranges.

When graphing time-series, it is sometimes useful to aggregate data into time buckets. These aggregations can help identify trends over time by grouping raw data into higher level aggregates. For example, you might want to average monthly raw data daily to achieve a smoother trend line or count the number of occurrences of non-numeric data.

The below video demonstrates how you can modify the above example query using aggregate functions to achieve a smoother trend line that groups data collected over the entire year into daily buckets. One thing to note is that the $__interval option in the time() function is a special variable that changes based on your selected time range. If you are collecting a lot of data, make sure you consider how many data points you are pulling in when running these queries.

Alternatively, you might have data that is not numeric in nature. In the below query, we update the example query to pull from a different table that contains usage metrics. However, usage metrics are stored as a string type where different user requests are stored as different string values. In this example, we want to count the number of times any user signs up over time.

Using window functions to apply calculations over windows of time

The query editor also supports window functions including increase, rate, sum, and moving average. In the below example, we have taken the previous example and augmented it with the increase() window function. This is a utility function that makes it easier to represent increases over windows of time versus writing a formula to calculate it in plain SQL. The function takes the time bucket specified in the GROUP BY clause and calculates the increase of the metric we are graphing over that specified window. We’ve shown both how the window function can calculate increase() over the 24 hour period, as well as how the window function updates when we change the time period to 10 minutes.

Combining SQL with the query editor to form complex queries

In addition to the visual query editor, Grafana continues to support writing raw SQL queries. You can use the query generated by the query editor as a jumping board for adjusting raw SQL. Using SQL, you can utilize JOINs to correlate time series data with relational metadata or other time series data.

In the below example, we show how you can use a JOIN to augment the usage metrics query with relational metadata. In this example, we group user signups by country. However, the country that a user comes from is stored in a separate relational table. By joining the two datasets, you can now filter the time series results by the appropriate relational metadata.

Leveraging TimescaleDB’s built-in functions

Although both the query editor and SQL editor interface work with PostgreSQL, they will also utilize time-series specific functions when TimescaleDB is enabled in Grafana and properly installed with PostgreSQL. You can utilize these functions to manipulate time-series more easily. In the below example, we show how the $__timeGroup macro automatically generates a query that utilizes the TimescaleDB function time_bucket(). You can then use the time_bucket() function to group data by arbitrary time periods by simply changing the bucket parameter. Other TimescaleDB functions work as well, and you can explore the options here.

Filling in gaps in data with the gap filling functionality

Depending on the types of graphs you want to display, sometimes you may want to fill in gaps in data. In the below example, we’ve zoomed in on the CPU utilization query to display more granular CPU utilization metrics in the form of a bar chart. You will notice a lot of gaps in the data, which result in empty spaces in the bar chart. The time() function in the GROUP BY clause includes two parameters: the first specifies the time interval in which to bucket time series, while the second specifies what value to fill the gaps with. This function uses the $__timeGroup() macro which you may also use in raw SQL queries, where the 3rd argument is the fill mode. In the below example, we’ve selected previous, which fills in gaps with the previous observed value. As a caveat, the previous value will not be pulled if it is not in the time range selected. You can observe this phenomenon in the example below as well. It’s important to note that if you build a stacked graph, you will also need to utilize this gap filling functionality so that the stacking renders properly.

Sharing configurable dashboards across your organization

Grafana comes with the ability to define template variables that can power drop-down selections. In the below example, we’ve created a template variable that pulls in the values of the request_type column. This template variable appears as a drop-down menu in the top left. We inserted the request_type variable in the WHERE clause of our query, thus allowing Grafana to populate the WHERE clause dynamically with the value selected from the drop-down menu. You could share this dashboard of user request types with your business organization, allowing them to filter the results based on the request types they are most interested in without changing the Grafana query itself.

Next Steps

In summary, we have shared both the rationale for contributing PostgreSQL/TimescaleDB integrations to Grafana, as well as specific examples of how the new query editor makes it easier to build dashboards in Grafana. Both Grafana and TimescaleDB are available to download for free as open-source software, and we would love for you to try both out and let us know what you think.

If you’re ready to get started, please download TimescaleDB (installation instructions) and Grafana (installation instructions).

For more information about the new query editor, check out the recent Grafana v5.3 release notes and learn more from their website. To read more about TimescaleDB, check out our website, documentation, and GitHub. If you are interested in learning about time-series data management and analytics, register for our webinar on October 23, 2018. We are also sponsoring GrafanaCon LA in February, so hope to see you there!

timeShift(GrafanaBuzz, 1w) Issue 64

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/10/12/timeshiftgrafanabuzz-1w-issue-64/

Welcome to TimeShift

This week the Grafana Labs team was busy prepping for the Grafana v5.3 stable release! Grafana v5.3 adds Stackdriver as a core datasource, a new graphical query editor for Postgres, enhancements to TV and kiosk mode, and a lot more. See the release section for a list of all the new features.

GrafanaCon

The GrafanaCon CFP window is scheduled to close Oct 15. Be sure and submit your talk before it’s too late.
Also, don’t forget to grab an early bird ticket to GrafanaCon before they’re sold out!

Know of an article that might be a good fit for an upcoming issue? Contact us!


GrafanaCon LA
Early Bird Tickets Now on Sale

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem. Learn about Grafana and new/upcoming features and projects in the broader ecosystem like Prometheus, Graphite, InfluxDB, Kubernetes, and more.

Get Your Ticket Now


Latest Stable Release: Grafana v5.3

Updates in this stable release
  • Stackdriver: Filter wildcards and regex matching are not yet supported #13495
  • Stackdriver: Support the distribution metric type for heatmaps #13559
  • Cloudwatch: Automatically set graph yaxis unit #13575, thx @mtanda

See everything new in Grafana v5.3.

Download Grafana v5.3 Now


From the Blogosphere

Grafana v5.3 Released: Carl Bergquist wrote an article that dives into the new features of Grafana v5.3 and how to get started using them.

Building a more reliable infrastructure with new Stackdriver tools and partners: This post from the Google Cloud team discusses Stackdriver and announces our partnership developing the Stackdriver datasource for Grafana.

Setting up Prometheus/Grafana Monitoring Dashboard for ForgeRock IDM: This installation walkthrough shows you how to visualize data from ForgeRock Identity Management using Prometheus and Grafana.

Monitor using Grafana and Prometheus.: In the first in a new series on monitoring with Grafana and Prometheus, Nitesh discusses basic components you might need to get a simple monitoring system for your servers up and running.


WEBINAR – Oct 16, 1pm EDT

Making the most out of your upgrade to Graphite 1.1

In this webinar, Graphite project maintainer Dan Cech will provide some history of the project, outline new features of Graphite 1.x, and show you how to best make use of them. Good for Graphite users of all levels, whether you’ve already upgraded to 1.1 or are still on an older version. Hope you can join us!

Register Now


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

All Things Open 2018 | Raleigh, NC – October 21-23, 2018:

Tom Wilkie: The RED Method – How to Instrument your Services – The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.

In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.

Register Now

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

InfluxDays 2018 | San Francisco – November 7-8, 2018:

Matt Toback: Optimizing the Grafana Platform for Flux – Flux, the new InfluxData Data Scripting Language (formerly IFQL), super-charges queries both for analytics and data science. Matt will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.

In this talk, Matt Toback from Grafana Labs will share the latest updates they have made with their Flux builder in Grafana.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

We are thrilled to be mentioned at Google Cloud Next London ’18!


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Grafana v5.3 Released

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/10/10/grafana-v5.3-released/

v5.3 Stable released!

Grafana v5.3 brings new features, many enhancements and bug fixes. This article will detail the major new features and enhancements.

Google Stackdriver

Grafana v5.3 ships with built-in support for Google Stackdriver and enables you to visualize your Stackdriver metrics in Grafana.

Getting started with the plugin is easy. Simply create a GCE Service account that has access to the Stackdriver API scope, download the Service Account key file from Google and upload it on the Stackdriver datasource config page in Grafana and you should have a secure server-to-server authentication setup. Like other core plugins, Stackdriver has built-in support for alerting. It also comes with support for heatmaps and basic variables.

If you’re already accustomed to the Stackdriver Metrics Explorer UI, you’ll notice that there are a lot of similarities to the query editor in Grafana. It is possible to add filters using wildcards and regular expressions. You can do Group By, Primary Aggregation and Alignment.

Alias By allows you to format the legend the way you want, and it’s a feature that is not yet present in the Metrics Explorer. Two other features that are only supported in the Grafana plugin are the abilities to manually set the Alignment Period in the query editor and to add Annotations queries.

The Grafana Stackdriver plugin comes with support for automatic unit detection. Grafana will try to map the Stackdriver unit type to a corresponding unit type in Grafana, and if successful the panel Y-axes will be updated accordingly to display the correct unit of measure. This is the first core plugin to provide support for unit detection, and it is our intention to provide support for this in other core plugins in the near future.

The datasource is still in the beta phase, meaning it’s currently in active development and is still missing one important feature – templating queries.
Please try it out, but be aware of that it might be subject to changes and possible bugs. We would love to hear your feedback.

Please read Using Google Stackdriver in Grafana for more detailed information on how to get started and use it.

TV and Kiosk Mode

We’ve improved the TV & kiosk mode to make it easier to use. There’s now an icon in the top bar that will let you cycle through the different view modes.

  1. In the first view mode, the sidebar and most of the buttons in the top bar will be hidden.
  2. In the second view mode, the top bar is completely hidden so that only the dashboard itself is shown.
  3. Hit the escape key to go back to the default view mode.

When switching view modes, the url will be updated to reflect the view mode selected. This allows a dashboard to be opened with a
certain view mode enabled. Additionally, this also enables playlists to be started with a certain view mode enabled.

Notification Reminders

Do you use Grafana alerting and have some notifications that are more important than others? Then it’s possible to set reminders so that you continue to be alerted until the problem is fixed. This is done on the notification channel itself and will affect all alerts that use that channel.
For additional examples of why reminders might be useful for you, see multiple series.

Learn how to enable and configure reminders here.

Postgres Query Builder

Grafana 5.3 comes with a new graphical query builder for Postgres. This brings Postgres integration more in line with some of the other datasources and makes it easier for both advanced users and beginners to work with timeseries in Postgres. Learn more about it in the documentation.

Improved OAuth Support for Gitlab

Grafana 5.3 comes with a new OAuth integration for Gitlab that enables configuration to only allow users that are a member of certain Gitlab groups to authenticate. This makes it possible to use Gitlab OAuth with Grafana in a shared environment without giving everyone access to Grafana.
Learn how to enable and configure it in the documentation.

Annotations

Grafana 5.3 brings improved support for native annotations and makes it possible to use template variables when filtering by tags.
Learn more about it in the documentation.

Variables

Grafana 5.3 ships with a brand new variable type named Text box which makes it easier and more convenient to provide free text input to a variable.
This new variable type will display as a free text input field with an optional prefilled default value.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

timeShift(GrafanaBuzz, 1w) Issue 63

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/10/05/timeshiftgrafanabuzz-1w-issue-63/

Welcome to TimeShift

This week we released Grafana 5.3.0beta-3 in prep for a stable release that should be available next week. In addition to details on the new beta, we have a lot of new and updated plugins to share, and our weekly roundup of Grafana-related articles from around the Internet.

Don’t forget to grab an early bird ticket to GrafanaCon before they’re sold out! Also, the CFP will close in a little less than two weeks so don’t wait until the last minute to submit a talk. If your talk is selected, we’ll comp your ticket or refund one you’ve already purchased. We’ve received some great proposals, but keep them coming! See grafanacon.org for more information.

Know of an article that might be a good fit for an upcoming issue? Contact us!

Enjoy!


GrafanaCon LA
Early Bird Tickets Now on Sale

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem.

Get Your Ticket Now


Latest Stable Release: Grafana 5.2.4

Bug Fixes
  • GrafanaCli: Fixed issue with grafana-cli install plugin resulting in corrupt http response from source error. Fixes #13079

See everything new in Grafana v5.2.4.

Download Grafana 5.2.4 Now


Latest Beta Release: Grafana 5.3.0-beta3

Major New Features
  • Stackdriver: Fix for missing ngInject #13511
  • Permissions: Fix for broken permissions selector #13507
  • Alerting: Alert reminders deduping not working as expected when running multiple Grafana instances #13492

Download Grafana 5.3.0-beta3 Now


From the Blogosphere

How to gather and display metrics in Red Hat OpenShift: Learn how to use Apache Spark Metrics and Prometheus to gather and display metrics with Grafana on Red Hat OpenShift.

Pillars of Observability: The first in an upcoming series to serve as an introduction to observability and what it means. This post discusses metrics, logs, and tracing and common solutions for each.

Lab #1 – Migrating to Grafana/InfluxDB: While switching from a monolithic approach to a services-oriented architecture, ABC Arbitrage also needed a monitoring tool to gather and report application metrics. This post walks through the decisions they made to build their new monitoring stack.

Real Time Analytics for IoT Data using Mosquitto, AWS Kinesis and InfluxDB: IoT devices generate a large amount of data. Learn how build a data pipeline through common open source components and gain actionable insight from this data.


Grafana Plugin Update

Lots of new plugin and plugin updates were published this week, including updates to two of our premium plugins. To update any of your plugins in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click.

NEW PLUGIN

Dark Sky Datasource – This new datasource plugin allows you to graph historical weather conditions and forecast data from the Dark Sky weather service in Grafana.

Install

NEW PLUGIN

Statusmap Panel – The new Statusmap panel plugin allows you to visualize multiple time series in one panel. The panel has several display modes. Discrete mode allows you to map discrete values to colors. The opacity and spectrum modes are similar to the heatmap panel.

Install

NEW PLUGIN

Annotation List Panel – This new panel shows a table of Grafana user annotations. The annotations in the list are clickable and will navigate to the panel and time range of the chosen annotation.
This can be very useful if you want an overview of events that have occurred and to be able to quickly navigate to that event.

Install

UPDATED PREMIUM PLUGIN


Datadog Datasource – The latest release of the Datadog Datasource now supports some additional functions. Single metric arithmetic allows you to multiply a metric value by a number (value * 100 for example). The following three functions are now also supported:

  • Arithmetic -> Log10 function
  • Smoothing -> Auto Smoother
  • Regression -> Trend Line

Install

UPDATED PREMIUM PLUGIN


Splunk Datasource – A bug for relative date ranges in Grafana has been fixed in the Splunk Datasource. Date ranges like Yesterday and Previous week should now always work.

Install

UPDATED PLUGIN

Trend Box and Alarm Box – The Trend Box and Alarm Box panels both received a fix for a visual bug in Grafana 5.x.

Install Trend Box
Install Alarm Box

UPDATED PLUGIN

Akumuli Datasource – The latest release of the Akumuli Datasource includes a new feature and a bug fix. The new feature is support for template variables of type interval that can be used with Downsample control.

Queries using the Top N feature now work properly when using aliasing (formatting the legend key).

Install


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

All Things Open 2018 | Raleigh, NC – October 21-23, 2018:

Tom Wilkie: The RED Method – How to Instrument your Services – The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.

In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.

Register Now

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

InfluxDays 2018 | San Francisco – November 7-8, 2018:

Matt Toback: Optimizing the Grafana Platform for Flux – Flux, the new InfluxData Data Scripting Language (formerly IFQL), super-charges queries both for analytics and data science. Matt will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.

In this talk, Matt Toback from Grafana Labs will share the latest updates they have made with their Flux builder in Grafana.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimize the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

We are extremely proud of the Grafana open source community. Thank you!!


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

timeShift(GrafanaBuzz, 1w) Issue 62

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/09/28/timeshiftgrafanabuzz-1w-issue-62/

Welcome to TimeShift

Big news this week – GrafanaCon early bird tickets are now on sale! We’ve released a limited number of early bird tickets, so grab yours before they’re sold out. Also, call for proposals is open until October 15, so don’t wait until the last minute to submit your talk. We’ve gotten some great proposals already, but the more the merrier. Keep an eye out at grafanacon.org for more updates.

See something we missed? Know of an article that might be a good fit for an upcoming issue? Contact us!

And now, on to the show!


Latest Stable Release: Grafana 5.2.4

Bug Fixes
  • GrafanaCli: Fixed issue with grafana-cli install plugin resulting in corrupt http response from source error. Fixes #13079

See everything new in Grafana v5.2.4.

Download Grafana 5.2.4 Now


Latest Beta Release: Grafana 5.3.0beta-1

Major New Features
  • Alerting: Notification reminders #7330, thx jbaublitz
  • Dashboard: TV & Kiosk mode changes, new cycle view mode button in dashboard toolbar #13025
  • OAuth: Gitlab OAuth with support for filter by groups #5623, thx BenoitKnecht
  • Postgres: Graphical query builder #10095, thx svenklemm

There are a lot of other new features and fixes including the grafana-cli fix included in 5.2.4, so please check out the release notes to see everything that’s new in Grafana 5.3.0beta-1.

Download Grafana 5.3.0beta-1 Now


GrafanaCon LA
Early Bird Tickets Now on Sale

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem.

Get Your Ticker Now


From the Blogosphere

8 Reasons Why You Should Attend GrafanaCon (and how to get your boss to send you): We understand – conferences can be expensive. But the things you learn, people you meet, and memories you make are priceless. This article covers some of the major benefits of attending GrafanaCon, which we hope will convince your company they’d be silly not to send you.

Instrumenting Porcupine With Prometheus & Grafana): Adam Bien’s Porcupine library makes it easy to configure dedicated executor services that act as application bulkheads. Sebastian has created an extension that exposes the Porcupine statistics via MicroProfile Metrics, which can be visualized in Grafana via Prometheus.

Getting Started on Monitoring with Prometheus and Grafana): This presentation from a recent meetup on monitoring with Prometheus touches on the “hows” and “whys” of monitoring as well as diving into that particulars of Prometheus and Grafana. Presentation slides from the meetup are included.

[Video] Boskey Savla – Monitoring VKE Kubernetes clusters with Prometheus/Grafana): In this video from VMware {code} Power Sessions you’ll see how a K8s cluster deployed with VMware Kubernetes Engine can be monitored using Prometheus and Grafana.

Automate Grafana Dashboard Import Process: This article talks about the components and process to automatically import dashboards into Grafana to monitor an application running on a Kubernetes cluster.


Grafana Plugin Update

This week we have an update to one of our premium plugins. To update any of your plugins in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click.

UPDATED PLUGIN

New Relic Datasource – The New Relic Datasource premium plugin now has full support for the Insights API and the NQRL query language. The latest release adds support for annotation queries for the Insights API.

Annotations in Grafana provide a way to mark points on the graph with rich events (like an outage or a deploy). When you hover over an annotation you can get event description and event tags.

Install


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

All Things Open 2018 | Raleigh, NC – October 21-23, 2018:

Tom Wilkie: The RED Method – How to Instrument your Services – The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.

In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.

Register Now

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:
David Kaltschmidt: On the OSS Path to Full Observability with Grafana – Grafana is coming "off the wall". To make it more useful for interactive debugging, David and his team have already integrated two pillars of observability – metrics and logs. They are currently adding tracing to complete the incident response experience. All to minimise the cost of context switching during those crucial minutes after getting paged.

This talk will demonstrate the various methods we've used to link the data together. Prometheus is providing the metrics. Via its histograms, request latencies can be extracted to inform each tracing span from Jaeger. Grafana also ensures that lines from your log aggregation system are annotated with span and trace IDs, as well as the other way around: associating logged values with spans.

David will show how these OSS parts should be deployed to achieve full observability in an engaging user experience that saves valuable minutes.

We are also a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

8 Reasons Why You Should Attend GrafanaCon (and how to get your boss to send you)

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/09/27/8-reasons-why-you-should-attend-grafanacon-and-how-to-get-your-boss-to-send-you/

GrafabaCon EU

Mark your calendars! GrafanaCon LA 2019 is scheduled for Feb. 25-26 in Los Angeles, California. The CFP is offically open and we encourage you to submit multiple proposals. All speakers who are selected will have their tickets comped. Speaking of tickets, tickets are officially on sale!

If you’ve attended GrafanaCon before, you already know how awesome it is. But if you’ve never been and you’re on the fence, read on to find out just what we can pack into a two-day conference. And whether you’re a newbie or an alum, stick around for some tips on how to convince your boss to send you.

1. Learning is fundamental.

It goes without saying that there’s a lot of information to be had at GrafanaCon. Because so many different people have embraced Grafana—from the monitoring tinkerer to Fortune 500 companies—GrafanaCon provides the opportunity to get first-hand accounts of monitoring at every scale. Here you can learn monitoring best practices from companies like eBay and Uber, and the ins and outs of the latest open source monitoring tools. GrafanaCon isn’t just about Grafana; it also encompasses the open source monitoring ecosystem at large. You can find out more about Prometheus, Graphite, InfluxDB, Kubernetes, and others. At the last GrafanaCon, attendees heard from more than 30 speakers over two days, covering topics ranging from how Bloomberg built its monitoring infrastructure, to an overview of the RED Method of instrumenting services (from its creator!), to a preview of the new Influx Functional Query Engine.

2. Networking opportunities galore.

Social media can bridge distance, and websites contain a wealth of information, but neither can be a true substitute for the energy of face-to-face interaction and instruction. Here’s your chance to meet your LinkedIn connections and open source co-contributors in real life. Find like-minded people, curious fellow engineers, hobbyists, and people just learning about time series data, data visualization, and open source software. While GrafanaCon has grown year over year, we believe it’s still an intimate environment where people can really get to know their tribe. In addition to the sessions, there are lunches, breaks, and an after-hours party scheduled for attendees to get their networking on.

GrafabaCon EU

3. Hands-on demos.

Where else can you get all the industry leaders and experts in one room? GrafanaCon is the perfect place to learn about new features in Grafana, get in-depth demos and tutorials, tips and tricks, and learn about new products in the extended Grafana landscape—all from the founders and creators themselves.

4. Be part of the open source software community.

More than ever, Fortune 500 companies and other enterprises realize the significant business value of open source software: more rapid feature releases, better code quality, reduced vendor lock-in, reduced costs. And as more and more companies adopt these technologies, being a part of the open source software community is valuable to its employees, too. It’s like a stamp of approval on your CV. Attending GrafanaCon can help you learn how to navigate an open source project, make contributions, and build relationships with maintainers and fellow contributors—all skills that can help you do your current job (or get your dream job).

5. You’ve got wisdom to impart, too.

You’ve been in the trenches, so you have your own monitoring story to tell. At GrafanaCon, the experiences you’ve had and the lessons you’ve learned can be immensely valuable to your fellow attendees. You could even be the hero!

6. Invest in yourself.

You’re awesome, and you’re worth it. Not only will you come back from GrafanaCon recharged with new ideas on monitoring and observability to benefit your company, but you’ll also get the chance to grow both professionally and personally.

7. Share with your coworkers.

Bring what you’ve gleaned back to your colleagues. You can even host a meetup or a training session, and invite your new GrafanaCon connections. Learning new things is great, but passing on what you’ve learned to others is priceless.

8. Have fun.

While the problems we solve through monitoring are important for businesses, we want to provide a safe, supportive, and above all, fun experience. Work hard, play hard: Break out of your shell, and come home with some new life-long connections. Our after-conference receptions are always fun, whether we’re bowling in NYC or hanging out at an arcade in Amsterdam. Also: swag.

Guess what? All of the above can be cited to convince your boss to send you to GrafanaCon. (Okay, maybe not #8. And lose the references to your CV.)

We understand that conferences can be expensive—the price of the ticket, travel, lodging, meals, and time off from our actual jobs. But think about what you get in return: the opportunity to get hands-on trials of new tools, learn how they’re implemented at scale by some of the biggest companies in the world, and crowdsource solutions for monitoring issues we’ve all struggled through. You’ll come back to work invigorated and inspired, armed with new ways of looking at and transforming the data you collect, important industry connections, and fresh ideas that will ultimately help your business succeed. Isn’t that worth your while?

Get Your GrafanaCon Ticket Now!

GrafabaCon EU

Grafana’s Explore UI: Taking a Deeper Dive into Data with Prometheus Queries

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/09/21/grafanas-explore-ui-taking-a-deeper-dive-into-data-with-prometheus-queries/

GrafabaCon EU
David Kaltschmidt, Director, UX – Grafana Labs

When there’s an incident, Grafana is often the starting point for figuring out a response. Users look at a time series panel and form a hypothesis. And in many situations, they’d like to dive deeper.

To help make that easier, Grafana Labs has created the Explore UI, which allows you to iterate quickly through Prometheus queries, while leaving your dashboards intact. “Grafana is currently on a journey to transition from dashboarding solutions to an observability platform,” Grafana Labs Director, UX David Kaltschmidt explained during his presentation at PromCon in August.

Explore UI was designed to fit into workflows for both troubleshooting and data exploration. And with Prometheus as a data source in 54,000 of over 186,000 Grafana installations—a 3x increase over last year—it made sense to focus on Prometheus queries first.

“We wanted to bring a lot more interactivity to queries and to Grafana,” said Kaltschmidt, “and we iterated a bit and came up with this more query-centric view, as opposed to the classic Grafana dashboards, which are more about visualizations.”

There’s now a new Explore icon in the panel menu of Grafana. This new section, which Kaltschmidt demoed at PromCon, focuses on the queries themselves. “For Prometheus specifically, we want to make the things that should be easy a bit easier,” Kaltschmidt said. “So for example, there’s a little metrics explorer here which groups the available metrics by prefix, so you can have your Alertmanager-related metrics here, and then you just click one, and it fills it out. That’s pretty cool. And then it’s like, hmm, this is a time series that’s monotonically increasing, maybe we should apply a rate function, so then you just click this button, and boom that works.” Especially these last query hints will play a big part in making Prometheus more approachable for novices.


New Explore UI interface

Kaltschmidt then showed how the same could be done for anything with buckets. Explore can fill out “what we hope is a good default function call for your buckets,” he said, by wrapping the expression with the correct histogram function and its parameters. These resulting expressions are meant to be quick starting points, which you can then iterate through queries.

A history was added, so that you can see information such as the last things you queried, how often they’ve been queried in the last 24 hours, and how long ago that was.

“It always executes two queries, one for the instant query for the table, and one for the graphs,” said Kaltschmidt. “You can also toggle the graph, and then you just see the table if you want to iterate more quickly.” It only takes one click on a label to filter by it and drill-down.

We’re already supporting a brand new feature of the Prometheus API: recording rules, which means we have their definitions available in Grafana. “If you want to start debugging a query that also contains recording rules, you can expand the rules and do this for all the recording rules that are in your query, in case you’ve forgotten what you’ve written there.”

A split view allows you to compare different Prometheus instances, for example, Dev vs. Prod. And multiple queries can be displayed in the same graph.

The last part of Kaltschmidt’s demo showed a faceted search within a query. Starting with a long list of jobs you can select a job, e.g., Alertmanager, and “this basically helps you find out what actual queries are available by the Alertmanager job,” he said. “Then you can look for a metric here, and it totally zones in on the labels that are available based on the previous selectors. So you only end up with suggestions that make sense to everything that came before.”


Split view and multuple queries on the same graph

Explore UI is still “super alpha,” Kaltschmidt concluded, but it’s behind the feature flag, and can be accessed if you’re using the latest Docker image. It will likely be released with Grafana v6.0. There are still things that need to be refined: “Support for completion needs a bit more work. We want Prometheus metric metadata from HELP line in the exposition. The Prometheus exposition format has this help text around what a query does, and when the ‘Write-ahead lock’ stuff is implemented, we can possibly show these query metadata, time series metadata also in the UI. That will be pretty cool.”

To enable, simply edit the Grafana config ini file:

[explore]
enabled = true

Then set up a datasource that supports Explore, e.g. Prometheus, and you’re ready to go… Explore. Feedback is welcome (@davkals)!

Watch the live demo:

Download the presentation slides

timeShift(GrafanaBuzz, 1w) Issue 61

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/09/21/timeshiftgrafanabuzz-1w-issue-61/

Welcome to TimeShift

This week we share articles covering Grafana’s alpha Explore UI, the many ways Logicify uses Grafana, building your own centralized monitoring stack, and more.

See something we missed? Know of an article that might be a good fit for an upcoming issue? Contact us!


Latest Stable Release: Grafana 5.2.4

Bug Fixes
  • GrafanaCli: Fixed issue with grafana-cli install plugin resulting in corrupt http response from source error. Fixes #13079

See everything new in Grafana v5.2.4.

Download Grafana 5.2.4 Now


Latest Beta Release: Grafana 5.3.0beta-1

Major New Features
  • Alerting: Notification reminders #7330, thx jbaublitz
  • Dashboard: TV & Kiosk mode changes, new cycle view mode button in dashboard toolbar #13025
  • OAuth: Gitlab OAuth with support for filter by groups #5623, thx BenoitKnecht
  • Postgres: Graphical query builder #10095, thx svenklemm

There are a lot of other new features and fixes including the grafana-cli fix included in 5.2.4, so please check out the release notes to see everything that’s new in Grafana 5.3.0beta-1.

Download Grafana 5.3.0beta-1 Now


GrafanaCon LA
Call for Proposals

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem.

Submit Your Proposal Today


From the Blogosphere

Grafana’s Explore UI: Taking a Deeper Dive into Data with Prometheus Queries: In this recap from the most recent PromCon in Munich, David Kaltschmidt demos the Explore UI alpha. This new interface allows you to iterate quickly through Prometheus queries, while leaving your dashboards intact. You can also try it yourself with a simple config edit.

Grafana, Graphite & Max Data Points: Steph was on a quest to find out why a Grafana singlestat panel was returning wacky values. Her investigation leads her to learning about Graphite’s default runtime consolidation. Our own Dieter Plaetinck wrote an article that touches on this and 24 other gotchas he’s encountered with Graphite, Grafana and Statsd.

Grafana as a Yet Another Tool for Technical Monitoring of Software Products We Build: Logicify uses Grafana in both their internal and external projects from monitoring the temperature of their offices to tracking user behavior in their eCommerce apps.

Services and resources monitoring with Prometheus and Grafana running on Docker: This article discusses building a centralized monitoring stack with Grafana and Prometheus to reduce context switching between multiple monitoring tools and more quickly improve overall system monitoring.

Monitoring Spark 2 performance via Grafana in Ambari-Metrics: This article covers adding a Spark 2 Dashboard to Grafana in Ambari-Metrics to monitor Spark applications for detailed resource usage statistics.


Grafana Plugin Update

One plugin update this week containing a few bug fixes. To update in your on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click at grafana.com.

UPDATED PLUGIN

Singlestat Math Panel – In this release an off-by-1px error in Firefox is fixed and the panel autosorts the thresholds on the option tab.

Install


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

Cloud Native Computing Stockholm | Stockholm, Sweden – September 25, 2018:

Torkel Ödegaard: What’s Going On at Grafana – In this session Torkel will provide an update on the project and discuss what’s new/what’s coming in the future. He’ll also cover some “best practices/interesting things seen in the wild,” and save some time for Q&A.

RSVP Now

CloudNative London 2018 | London, United Kingdom – September 26-28, 2018:

Tom Wilkie: Monitoring Kubernetes With Prometheus – In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOS’s Prometheus Operator, or using Prometheus Ksonnet Mixin.

Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.

Register Now

All Things Open 2018 | Raleigh, NC – October 21-23, 2018:

Tom Wilkie: The RED Method – How to Instrument your Services – The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.

In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.

Register Now

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now

KubeCon + CloudNativeCon North America 2018 | Seattle, WA – December 10-13, 2018:

We are a proud sponsor of the Cloud Native Computing Foundation’s flagship conference. Join Kubernetes, Prometheus, OpenTracing, Fluentd, gRPC, containerd, rkt, CNI, Envoy, Jaeger, Notary, TUF, Vitess, CoreDNS, NATS, Linkerd and Helm as the community gathers for four days to further the education and advancement of cloud native computing.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

I’m sure they’d give you a volume discount. 🙂


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

timeShift(GrafanaBuzz, 1w) Issue 60

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/09/14/timeshiftgrafanabuzz-1w-issue-60/

Welcome to TimeShift

This week we’re sharing articles on monitoring mixins, cloud native monitoring, monitoring your microservices, and a unique way to know when your software license is going to expire.
See something we missed? Know of an article that might be a good fit for an upcoming issue? Contact us!


Latest Stable Release: Grafana 5.2.4

Bug Fixes
  • GrafanaCli: Fixed issue with grafana-cli install plugin resulting in corrupt http response from source error. Fixes #13079

See everything new in Grafana v5.2.4.

Download Grafana 5.2.4 Now


Latest Beta Release: Grafana 5.3.0beta-1

Major New Features
  • Alerting: Notification reminders #7330, thx jbaublitz
  • Dashboard: TV & Kiosk mode changes, new cycle view mode button in dashboard toolbar #13025
  • OAuth: Gitlab OAuth with support for filter by groups #5623, thx BenoitKnecht
  • Postgres: Graphical query builder #10095, thx svenklemm

There are a lot of other new features and fixes including the grafana-cli fix included in 5.2.4, so please check out the release notes to see everything that’s new in Grafana 5.3.0beta-1.

Download Grafana 5.3.0beta-1 Now


GrafanaCon LA
CFP Now Open!

Join us in Los Angeles, California February 25-26, 2019 for 2 days of talks focused on Grafana and the open source monitoring ecosystem.

Submit You CFP Today


From the Blogosphere

Everything You Need to Know About Monitoring Mixins: In this recap from the most recent PromCon in Munich, Tom Wilkie dives into monitoring mixins – a solution for packaging together templates for Grafana dashboards and Prometheus alerts related to a specific piece of software. In addition to the recap, we’ve included the video demo and his presentation slides.

Reporting End of License on Temp Products with Chocolatey, Python, Telegraf and Grafana: Fabien walks us through how he uses Grafana alerts to let him know when his software licenses are about to expire.

IoT / SmartHome – Send EnOcean Sensor Data to Kafka: This article walks you through the steps to get your IoT sensor data visualized in Grafana. An EnOcean sensor will send data to Jeedom, which will feed a MQTT broker and then a synchronizer will send each received message to a Kafka topic. Then the latest value will be exposed to Prometheus and displayed on a Grafana dashboard.

Cloud Native Monitoring and Visualization with Prometheus and Grafana: A brief article on setting up a cloud native monitoring solution. Learn the basics of monitoring and observability, and get familiar with Prometheus and Grafana to make use of them in your Kubernetes environment.

Go Microservices Blog Series, Part 15 – Monitoring with Prometheus.: Part 15 in the Go Microservices blog series tackles monitoring with Prometheus and Grafana. You’ll learn about service discovery, adding endpoints, setting up the stack, and see some example queries to get you going.

Helm Chart Exporter: Ever wanted to know what versions of software are running on your Kubernetes cluster? Now, assuming you’re using Helm to install everything, you can.


Grafana Plugin Update

We have four plugin updates to share this week. Updating Grafana plugins is easy – for on-prem Grafana, use the grafana-cli tool, or for Hosted Grafana update with one-click at grafana.com.

UPDATED PLUGIN

USGS Water Services Data Source – The most important change is that USGS moved their service URL and this now avoids the 301 error. Also, this now nicely picks the mean daily value when you zoom out the instantaneous service to a large time window.

Install

UPDATED PLUGIN

Clickhouse Data Source – Updates to Clickhouse Data Source 1.7 include:

  • Provide $adhoc macros for using ad-hoc filters in inner queries (thx to @vavrusa)
  • Allow to set custom query for ad-hoc filter via adhoc_query_filter variable
  • Provide new Round value $step for auto-rounding according to graph resolution changes

Install

UPDATED PLUGIN

Akumuli Data Source – Minor bug fixed in the latest issue. Limit on number of returned data-points was set using the maxDataPoints value. That values was too small in some cases.

Install

UPDATED PLUGIN

Breadcrumb Panel – The Breadcrumb Panel received some minor style tweaks and fixed an issue with the display of the panel title.

Install


We’re Hiring!

We’ve added new open positions at Grafana Labs! Do you love open source software? Do you thrive on tackling complex challenges to build the future? Want to work with awesome people? Be the next to join our team!

View our Open Positions


Upcoming Events

In between code pushes we like to speak at, sponsor and attend all kinds of conferences and meetups. We also like to make sure we mention other Grafana-related events happening all over the world. If you’re putting on just such an event, let us know and we’ll list it here.

Meetup Workshop: Monitoring with Prometheus and Grafana | Belfast, Northern Ireland – September 18, 2018:

If you’re in Belfast, or are going to be in September, this could be a great Meetup to attend. Topics include: Architecture, Prometheus, Alertmanager, Pushgateway, Telegraf, JMX exporter, Grafana, and more!

RSVP Now

Cloud Native Computing Stockholm | Stockholm, Sweden – September 25, 2018:

Torkel Ödegaard: What’s Going On at Grafana – In this session Torkel will provide an update on the project and discuss what’s new/what’s coming in the future. He’ll also cover some “best practices/interesting things seen in the wild,” and save some time for Q&A.

RSVP Now

CloudNative London 2018 | London, United Kingdom – September 26-28, 2018:

Tom Wilkie: Monitoring Kubernetes With Prometheus – In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOS’s Prometheus Operator, or using Prometheus Ksonnet Mixin.

Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.

Register Now

All Things Open 2018 | Raleigh, NC – October 21-23, 2018:

Tom Wilkie: The RED Method – How to Instrument your Services – The RED Method defines three key metrics you should measure for every microservice in your architecture; inspired by the USE Method from Brendan Gregg, it gives developers a template for instrumenting their services and building dashboards in a consistent, repeatable fashion.

In this talk we will discuss patterns of application instrumentation, where and when they are applicable, and how they can be implemented with Prometheus. We’ll cover Google’s Four Golden Signals, the RED Method, the USE Method, and Dye Testing. We’ll also discuss why consistency is an important approach for reducing cognitive load. Finally we’ll talk about the limitations of these approaches and what can be done to overcome them.

Register Now

OSMC 2018 | Nuremberg, Germany – November 5-8, 2018:

David Kaltschmidt: Logging is Coming to Grafana – Grafana is an OSS dashboarding platform with a focus on visualizing time series data as beautiful graphs. Now we’re adding support to show your logs inside Grafana as well. Adding support for log aggregation makes Grafana an even better tool for incident response: First, the metric graphs help in a visual zoning in on the issue. Then you can seamlessly switch over to view and search related log files, allowing you to better understand what your software was doing while the issue was occurring. The main part of this talk shows how to deploy the necessary parts for this integrated experience. In addition I’ll show the latest features of Grafana both for creating dashboards and maintaining their configuration. The last 10-15 will be reserved for a Q&A.

Register Now


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard or monitoring related tweet and show it off! #monitoringLove

Indeed! Be sure to checkout talks from the fine folks at FermiLab and CERN from previous GrafanaCons too.


How are we doing?

Hope you enjoyed this issue of TimeShift. What do you think? Are there other types of content you’d like to see here? Submit a comment on this issue below, or post something at our community forum.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.