Tag Archives: PRS

Community profile: Dave Akerman

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/community-profile-dave-akerman/

This column is from The MagPi issue 61. You can download a PDF of the full issue for free, or subscribe to receive the print edition through your letterbox or the digital edition on your tablet. All proceeds from the print and digital editions help the Raspberry Pi Foundation achieve our charitable goals.

The pinned tweet on Dave Akerman’s Twitter account shows a table displaying the various components needed for a high-altitude balloon (HAB) flight. Batteries, leads, a camera and Raspberry Pi, plus an unusually themed payload. The caption reads ‘The Queen, The Duke of York, and my TARDIS”, and sums up Dave’s maker career in a heartbeat.

David Akerman on Twitter

The Queen, The Duke of York, and my TARDIS 🙂 #UKHAS #RaspberryPi

Though writing software for industrial automation pays the bills, the majority of Dave’s time is spent in the world of high-altitude ballooning and the ever-growing community that encompasses it. And, while he makes some money sending business-themed balloons to near space for the likes of Aardman Animations, Confused.com, and the BBC, Dave is best known in the Raspberry Pi community for his use of the small computer in every payload, and his work as a tutor alongside the Foundation’s staff at Skycademy events.

Dave Akerman The MagPi Raspberry Pi Community Profile

Dave continues to help others while breaking records and having a good time exploring the atmosphere.

Dave has dedicated many hours and many, many more miles to assist with the Foundation’s Skycademy programme, helping to explore high-altitude ballooning with educators from across the UK. Using a Raspberry Pi and various other pieces of lightweight tech, Dave and Foundation staff member James Robinson explored the incorporation of high-altitude ballooning into education. Through Skycademy, educators were able to learn new skills and take them to the classroom, setting off their own balloons with their students, and recording the results on Raspberry Pis.

Dave Akerman The MagPi Raspberry Pi Community Profile

Dave’s most recent flight broke a new record. On 13 August 2017, his HAB payload was able to send back the highest images taken by any amateur flight.

But education isn’t the only reason for Dave’s involvement in the HAB community. As with anyone passionate about a specific hobby, Dave strives to break records. The most recent record-breaking flight took place on 13 August 2017, when Dave’s Raspberry Pi Zero HAB sent home the highest images taken by any amateur high-altitude balloon launch: at 43014 metres. No other HAB balloon has provided images from such an altitude, and the lightweight nature of the Pi Zero definitely helped, as Dave went on to mention on Twitter a few days later.

Dave Akerman The MagPi Raspberry Pi Community Profile

Dave is recognised as being the first person to incorporate a Raspberry Pi into a HAB payload, and continues to break records with the help of the little green board. More recently, he’s been able to lighten the load by using the Raspberry Pi Zero.

When the first Pi made its way to near space, Dave tore the computer apart in order to meet the weight restriction. The Pi in the Sky board was created to add the extra features needed for the flight. Since then, the HAT has experienced a few changes.

Dave Akerman The MagPi Raspberry Pi Community Profile

The Pi in the Sky board, created specifically for HAB flights.

Dave first fell in love with high-altitude ballooning after coming across the hobby in a video shared on a photographic forum. With a lifelong interest in space thanks to watching the Moon landings as a boy, plus a talent for electronics and photography, it seems a natural progression for him. Throw in his coding skills from learning to program on a Teletype and it’s no wonder he was ready and eager to take to the skies, so to speak, and capture the curvature of the Earth. What was so great about using the Raspberry Pi was the instant gratification he got from receiving images in real time as they were taken during the flight. While other devices could control a camera and store captured images for later retrieval, thanks to the Pi Dave was able to transmit the files back down to Earth and check the progress of his balloon while attempting to break records with a flight.

Dave Akerman The MagPi Raspberry Pi Community Profile Morph

One of the many commercial flights Dave has organised featured the classic children’s TV character Morph, a creation of the Aardman Animations studio known for Wallace and Gromit. Morph took to the sky twice in his mission to reach near space, and finally succeeded in 2016.

High-altitude ballooning isn’t the only part of Dave’s life that incorporates a Raspberry Pi. Having “lost count” of how many Pis he has running tasks, Dave has also created radio receivers for APRS (ham radio data), ADS-B (aircraft tracking), and OGN (gliders), along with a time-lapse camera in his garden, and he has a few more Pi for tinkering purposes.

The post Community profile: Dave Akerman appeared first on Raspberry Pi.

AWS Documentation is Now Open Source and on GitHub

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/aws-documentation-is-now-open-source-and-on-github/

Earlier this year we made the AWS SDK developer guides available as GitHub repos (all found within the awsdocs organization) and invited interested parties to contribute changes and improvements in the form of pull requests.

Today we are adding over 138 additional developer and user guides to the organization, and we are looking forward to receiving your requests. You can fix bugs, improve code samples (or submit new ones), add detail, and rewrite sentences and paragraphs in the interest of accuracy or clarity. You can also look at the commit history in order to learn more about new feature and service launches and to track improvements to the documents.

Making a Contribution
Before you get started, read the Amazon Open Source Code of Conduct and take a look at the Contributing Guidelines document (generally named CONTRIBUTING.md) for the AWS service of interest. Then create a GitHub account if you don’t already have one.

Once you find something to change or improve, visit the HTML version of the document and click on Edit on GitHub button at the top of the page:

This will allow you to edit the document in source form (typically Markdown or reStructuredText). The source code is used to produce the HTML, PDF, and Kindle versions of the documentation.

Once you are in GitHub, click on the pencil icon:

This creates a “fork” — a separate copy of the file that you can edit in isolation.

Next, make an edit. In general, as a new contributor to an open source project, you should gain experience and build your reputation by making small, high-quality edits. I’ll change “dozens of services” to “over one hundred services” in this document:

Then I summarize my change and click Propose file change:

I examine the differences to verify my changes and then click Create pull request:

Then I review the details and click Create pull request again:

The pull request (also known as a PR) makes its way to the Elastic Beanstalk documentation team and they get to decide if they want to accept it, reject it, or to engage in a conversation with me to learn more. The teams endeavor to respond to PRs within 48 hours, and I’ll be notified via GitHub whenever the status of the PR changes.

As is the case with most open source projects, a steady stream of focused, modest-sized pull requests is preferable to the occasional king-sized request with dozens of edits inside.

If I am interested in tracking changes to a repo over time, I can Watch and/or Star it:

If I Watch a repo, I’ll receive an email whenever there’s a new release, issue, or pull request for that service guide.

Go Fork It
This launch gives you another way to help us to improve AWS. Let me know what you think!

Jeff;

UK Govt. Met With Copyright Holders Dozens of Times in Just Three Months

Post Syndicated from Andy original https://torrentfreak.com/uk-govt-met-with-copyright-holders-dozens-of-times-in-just-three-months-180310/

While doing business with clients and suppliers is the usual day-to-day routine for most businesses, companies in the entertainment sector seem keener than most to spend time with those in power.

Whether there’s pressure to be applied in respect of upcoming changes in policy or long-term plans for modifying legislation, at least a few times a year news breaks of rightsholders having private meetings with officials. Most of the time, however, the head-to-heads fly under the radar.

This week, however, the UK government published a response to a Freedom of Information Request which asked for details of meetings between the government and copyright owner organizations, enforcement organizations, and collection societies (think BPI, MPA, FACT, Publishers Association, PRS, etc) including times, dates and topics discussed.

The request asked for details of meetings held between May 2016 and April 2017 but the government declined to provide all of this information since the effort required to extract the information “would exceed the cost limit.”

Given the amount of data published, this isn’t a surprise. Even though the government chose to limit the response to events held between January 16, 2017 and April 17, 2017, the meetings between the government and the above groups number in their dozens.

January 2017 got off to a pretty slow start but week three and beyond saw a flurry of meetings with groups and companies such as ITV, BBC, PRS for Music, Copyright Licensing Agency and several other organizations to discuss the EU’s Digital Single Market proposals.

On January 18, 2017 Time Warner had a meeting to discuss content protection and analytics, followed a day later by the Premier League who were booked in to discuss “illicit streaming devices” (a topic mirrored in March during a meeting with the Audiovisual Anti-Piracy Alliance).

Just a few days later the Police Intellectual Property Crime Unit held a “Partnership Working Group Meeting involving industry” and two days after that the police, Trading Standards, and the EU Police Agency convened to discuss enforcement activity.

January 26, 2017 saw an IP Outreach Workshop involving members of the IP Crime Group. This was potentially a big meeting. The IPCG consists of several regional police forces, PIPCU, National Crime Agency, Crown Prosecution Service, Department of Culture, Media and Sport, Trading Standards, HMRC, IFPI, BPI, FACT, Sky TV, PRS, FAST and the Publishers Association, to name just a few.

As the first month of the year was drawing to a close, Amazon met with the government to discuss “current procedures for removing copyright, design and trademark infringing material from their platform.” A similar meeting was held with eBay on February 1 and on February 20, Facebook had its turn on the same topic.

All three companies had come in for criticism from copyright holders for not doing enough to stem the tide of infringing content available on their platforms, particularly so-called Kodi boxes that provide access to movies, shows, and live TV.

However, in the months that followed they each responded positively, with eBay, Amazon and Facebook announcing restrictions on devices sold. While all three platforms still have a problem with infringing device sales, the situation appears to have improved since last year.

On the final day of January 2017, the MPAA attended a meeting to discuss the looming Digital Economy Bill and digital TV piracy. A couple of days later they were back again for a “business awareness seminar” with other big shots including the Alliance for IP, the Anti-Counterfeiting Group, Trading Standards and the Premier League.

However, given the dozens that took place, perhaps one of the more interesting meetings in terms of the mix of those in attendance took place February 7.

Titled “Organized Crime Task Force Meeting – Belfast” it was attended by the Police Service of Northern Ireland, the National Crime Agency, Trading Standards, HM Revenue and Customs, the Border Force, and (spot the odd one out) the Federation Against Copyright Theft.

This seems to suggest that FACT (a private company) is effectively embedded at the highest level of law enforcement, something that has made people very uncomfortable in the past.

Later in February, there was a roundtable meeting with the Alliance for IP, MPAA, Publishers’ Association, BPI, Premier League and Federation Against Copyright Theft (again) to discuss Brexit, the Digital Single Market, IP enforcement and industrial strategy. A similar meeting was held in March which was attended by UK Music, BPI, PRS, Featured Artists Coalition, and many more.

The full list of meetings, which number in their dozens for just a three-month period, can be found here pdf. Whether the volume is representative of other three-month periods isn’t clear but it seems reasonable to conclude that copyright organizations have the ears of government officials in the UK on an almost continual basis.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Grafana v5.0 Released

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2018/03/01/grafana-v5.0-released/

v5.0 Stable Released

We have been working on Grafana v5 for most of 2017 and it’s finally ready! This release is important
in a different way than previous releases as main focus has been on improving the core Grafana features and attributes.
That means vastly improved UX and page design, easier and more flexible dashboard building enabled by a
new grid layout system. Better support for large installations with the addition of Dashboard Folders, Teams and Permissions.
Improvements to provisioning/cloud-native setups by making datasources & dashboards configurable from files.

This is the most substantial update that Grafana has ever seen.

Download Grafana 5.0 Now

What’s New in Grafana v5.0

Video showing new features


New Dashboard Layout Engine

The new dashboard layout engine allows for much easier movement and sizing of panels, as other panels now move out of the way in
a very intuitive way. Panels are sized independently, so rows are no longer necessary to create layouts. This opens
up many new types of layouts where panels of different heights can be aligned easily. Checkout the new grid in the video
above or on the play site. All your existing dashboards will automatically migrate to the
new position system and look close to identical. The new panel position makes dashboards saved in v5.0 incompatible
with older versions of Grafana.

New UX

Almost every page has seen significant UX improvements. All pages (except dashboard pages) have a new tab-based layout that improves navigation between pages. The side menu has also changed quite a bit. You can still hide the side menu completely if you click on the Grafana logo.

Dashboard Settings

Dashboard pages have a new header toolbar where buttons and actions are now all moved to the right. All the dashboard
settings views have been combined with a side nav which allows you to easily move between different setting categories.

New Light Theme

This theme has not seen a lot of love in recent years and we felt it was time to give it a major overhaul. We are very happy with the result.

Dashboard Folders

The big new feature that comes with Grafana v5.0 is dashboard folders. Now you can organize your dashboards in folders,
which is very useful if you have a lot of dashboards or multiple teams.

  • New search design adds expandable sections for each folder, starred and recently viewed dashboards.
  • New manage dashboard pages enable batch actions and views for folder settings and permissions.
  • Set permissions on folders and have dashboards inherit the permissions.

Teams

A team is a new concept in Grafana v5. They are simply a group of users that can be used in the new permission system for dashboards and folders. Only an admin can create teams.
We hope to do more with teams in future releases like integration with LDAP and a team landing page.

Permissions

You can assign permissions to folders and dashboards. The default user role-based permissions can be removed and
replaced with specific teams or users enabling more control over what a user can see and edit.

Dashboard permissions only limits what dashboards & folders a user can view & edit not which
data sources a user can access nor what queries a user can issue.

Provisioning from configuration

In previous versions of Grafana, you could only use the API for provisioning data sources and dashboards.
But that required the service to be running before you started creating dashboards and you also needed to
set up credentials for the HTTP API. In v5.0 we decided to improve this experience by adding a new active
provisioning system that uses config files. This will make GitOps more natural as data sources and dashboards can
be defined via files that can be version controlled. We hope to extend this system to later add support for users, orgs
and alerts as well.

Data sources

Data sources can now be setup using config files. These data sources are by default not editable from the Grafana GUI.
It’s also possible to update and delete data sources from the config file. More info in the data source provisioning docs.

Dashboards

We also deprecated the [dashboard.json] in favor of our new dashboard provisioner that keeps dashboards on disk
in sync with dashboards in Grafana’s database. The dashboard provisioner has multiple advantages over the old
[dashboard.json] feature. Instead of storing the dashboard in memory we now insert the dashboard into the database,
which makes it possible to star them, use one as the home dashboard, set permissions and other features in Grafana that
expects the dashboards to exist in the database. More info in the dashboard provisioning docs

Graphite Tags & Integrated Function Docs

The Graphite query editor has been updated to support the latest Graphite version (v1.1) that adds
many new functions and support for querying by tags. You can now also view function documentation right in the query editor!

Read more on Graphite Tag Support.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

Download

Head to download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports & feedback!

Grafana 4.6 Released

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/10/26/grafana-4.6-released/

Release Highlights

The Grafana 4.6 release contains some exciting and much anticipated new additions:

This is a big release so check out the other features and fixes in the Changelog section below.

Annotations

Annotations provide a way to mark points on the graph with rich events. You can now add annotation events and regions right from the graph panel! Just hold CTRL/CMD + click or drag region to open the Add Annotation view. The
Annotations documentation is updated to include details on this new exciting feature.

Cloudwatch

Cloudwatch now supports alerting. You can now setup alert rules for any Cloudwatch metric!

Postgres

Grafana v4.6 now ships with a built-in datasource plugin for PostgreSQL. Have logs or metric data in Postgres? You can now visualize that data and
define alert rules on it like any of our other data sources.

Prometheus

New enhancements include support for instant queries (for a single point in time instead of a time range) and improvements to query editor in the form of autocomplete for label names and label values.

This makes exploring and filtering Prometheus data much easier.

Changelog

Here are just a few highlights from the Changelog.

New Features

  • Annotations: Add support for creating annotations from graph panel #8197
  • GCS: Adds support for Google Cloud Storage #8370 thx @chuhlomin
  • Prometheus: Adds /metrics endpoint for exposing Grafana metrics. #9187
  • Jaeger: Add support for open tracing using jaeger in Grafana. #9213
  • Unit types: New date & time unit types added, useful in singlestat to show dates & times. #3678, #6710, #2764
  • Prometheus: Add support for instant queries #5765, thx @mtanda
  • Cloudwatch: Add support for alerting using the cloudwatch datasource #8050, thx @mtanda
  • Pagerduty: Include triggering series in pagerduty notification #8479, thx @rickymoorhouse
  • Prometheus: Align $__interval with the step parameters. #9226, thx @alin-amana
  • Prometheus: Autocomplete for label name and label value #9208, thx @mtanda
  • Postgres: New Postgres data source #9209, thx @svenklemm
  • Datasources: Make datasource HTTP requests verify TLS by default. closes #9371, #5334, #8812, thx @mattbostock

Minor

  • SMTP: Make it possible to set specific HELO for smtp client. #9319
  • Alerting: Add diff and percent diff as series reducers #9386, thx @shanhuhai5739
  • Slack: Allow images to be uploaded to slack when Token is present #7175, thx @xginn8
  • Table: Add support for displaying the timestamp with milliseconds #9429, thx @s1061123
  • Hipchat: Add metrics, message and image to hipchat notifications #9110, thx @eloo
  • Kafka: Add support for sending alert notifications to kafka #7104, thx @utkarshcmu

Tech

  • Webpack: Changed from systemjs to webpack (see readme or building from source guide for new build instructions). Systemjs is still used to load plugins but now plugins can only import a limited set of dependencies. See PLUGIN_DEV.md for more details on how this can effect some plugins.

Download

Head to the v4.6 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports, helping out on our community site and providing feedback!

Self-Driving Cars Should Be Open Source

Post Syndicated from Bozho original https://techblog.bozho.net/self-driving-cars-open-source/

Self-driving cars are (will be) the pinnacle of consumer products automation – robot vacuum cleaners, smart fridges and TVs are just toys compared to self-driving cars. Both in terms of technology and in terms of impact. We aren’t yet on level 5 self driving cars , but they are behind the corner.

But as software engineers we know how fragile software is. And self-driving cars are basically software, so we can see all the risks involved with putting our lives in the hands anonymous (from our point of view) developers and unknown (to us) processes and quality standards. One may argue that this has been the case for every consumer product ever, but with software is different – software is way more complex than anything else.

So I have an outrageous proposal – self-driving cars should be open source. We have to be able to verify and trust the code that’s navigating our helpless bodies around the highways. Not only that, but we have to be able to verify if it is indeed that code that is currently running in our car, and not something else.

In fact, let me extend that – all cars should be open source. Before you say “but that will ruin the competitive advantage of manufacturers and will be deadly for business”, I don’t actually care how they trained their neural networks, or what their datasets are. That’s actually the secret sauce of the self-driving car and in my view it can remain proprietary and closed. What I’d like to see open-sourced is everything else. (Under what license – I’d be fine to even have it copyrighted and so not “real” open source, but that’s a separate discussion).

Why? This story about remote carjacking using the entertainment system of a Jeep is a scary example. Attackers that reverse engineer the car software can remotely control everything in the car. Why did that happen? Well, I guess it’s complicated and we have to watch the DEFCON talk.

And also read the paper, but a paragraph in wikipedia about the CAN bus used in most cars gives us a hint:

CAN is a low-level protocol and does not support any security features intrinsically. There is also no encryption in standard CAN implementations, which leaves these networks open to man-in-the-middle packet interception. In most implementations, applications are expected to deploy their own security mechanisms; e.g., to authenticate incoming commands or the presence of certain devices on the network. Failure to implement adequate security measures may result in various sorts of attacks if the opponent manages to insert messages on the bus. While passwords exist for some safety-critical functions, such as modifying firmware, programming keys, or controlling antilock brake actuators, these systems are not implemented universally and have a limited number of seed/key pair

I don’t know in what world it makes sense to even have a link between the entertainment system and the low-level network that operates the physical controls. As apparent from the talk, the two systems are supposed to be air-gapped, but in reality they aren’t.

Rookie mistakes were abound – unauthenticated “execute” method, running as root, firmware is not signed, hard-coded passwords, etc. How do we know that there aren’t tons of those in all cars out there right now, and in the self-driving cars of the future (which will likely use the same legacy technologies of the current cars)? Recently I heard a negative comment about the source code of one of the self-driving cars “players”, and I’m pretty sure there are many of those rookie mistakes.

Why this is this even more risky for self-driving cars? I’m not an expert in car programming, but it seems like the attack surface is bigger. I might be completely off target here, but on a typical car you’d have to “just” properly isolate the CAN bus. With self-driving cars the autonomous system that watches the surrounding and makes decisions on what to do next has to be connected to the CAN bus. With Tesla being able to send updates over the wire, the attack surface is even bigger (although that’s actually a good feature – to be able to patch all cars immediately once a vulnerability is discovered).

Of course, one approach would be to introduce legislation that regulates car software. It might work, but it would rely on governments to to proper testing, which won’t always be the case.

The alternative is to open-source it and let all the white-hats find your issues, so that you can close them before the car hits the road. Not only that, but consumers like me will feel safer, and geeks would be able to verify whether the car is really running the software it claims to run by verifying the fingerprints.

Richard Stallman might be seen as a fanatic when he advocates against closed source software, but in cases like … cars, his concerns seem less extreme.

“But the Jeep vulnerability was fixed”, you may say. And that might be seen as being the way things are – vulnerabilities appear, they get fixed, life goes on. No person was injured because of the bug, right? Well, not yet. And “gaining control” is the extreme scenario – there are still pretty bad scenarios, like being able to track a car through its GPS, or cause panic by controlling the entertainment system. It might be over wifi, or over GPRS, or even by physically messing with the car by inserting a flash drive. Is open source immune to those issues? No, but it has proven to be more resilient.

One industry where the problem of proprietary software on a product that the customer bought is … tractors. It turns out farmers are hacking their tractors, because of multiple issues and the inability of the vendor to resolve them in a timely manner. This is likely to happen to cars soon, when only authorized repair shops are allowed to touch anything on the car. And with unauthorized repair shops the attack surface becomes even bigger.

In fact, I’d prefer open source not just for cars, but for all consumer products. The source code of a smart fridge or a security camera is trivial, it would rarely mean sacrificing competitive advantage. But refrigerators get hacked, security cameras are active part of botnets, the “internet of shit” is getting ubiquitous. A huge amount of these issues are dumb, beginner mistakes. We have the right to know what shit we are running – in our frdges, DVRs and ultimatey – cars.

Your fridge may soon by spying on you, your vacuum cleaner may threaten your pet in demand of “ransom”. The terrorists of the future may crash planes without being armed, can crash vans into crowds without being in the van, and can “explode” home equipment without being in the particular home. And that’s not just a hypothetical.

Will open source magically solve the issue? No. But it will definitely make things better and safer, as it has done with operating systems and web servers.

The post Self-Driving Cars Should Be Open Source appeared first on Bozho's tech blog.

Grafana 4.5 Released

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/09/13/grafana-4.5-released/

Grafana v4.5 is now available for download. This release has some really significant improvements to Prometheus, Elasticsearch, MySQL and to the Table panel.

Prometheus Query Editor

The new query editor has full syntax highlighting. As well as auto complete for metrics, functions, and range vectors. There is also integrated function docs right from the query editor!

Elasticsearch: Add ad-hoc filters from the table panel

Create column styles that turn cells into links that use the value in the cell (or other other row values) to generate a url to another dashboard or system. Useful for
using the table panel as way to drilldown into dashboard with more detail or to ticket system for example.

Query Inspector

Query Inspector is a new feature that shows query requests and responses. This can be helpful if a graph is not shown or shows something very different than what you expected.
More information here.

Changelog

New Features

  • Table panel: Render cell values as links that can have an url template that uses variables from current table row. #3754
  • Elasticsearch: Add ad hoc filters directly by clicking values in table panel #8052.
  • MySQL: New rich query editor with syntax highlighting
  • Prometheus: New rich query editor with syntax highlighting, metric & range auto complete and integrated function docs. #5117

Enhancements

  • GitHub OAuth: Support for GitHub organizations with 100+ teams. #8846, thx @skwashd
  • Graphite: Calls to Graphite api /metrics/find now include panel or dashboad time range (from & until) in most cases, #8055
  • Graphite: Added new graphite 1.0 functions, available if you set version to 1.0.x in data source settings. New Functions: mapSeries, reduceSeries, isNonNull, groupByNodes, offsetToZero, grep, weightedAverage, removeEmptySeries, aggregateLine, averageOutsidePercentile, delay, exponentialMovingAverage, fallbackSeries, integralByInterval, interpolate, invert, linearRegression, movingMin, movingMax, movingSum, multiplySeriesWithWildcards, pow, powSeries, removeBetweenPercentile, squareRoot, timeSlice, closes #8261
  • Elasticsearch: Ad-hoc filters now use query phrase match filters instead of term filters, works on non keyword/raw fields #9095.

Breaking change

  • InfluxDB/Elasticsearch: The panel & data source option named “Group by time interval” is now named “Min time interval” and does now always define a lower limit for the auto group by time. Without having to use > prefix (that prefix still works). This should in theory have close to zero actual impact on existing dashboards. It does mean that if you used this setting to define a hard group by time interval of, say “1d”, if you zoomed to a time range wide enough the time range could increase above the “1d” range as the setting is now always considered a lower limit.

This option is now rennamed (and moved to Options sub section above your queries):
image|519x120

Datas source selection & options & help are now above your metric queries.
image|690x179

Minor Changes

  • InfluxDB: Change time range filter for absolute time ranges to be inclusive instead of exclusive #8319, thx @Oxydros
  • InfluxDB: Added paranthesis around tag filters in queries #9131

Bug Fixes

  • Modals: Maintain scroll position after opening/leaving modal #8800
  • Templating: You cannot select data source variables as data source for other template variables #7510
  • Security: Security fix for api vulnerability (in multiple org setups).

Download

Head to the v4.5 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports, helping out on our community site and providing feedback!

timeShift(GrafanaBuzz, 1w) Issue 5

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/07/21/timeshiftgrafanabuzz-1w-issue-5/

We cover a lot of ground in this week’s timeShift. From diving into building your own plugin, finding the right dashboard, configuration options in the alerting feature, to monitoring your local weather, there’s something for everyone. Are you writing an article about Grafana, or have you come across an article you found interesting? Please get in touch, we’ll add it to our roundup.


From the Blogosphere

  • Going open-source in monitoring, part III: 10 most useful Grafana dashboards to monitor Kubernetes and services: We have hundreds of pre-made dashboards ready for you to install into your on-prem or hosted Grafana, but not every one will fit your specific monitoring needs. In part three of the series, Sergey discusses is experiences with finding useful dashboards and shows off ten of the best dashboards you can install for monitoring Kubernetes clusters and the services deployed on them.

  • Using AWS Lambda and API gateway for server-less Grafana adapters: Sometimes you’ll want to visualize metrics from a data source that may not yet be supported in Grafana natively. With the plugin functionality introduced in Grafana 3.0, anyone can create their own data sources. Using the SimpleJson data source, Jonas describes how he used AWS Lambda and AWS API gateway to write data source adapters for Grafana.

  • How to Use Grafana to Monitor JMeter Non-GUI Results – Part 2: A few issues ago we listed an article for using Grafana to monitor JMeter Non-GUI results, which required a number of non-trivial steps to complete. This article shows of an easier way to accomplish this that doesn’t require any additional configuration of InfluxDB.

  • Programming your Personal Weather Chart: It’s always great to see Grafana used outside of the typical dev-ops usecase. This article runs you through the steps to create your own weather chart and show off your local weather stats in Grafana. BONUS: Rob shows off a magic mirror he created, which can display this data.

  • vSphere Performance data – Part 6 – The Dashboard(s): This 6-part series goes into a ton of detail and walks you through the various methods of retrieving vSphere performance data, storing the data in a TSDB, and creating dashboards for the metrics. Part 6 deals specifically with Grafana, but I highly recommend reading all of the articles, as it chronicles the journey of metrics exploration, storage, and visualization from someone who had no prior experience with time series data.

  • Alerting in Grafana: Alerting in Grafana is a fairly new feature and one that we’re continuing to iterate on. We’re soon adding additional data source support, new notification channels, clustering, silencing rules, and more. This article steps you through all the configuration options to get you to your first alert.


Plugins and Dashboards

It can seem like work slows during July and August, but we’re still seeing a lot of activity in the community. This week we have a new graph panel to show off that gives you some unique looking dashboards, and an update to the Zabbix data source, which adds some really great features. You can install both of the plugins now on your on-prem Grafana via our cli, or with one-click on GrafanaCloud.

NEW PLUGIN

Bubble Chart Panel This super-cool looking panel groups your tag values into clusters of circles. The size of the circle represents the aggregated value of the time series data. There are also multiple color schemes to make those bubbles POP (pun intended)! Currently it works against OpenTSDB and Bosun, so give it a try!

Install Now

UPDATED PLUGIN

Zabbix Alex has been hard at work, making improvements on the Zabbix App for Grafana. This update adds annotations, template variables, alerting and more. Thanks Alex! If you’d like to try out the app, head over to http://play.grafana-zabbix.org/dashboard/db/zabbix-db-mysql?orgId=2

Install 3.5.1 Now


This week’s MVC (Most Valuable Contributor)

Open source software can’t thrive without the contributions from the community. Each week we’ll recognize a Grafana contributor and thank them for all of their PRs, bug reports and feedback.

mk-dhia (Dhia)
Thank you so much for your improvements to the Elasticsearch data source!


Tweet of the Week

We scour Twitter each week to find an interesting/beautiful dashboard and show it off! #monitoringLove

This week’s tweet comes from @geek_dave

Great looking dashboard Dave! And thank you for adding new features and keeping it updated. It’s creators like you who make the dashboard repository so awesome!


Upcoming Events

We love when people talk about Grafana at meetups and conferences.

Monday, July 24, 2017 – 7:30pm | Google Campus Warsaw


Ząbkowska 27/31, Warsaw, Poland

Iot & HOME AUTOMATION #3 openHAB, InfluxDB, Grafana:
If you are interested in topics of the internet of things and home automation, this might be a good occasion to meet people similar to you. If you are into it, we will also show you how we can all work together on our common projects.

RSVP


Tell us how we’re Doing.

We’d love your feedback on what kind of content you like, length, format, etc – so please keep the comments coming! You can submit a comment on this article below, or post something at our community forum. Help us make this better.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

timeShift(GrafanaBuzz, 1w) Issue 3

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/07/07/timeshiftgrafanabuzz-1w-issue-3/

Many in the US were on holiday for Independence Day earlier this week, but that didn’t slow us down: team Stockholm even shipped a new Grafana release. This issue of timeShift has plenty of great articles to highlight. If you know of a recent article about Grafana, or are writing one yourself, please get in touch, we’d be happy to feature it here.


Grafana 4.4 Released

Grafana v4.4 is now Available for download

Dashboard history and version control is here! A big thanks to Walmart Labs for their massive code contribution.

Check out what’s new in Grafana 4.4 in the release announcement.


From the Blogosphere

Plugins and Dashboards

We are excited that there have been over 100,000 plugin installations since we launched the new plugable architecture in Grafana v3. You can discover and install plugins in your own on-premises or Hosted Grafana instance from our website. Below are some recent additions and updates.

Zabbix Updated to v3.5.0 CHANGELOG.md

  • rate() function, which calculates per-second rate for growing counters.
  • Template query format. New format is {group}{host}{app}{item}. It allows to use names with dot.
  • Improved performance of groupBy() functions (at 6-10x faster than old).
  • lots of bug fixes and more

In addition to the plugins available for download, there are hundreds of pre-made dashboards ready for you to import into Grafana to get up and running quickly. Check out some of the popular dashboards.

Server Metrics (Collectd) Collectd/Graphite Server metrics dashboard (Load,CPU, Memory, Temp etc).

Data Source: Graphite | Collector: Collectd

Apache Overview System stats for uptime, cpu count, RAM, free memory %, and panels for load, I/O and network traffic. Apache workers and scoreboard panels and uptime and CPU load single stats.

Data Source: InfluxDB | Collector: Telegraf

Node Exporter Server Metrics A simple dashboard configured to be able to view multiple servers side by side.

Data Source: Prometheus | Collector: Nodeexporter

This week’s MVC (Most Valuable Contributor)

Each week we’ll recognize a Grafana contributor and thank them for all of their PRs, bug reports and feedback. Many of the fixes and improvements come from our fantastic community!

ryantxu (Ryan McKinley)

Ryan has contributed PR’s to Grafana as well as being the author of 4 well-maintained plugins (Ajax Panel, Discrete Panel, Plotly Panel and Influx Admin plugins). Thank you for all your hard work!

What do you think?

Anything in particular you’d like to see in this series of posts? Too long? Too short? Boring? Let us know. Comment on this article below, or post something at our community forum. With your help, we can make this a worthwhile resource.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

Grafana 4.4 Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/07/04/grafana-4.4-release/

Grafana v4.4 is now available for download.

Release Highlights

The main highlight of this release is the new Dashboard History feature that was co-developed with Walmart Labs. Big thanks to them for this massive contribution!

Dashboard History

This features introduces version control for dashboards. Every time a dashboard is saved, a copy of the dashboard is saved as a previous version. A dashboard version can be compared with other versions and a previous version can be restored.

The diff feature provides a textual summary of the comparison as well as a more detailed JSON diff.

Other Highlights From The Changelog

Enhancements:

  • Elasticsearch: Added filter aggregation label #8420, thx @tianzk
  • Sensu: Added option for source and handler #8405, thx @joemiller
  • CSV: Configurable csv export datetime format #8058, thx @cederigo
  • Table Panel: Column style that preserves formatting/indentation (like pre tag) #6617
  • DingDing: Add DingDing Alert Notifier #8473 thx @jiamliang

Minor Enhancements

Bug Fixes:

  • Graph: Bug fix for negative values in histogram mode #8628

Lots more enhancements and fixes can be found in the Changelog.

Download

Head to the v4.4 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports, helping out on our community site and providing feedback!

timeShift(GrafanaBuzz, 1w) Issue 2

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/06/30/timeshiftgrafanabuzz-1w-issue-2/

A big thank you to everyone for the likes, retweets, comments and questions from last week’s timeShift debut. We were delighted to learn that people found this new resource useful, and are excited to continue to publish weekly issues. If you know of a recent article about Grafana, or are writing one yourself, please get in touch, we’d be happy to feature it here.

From the Blogosphere

Plugins and Dashboards

We are excited that there have been over 100,000 plugin installations since we launched the new plugable architecture in Grafana v3. You can discover and install plugins in your own on-premises or Hosted Grafana instance from our website. Below are some recent additions and updates.

SimpleJson SimpleJson is a generic backend datasource that has been the foundation of a number of Grafana data source plugins. It’s also a mechanism by which any application can expose metrics over http directly to Grafana. The newest version adds basic auth.

NetXMS Grafana datasource for NetXMS open source monitoring system.

GoogleCalendar This plugin shows the event description as an annotation on your graphs.

Discrete Panel Show discrete values in a horizontal graph. This panel now supports results from the table format.

Alarm Box This panel shows the total count of values across all series. This update adds a new option to customize how the display and color values are calculated.

Status Dot This panel shows a colored dot for each series; useful to monitor latest values at a glance.

This week’s MVC (Most Valuable Contributor)

Each week we’ll recognize a Grafana contributor and thank them for all of their PRs, bug reports and feedback. A majority of fixes and improvements come from our fantastic community!

mtanda (Mitsuhiro Tanda)

159 PR’s during the last 2 years and still going strong. Thank you for your contributions mtanda!

What do you think?

Anything in particular you’d like to see in this series of posts? Too long? Too short? Boring? Let us know. Comment on this article below, or post something at our community forum. With your help, we can make this a worthwhile resource.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

timeShift(GrafanaBuzz, 1w) Issue 1

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/06/23/timeshiftgrafanabuzz-1w-issue-1/

Introducing timeShift

TimeShift is a new blog series we’ve created to provide a weekly curated list of links and articles centered around Grafana and the growing Grafana community. Each week we come across great articles from people who have written about how they are using Grafana, how to build effective dashboards, and a lot of discussion about the state of open source monitoring. We want to collect this information in one place and post an article every Friday afternoon highlighting some of this great content.

From the Blogosphere

We see a lot of articles covering the devops side of monitoring, but it’s interesting to see how people are using Grafana for different use cases.

Plugins and Dashboards

We are excited that there have been over 100,000 plugin installations since we launched the new plugable architecture in Grafana v3. You can discover and install plugins in your own on-premises or Hosted Grafana instance from our website. Below are some recent additions and updates.

Carpet plot A varient of the heatmap graph panel with additional display options.

DalmatinerDB No-fluff, purpose-built metric database.

Gnocchi This plugin was renamed. Users should uninstall the old version and install this new version.

This week’s MVC (Most Valuable Contributor)

Each week we’ll recognize a Grafana contributor and thank them for all of their PRs, bug reports and feedback. A majority of fixes and improvements come from our fantastic community!

thuck (Denis Doria)

Thank you for all of your PRs!

What do you think?

Anything in particular you’d like to see in this series of posts? Too long? Too short? Boring as shit? Let us know. Comment on this article below, or post something at our community forum. With your help, we can make this a worthwhile resource.

Follow us on Twitter, like us on Facebook, and join the Grafana Labs community.

maltrail – Malicious Traffic Detection System

Post Syndicated from Darknet original http://feedproxy.google.com/~r/darknethackers/~3/j8PZvwn91Sk/

Maltrail is a malicious traffic detection system, utilizing publicly available (black)lists containing malicious and/or generally suspicious trails, along with static trails compiled from various AV reports and custom user-defined lists, where trail can be anything from domain name (e.g. zvpprsensinaix.com for Banjori malware), URL (e.g….

Read the full post at darknet.org.uk

Grafana 4.3 Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/05/23/grafana-4.3-release/

Grafana v4.3 is now available for download.

Release Highlights

  • New Heatmap Panel
  • Graph Panel Histogram Mode
  • Elasticsearch Histogram Aggregation
  • Prometheus Table data format
  • New MySQL Data Source (alpha version to get some early feedback)
  • Dashed lines in the Graph Panel
  • 60+ small fixes and improvements, most of them contributed by our fantastic community!

Check out the New Features in v4.3 Dashboard on the Grafana Play site for
a showcase of these new features.

Histogram Support

A Histogram is a kind of bar chart that groups numbers into ranges, often called buckets or bins. Taller bars show that more data falls in that range.

The Graph Panel now supports Histograms.

Histogram Aggregation Support for Elasticsearch

Elasticsearch is the only supported data source that can return pre-bucketed data (data that is already grouped into ranges). With other data sources there is a risk of returning inaccurate data in a histogram due to using already aggregated data rather than raw data. This release adds support for Elasticsearch pre-bucketed data that can be visualized with the new Heatmap Panel.

Heatmap Panel

The Histogram support in the Graph Panel does not show changes over time – it aggregates all the data together for the chosen time range. To visualize a histogram over time, we have built a new Heatmap Panel.

Every column in a Heatmap is a histogram snapshot. Instead of visualizing higher values with higher bars, a heatmap visualizes higher values with color. The histogram shown above is equivalent to one column in the heatmap shown below.

The Heatmap panel also works with Elasticsearch Histogram Aggregations for more accurate server side bucketing.

MySQL Data Source (alpha)

This release includes a new core data source for MySQL. You can write any possible MySQL query and format it as either Time Series or Table Data allowing it be used with the Graph Panel, Table Panel and SingleStat Panel.

We are still working on the MySQL data source. As it’s missing some important features, like templating and macros and future changes could be breaking, we are
labeling the state of the data source as Alpha. Instead of holding up the release of v4.3 we are including it in its current shape to get some early feedback. So please try it out and let us know what you think on twitter or on our community forum. Is this a feature that you would use? How can we make it better?

The query editor can show the generated and interpolated SQL that is sent to the MySQL server.

The query editor will also show any errors that resulted from running the query (very useful when you have a syntax error!).

Dashed Lines in the Graph Panel

A new Dashes option has been added to Series overrides in the Graph Panel.

Health Check Endpoint

Now you can monitor the monitoring with the Health Check Endpoint! The new /api/health endpoint returns HTTP 200 OK if everything is up and HTTP 503 Error if the Grafana database cannot be pinged.

Lazy Load Panels

Grafana now delays loading panels until they become visible (scrolled into view). This means panels out of view are not sending requests thereby reducing the load on your time series database.

Prometheus – Table Data (column per label)

The Prometheus data source now supports the Table Data format by automatically assigning a column to a label. This makes it really easy to browse data in the table panel.

Improved Alerting Annotation

When an alert is fired, the annotation on the graph now shows information about execution errors or if it was fired due to no data.

Other Highlights From The Changelog

Changes:

  • Table: Support to change column header text #3551
  • InfluxDB: influxdb query builder support for ORDER BY and LIMIT (allows TOPN queries) #6065 Support influxdb’s SLIMIT Feature #7232 thx @thuck
  • Graph: Support auto grid min/max when using log scale #3090, thx @bigbenhur
  • Prometheus: Make Prometheus query field a textarea #7663, thx @hagen1778
  • Server: Support listening on a UNIX socket #4030, thx @mitjaziv

Fixes:

  • MySQL: 4-byte UTF8 not supported when using MySQL database (allows Emojis in Dashboard Names) #7958
  • Dashboard: Description tooltip is not fully displayed #7970

Lots more enhancements and fixes can be found in the Changelog.

Download

Head to the v4.3 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports, helping out on our community site and providing feedback!

Grafana 4.3 Beta Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/05/12/grafana-4.3-beta-release/

Grafana v4.3 Beta is now available for download.

Release Highlights

  • New Heatmap Panel
  • Graph Panel Histogram Mode
  • Elasticsearch Histogram Aggregation
  • Prometheus Table data format
  • New MySQL Data Source (alpha version to get some early feedback)
  • 60+ small fixes and improvements, most of them contributed by our fantastic community!

Check out the New Features in v4.3 Dashboard on the Grafana Play site for
a showcase of these new features.

Histogram Support

A Histogram is a kind of bar chart that groups numbers into ranges, often called buckets or bins. Taller bars show that more data falls in that range.

The Graph Panel now supports Histograms.

Histogram Aggregation Support for Elasticsearch

Elasticsearch is the only supported data source that can return pre-bucketed data (data that is already grouped into ranges). With other data sources there is a risk of returning inaccurate data in a histogram due to using already aggregated data rather than raw data. This release adds support for Elasticsearch pre-bucketed data that can be visualized with the new Heatmap Panel.

Heatmap Panel

The Histogram support in the Graph Panel does not show changes over time – it aggregates all the data together for the chosen time range. To visualize a histogram over time, we have built a new Heatmap Panel.

Every column in a Heatmap is a histogram snapshot. Instead of visualizing higher values with higher bars, a heatmap visualizes higher values with color. The histogram shown above is equivalent to one column in the heatmap shown below.

The Heatmap panel also works with Elasticsearch Histogram Aggregations for more accurate server side bucketing.

MySQL Data Source (alpha)

This release includes a new core data source for MySQL. You can write any possible MySQL query and format it as either Time Series or Table Data allowing it be used with the Graph Panel, Table Panel and SingleStat Panel.

We are still working on the MySQL data source. As it’s missing some important features, like templating and macros and future changes could be breaking, we are
labeling the state of the data source as Alpha. Instead of holding up the release of v4.3 we are including it in its current shape to get some early feedback. So please try it out and let us know what you think on twitter or on our community forum. Is this a feature that you would use? How can we make it better?

The query editor can show the generated and interpolated SQL that is sent to the MySQL server.

The query editor will also show any errors that resulted from running the query (very useful when you have a syntax error!).

Health Check Endpoint

Now you can monitor the monitoring with the Health Check Endpoint! The new /api/health endpoint returns HTTP 200 OK if everything is up and HTTP 503 Error if the Grafana database cannot be pinged.

Lazy Load Panels

Grafana now delays loading panels until they become visible (scrolled into view). This means panels out of view are not sending requests thereby reducing the load on your time series database.

Prometheus – Table Data (column per label)

The Prometheus data source now supports the Table Data format by automatically assigning a column to a label. This makes it really easy to browse data in the table panel.

Other Highlights From The Changelog

Changes:

  • Table: Support to change column header text #3551
  • InfluxDB: influxdb query builder support for ORDER BY and LIMIT (allows TOPN queries) #6065 Support influxdb’s SLIMIT Feature #7232 thx @thuck
  • Graph: Support auto grid min/max when using log scale #3090, thx @bigbenhur
  • Prometheus: Make Prometheus query field a textarea #7663, thx @hagen1778
  • Server: Support listening on a UNIX socket #4030, thx @mitjaziv

Fixes:

  • MySQL: 4-byte UTF8 not supported when using MySQL database (allows Emojis in Dashboard Names) #7958
  • Dashboard: Description tooltip is not fully displayed #7970

Lots more enhancements and fixes can be found in the Changelog.

Download

Head to the v4.3 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports, helping out on our community site and providing feedback!

Analyze your GitHub Project With Elasticsearch And Grafana

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/05/10/analyze-your-github-project-with-elasticsearch-and-grafana/

The Dream

I have, for a long time, wished there was a way to easily export GitHub issues and comments to
Elasticsearch. The standard GitHub graphs for commits and traffic are great but I have
really been missing graphs and analytics on issues and comments.

If we had issues & comments in Elasticsearch, with a well-defined index mapping, we could do some
interesting analytics. For example:

  • Look at project history in terms of issues created
  • Look at project history in terms of comments (can be a measure of community engagement)
  • See how different labels trend over time.
  • Look at distributions (histograms) on the number of issues or comments created per user. Are there a few very active users that represent 70% or 90% of all issues & comments?
  • How long do PRs stay open?
  • How long until issues get their first response?

Why Elasticsearch?

Grafana is most often used with time series databases like Graphite, but for this sort of use case,
it’s about much more than measurements. Part of the power of Grafana is bringing together data from
many different places, and leveraging the strengths of its diverse set of data sources.

Elasticsearch isn’t technically a time series database, but it’s been one of our fastest growing data source
because it really shines for use cases like this. Plus, Grafana’s support for Elasticsearch is getting
better and better.

Elasticsearch is not only a document search DB. Its real power is in the kinds of aggregations you can do. It’s not ideal
for the high volume & high-resolution time series workloads that most time series databases can handle, but for
data with high cardinality (like documents with usernames, issue numbers, etc) it can really shine. It also allows
you to do ad-hoc filtering in a way that time series would not allow, as it would require a unique time series
for every possible filter condition and value.

The GitHub API Crawler

So a few weekends ago I had some left over programming energy and spent a few hours hacking together
this node.js app that uses the GitHub API to crawl all issues and comments which it
then saves as separate documents in Elasticsearch.

It stores them in Elasticsearch with this index mapping:

"mappings": {
  "issue": {
    "properties": {
      "title":            { "type": "text"  },
        "state":            { "type": "keyword"  },
        "repo":             { "type": "keyword"  },
        "labels":           { "type": "keyword"  },
        "number":           { "type": "keyword"  },
        "comments":         { "type": "long"  },
        "assignee":         { "type": "keyword"  },
        "user_login":       { "type": "keyword"  },
        "milestone":        { "type": "keyword"  },
        "created_at":       { "type": "date"  },
        "closed_at":        { "type": "date"  },
        "updated_at":       { "type": "date"  },
        "is_pull_request":  { "type": "boolean"  },
    }
  },
    "comment": {
      "properties": {
        "issue":           { "type": "keyword"  },
        "repo":            { "type": "keyword"  },
        "user_login":      { "type": "keyword"  },
        "created_at":      { "type": "date"     },
      }
    }
}

There are some more numeric fields being saved for reactions that do not need to be defined
in the index mapping.

The Dashboards

With the data finally collected, I built two dashboards; one focused on issues and another one
focused on comments. Both dashboards are templated and allow you to specify which repository
to look at and the granularity (group by time) of the data. You can also add any ad-hoc filter. For example,
only look at issues created by a specific user, or only look at issues with no comments.

Check out the dasboard on our play site. I configured the
github-to-es collector to fetch issues and comments for the main Kubernetes repo, the
main Grafana repo and the Microsoft VS Code editor repository.

The second dashboard shows comment analytics:

Useful How?

I am not exactly sure how useful this data & dashboards are yet. It was mostly a fun hobby project to see some trends and stats
for issue and comment volume. But this could also be useful data that can help you track things like issue label stats. Stats that could
be used to improve categorizing issues and visualizing changes in labeling trends. For example, the graphs could answer questions like:
How did a concerted effort to improve docs change the trend of issues labeled question?

Try it and help me improve it

Check out the GitHub repo grafana/github-to-es it has a basic README with instructions
for how to get started.

Once you have the import working you need to add an Elasticsearch data source in Grafana. For index name you specify github
and for the Timestamp field you specify created_at. Then you can import the the two dashboards i published on Grafana.com:

There are some limitations for how many issues and comments that can be imported in the initial full import due to the paging limit
in the GitHub API. GitHub API returns a maximum of 100 issues or comments per “page” and has a page limit of a maximum of 400 pages. This
means that the full import can only handle 40,000 issues and 40,000 comments.

More data & more cool graphs

There are probably many more interesting queries you can build and the collector could also be improved to fetch and store more fields.

For example:

  • Collect stars & fork stats (needs to be recorded as snapshot docs as there is no API to get historical data for this)
  • Calculate time between issue created and first comment during issue fetching to have that as a field on the issue docs
  • PR details, currently issue API does not include merge status (only a flag if its a PR)
  • Commit docs

There are probably a lot more cool things you can collect & query.

Until next time, keep on graphing!
Torkel Ödegaard
Grafana Creator & Project Lead

Grafana 4.2 Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/03/22/grafana-4.2-release/

Grafana v4.2 Stable is now available for download.
Just like the last release this one contains lots bug fixes and minor improvements.
We are very happy to say that 41 of 53 issues was closed by pull requests from the community.
Big thumbs up!

Release Highlights

  • Hipchat: Adds support for sending alert notifications to hipchat #6451, thx @jregovic
  • Telegram: Added Telegram alert notifier #7098, thx @leonoff
  • LINE: Add LINE as alerting notification channel #7301, thx @huydx
  • Templating: Make $__interval and $__interval_ms global built in variables that can be used in by any datasource (in panel queries), closes #7190, closes #6582
  • Alerting: Adds deduping of alert notifications #7632
  • Alerting: Better information about why an alert triggered #7035
  • Orgs: Sharing dashboards using Grafana share feature will now redirect to correct org. #6948
  • Full changelog

New alert notification channels

This release adds five new alert notifications channels, all of them contributed by the community.

  • Hipchat
  • Telegram
  • LINE
  • Pushover
  • Threema

Templating

We added two new global built in variables in grafana. $__interval and $__interval_ms are now reserved template names in grafana and can be used by any datasource.
We might add more global built in variables in the future and if we do we will prefix them with $__. So please avoid using that in your template variables.

Dedupe alert notifications when running multiple servers

In this release we will dedupe alert notificiations when you are running multiple servers.
This makes it possible to run alerting on multiple servers and only get one notification.

We currently solve this with sql transactions which puts some limitations for how many servers you can use to execute the same rules.
3-5 servers should not be a problem but as always, it depends on how many alerts you have and how frequently they execute.

Next up for a better HA situation is to add support for workload balancing between Grafana servers.

Alerting more info

You can now see the reason why an alert triggered in the alert history. Its also easier to detect when an alert is set to alerting due to the no_data option.

Improved support for multi-org setup

When loading dashboards we now set an query parameter called orgId. So we can detect from which org an user shared a dashboard.
This makes it possible for users to share dashboards between orgs without changing org first.

We aim to introduce dashboard groups sometime in the future which will introduce access control and user groups within one org.
Making it possible to have users in multiple groups and have detailed access control.

Upgrade & Breaking changes

If your using https in grafana we now force you to use tls 1.2 and the most secure ciphers.
We think its better to be secure by default rather then making it configurable.
If you want to run https with lower versions of tls we suggest you put a reserve proxy in front of grafana.

If you have template variables name $__interval or $__interval_ms they will no longer work since these keywords
are reserved as global built in variables. We might add more global built in variables in the future and if we do, we will prefix them with $__. So please avoid using that in your template variables.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

Download

Head to v4.2 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports & feedback!

Grafana 4.2 Beta Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/02/27/grafana-4.2-beta-release/

Grafana v4.2 beta

Grafana v4.2 Beta is now available for download.
Just like the last release this one contains lots bug fixes and minor improvements.
We are very happy to say that 27 of 40 issues was closed by pull requests from the community.
Big thumbs up!

Release Highlights

  • Hipchat: Adds support for sending alert notifications to hipchat #6451, thx @jregovic
  • Telegram: Added Telegram alert notifier #7098, thx @leonoff
  • LINE: Add LINE as alerting notification channel #7301, thx @huydx
  • Templating: Make $__interval and $__interval_ms global built in variables that can be used in by any datasource (in panel queries), closes #7190, closes #6582
  • Alerting: Adds deduping of alert notifications #7632
  • Alerting: Better information about why an alert triggered #7035
  • Orgs: Sharing dashboards using Grafana share feature will now redirect to correct org. #6948
  • Full changelog

New alert notification channels

This release adds five new alert notifications channels, all of them contributed by the community.

  • Hipchat
  • Telegram
  • LINE
  • Pushover
  • Threema

Templating

We added two new global built in variables in grafana. $__interval and $__interval_ms are now reserved template names in grafana and can be used by any datasource.
We might add more global built in variables in the future and if we do we will prefix them with $__. So please avoid using that in your template variables.

Dedupe alert notifications when running multiple servers

In this release we will dedupe alert notificiations when you are running multiple servers.
This makes it possible to run alerting on multiple servers and only get one notification.

We currently solve this with sql transactions which puts some limitations for how many servers you can use to execute the same rules.
3-5 servers should not be a problem but as always, it depends on how many alerts you have and how frequently they execute.

Next up for a better HA situation is to add support for workload balancing between Grafana servers.

Alerting more info

You can now see the reason why an alert triggered in the alert history. Its also easier to detect when an alert is set to alerting due to the no_data option.

Improved support for multi-org setup

When loading dashboards we now set an query parameter called orgId. So we can detect from which org an user shared a dashboard.
This makes it possible for users to share dashboards between orgs without changing org first.

We aim to introduce dashboard groups sometime in the future which will introduce access control and user groups within one org.
Making it possible to have users in multiple groups and have detailed access control.

Upgrade & Breaking changes

If your using https in grafana we now force you to use tls 1.2 and the most secure ciphers.
We think its better to be secure by default rather then making it configurable.
If you want to run https with lower versions of tls we suggest you put a reserve proxy in front of grafana.

If you have template variables name $__interval or $__interval_ms they will no longer work since these keywords
are reserved as global built in variables. We might add more global built in variables in the future and if we do, we will prefix them with $__. So please avoid using that in your template variables.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

Download

Head to v4.2-beta download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports & feedback!

Grafana 4.1 Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/01/11/grafana-4.1-release/

Grafana v4.1 is now available for download.
The new version contains a lot of improvement and the majority of them come from pull requests.
The fact that we can release 4.1.0 so quickly after 4.0 is all thanks to the Grafana community and all the contributors.
Big thumbs up!

Release Highlights

  • Graph: Support for shared tooltip on all graphs as you hover over one graph. #1578, #6274
  • Victorops: Add VictorOps notification integration #6411, thx @ichekrygin
  • Opsgenie: Add OpsGenie notification integratiion #6687, thx @kylemcc
  • Cloudwatch: Make it possible to specify access and secret key on the data source config page #6697
  • Elasticsearch: Added support for Elasticsearch 5.x #5740, thx @lpic10
  • Panel: Added help text for panels. #4079, thx @utkarshcmu
  • Full changelog

Shared tooltip

Showing the tooltip on all panels at the same time has been a long standing request in Grafana and we are really happy to finally be able to release it.
You can enable/disable the shared tooltip from the dashboard settings menu or cycle between default, shared tooltip and shared crosshair by pressing CTRL + O or CMD + O.

Help text for panel

You can set a help text in the general tab on any panel. The help text is using Markdown to enable better formating and linking to other sites that can provide more information.

Panels with a help text available have a little indicator in the top left corner. You can show the help text by hovering over the icon.

Easier Cloudwatch configuration

In Grafana 4.1.0 you can configure your Cloudwatch data source with access key and secret key directly in the data source configuration page.
This enables people to use the Cloudwatch data source without having access to the filesystem where Grafana is running.

Once the access key and secret key have been saved the user will no longer be able to view them.

Upgrade & Breaking changes

Elasticsearch 1.x is no longer supported. Please upgrade to Elasticsearch 2.x or 5.x. Otherwise Grafan 4.1.0 contains no breaking changes.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

Download

Head to v4.1 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports & feedback!

Grafana 4.1 Beta Release

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2016/12/21/grafana-4.1-beta-release/

Grafana v4.1 Beta is now available for download.
The new version contains a lot of improvement and the majority of them come from pull requests.
The fact that we can release 4.1.0-beta1 so quickly after 4.0 is all thanks to the Grafana community and all the contributors.
Big thumbs up!

Release Highlights

  • Graph: Support for shared tooltip on all graphs as you hover over one graph. #1578, #6274
  • Victorops: Add VictorOps notification integration #6411, thx @ichekrygin
  • Opsgenie: Add OpsGenie notification integratiion #6687, thx @kylemcc
  • Cloudwatch: Make it possible to specify access and secret key on the data source config page #6697
  • Elasticsearch: Added support for Elasticsearch 5.x #5740, thx @lpic10
  • Panel: Added help text for panels. #4079, thx @utkarshcmu
  • Full changelog

Shared tooltip

Showing the tooltip on all panels at the same time has been a long standing request in Grafana and we are really happy to finally be able to release it.
You can enable/disable the shared tooltip from the dashboard settings menu or cycle between default, shared tooltip and shared crosshair by pressing CTRL + O or CMD + O.

Help text for panel

You can set a help text in the general tab on any panel. The help text is using Markdown to enable better formating and linking to other sites that can provide more information.

Panels with a help text available have a little indicator in the top left corner. You can show the help text by hovering over the icon.

Easier Cloudwatch configuration

In Grafana 4.1.0 you can configure your Cloudwatch data source with access key and secret key directly in the data source configuration page.
This enables people to use the Cloudwatch data source without having access to the filesystem where Grafana is running.

Once the access key and secret key have been saved the user will no longer be able to view them.

Upgrade & Breaking changes

Elasticsearch 1.x is no longer supported. Please upgrade to Elasticsearch 2.x or 5.x. Otherwise Grafan 4.1.0-beta1 contains no breaking changes.

Changelog

Checkout the CHANGELOG.md file for a complete list
of new features, changes, and bug fixes.

Download

Head to v4.1 download page for download links & instructions.

Thanks

A big thanks to all the Grafana users who contribute by submitting PRs, bug reports & feedback!