merge updates from upstream/master

Gemfile

@@ -13,3 +13,7 @@ gem 'rb-inotify', :require => false
 gem 'rb-fsevent', :require => false
 gem 'rb-fchange', :require => false
 gem 'nanoc-cachebuster'
+
+group :test do
+  gem 'rspec'
+end

Gemfile.lock

@@ -10,6 +10,7 @@ GEM
     colored (1.2)
     cri (2.6.1)
       colored (~> 1.2)
+    diff-lcs (1.2.5)
     em-websocket (0.5.1)
       eventmachine (>= 0.12.9)
       http_parser.rb (~> 0.6.0)
@@ -61,6 +62,19 @@ GEM
     rb-inotify (0.9.5)
       ffi (>= 0.5.0)
     redcarpet (3.2.0)
+    rspec (3.5.0)
+      rspec-core (~> 3.5.0)
+      rspec-expectations (~> 3.5.0)
+      rspec-mocks (~> 3.5.0)
+    rspec-core (3.5.4)
+      rspec-support (~> 3.5.0)
+    rspec-expectations (3.5.0)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.5.0)
+    rspec-mocks (3.5.0)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.5.0)
+    rspec-support (3.5.0)
     slop (3.6.0)
     thor (0.19.1)
     timers (4.0.1)
@@ -84,3 +98,7 @@ DEPENDENCIES
   rb-fsevent
   rb-inotify
   redcarpet
+  rspec
+
+BUNDLED WITH
+   1.11.2

content/blog/2015-06-01-advanced-service-discovery.md

@@ -90,7 +90,7 @@ Relabeling works as follows:
 - A regular expression is matched against the resulting string.
 - A new value based on those matches is assigned to another label.
 
-Mutiple relabeling rules can be defined for each scrape configuration. A simple one
+Multiple relabeling rules can be defined for each scrape configuration. A simple one
 that squashes two labels into one, looks as follows:
 
 ```

content/blog/2016-03-03-custom-alertmanager-templates.md

@@ -20,7 +20,7 @@ ones.
 In this blog post, we will walk through a simple customization of Slack
 notifications.
 
-We use this simple Alertmanager configuartion that sends all alerts to Slack:
+We use this simple Alertmanager configuration that sends all alerts to Slack:
 
 ```yaml
 global:

content/blog/2016-09-07-interview-with-shuttlecloud.md

@@ -5,7 +5,7 @@ kind: article
 author_name: Brian Brazil
 ---
 
-*Continuing our series of interviews with users of Prometheus, ShuttleCloud talks about how they began using Promethus. Ignacio from ShuttleCloud also explained how [Prometheus Is Good for Your Small Startup](https://www.youtube.com/watch?v=gMHa4Yh8avk) at PromCon 2016.*
+*Continuing our series of interviews with users of Prometheus, ShuttleCloud talks about how they began using Prometheus. Ignacio from ShuttleCloud also explained how [Prometheus Is Good for Your Small Startup](https://www.youtube.com/watch?v=gMHa4Yh8avk) at PromCon 2016.*
 
 ## What does ShuttleCloud do?
 

content/blog/2016-10-12-interview-with-justwatch.md

+---
+title: Interview with JustWatch
+created_at: 2016-10-12
+kind: article
+author_name: Brian Brazil
+---
+
+*Continuing our series of interviews with users of Prometheus, JustWatch talks
+about how they established their monitoring.*
+
+## Can you tell us about yourself and what JustWatch does?
+
+For consumers, [JustWatch](https://www.justwatch.com) is a streaming search
+engine that helps to find out where to watch movies and TV shows legally online
+and in theaters. You can search movie content across all major streaming
+providers like Netflix, HBO, Amazon Video, iTunes, Google Play, and many others
+in 17 countries.
+
+For our clients like movie studios or Video on Demand providers, we are an
+international movie marketing company that collects anonymized data about
+purchase behavior and movie taste of fans worldwide from our consumer apps. We
+help studios to advertise their content to the right audience and make digital
+video advertising a lot more efficient in minimizing waste coverage.
+
+![JustWatch logo](/assets/blog/2016-10-12/JW_logo_long_black.jpg)
+
+Since our launch in 2014 we went from zero to one of the largest 20k websites
+internationally without spending a single dollar on marketing - becoming the
+largest streaming search engine worldwide in under two years. Currently, with
+an engineering team of just 10, we build and operate a fully dockerized stack
+of about 50 micro- and macro-services, running mostly on
+[Kubernetes](https://kubernetes.io).
+
+## What was your pre-Prometheus monitoring experience?
+
+At prior companies many of us worked with most of the open-source monitoring
+products there are. We have quite some experience working with
+[Nagios](https://www.nagios.org/), [Icinga](https://www.icinga.org/),
+[Zabbix](http://www.zabbix.com/),
+[Monit](https://mmonit.com/monit/documentation/),
+[Munin](http://munin-monitoring.org/), [Graphite](https://graphiteapp.org/) and
+a few other systems. At one company I helped build a distributed Nagios setup
+with Puppet. This setup was nice, since new services automatically showed up in
+the system, but taking instances out was still painful. As soon as you have
+some variance in your systems, the host and service based monitoring suites
+just don’t fit quite well. The label-based approach Prometheus took was
+something I always wanted to have, but didn’t find before.
+
+
+## Why did you decide to look at Prometheus?
+
+At JustWatch the public Prometheus announcement hit exactly the right time. We
+mostly had blackbox monitoring for the first few months of the company -
+[CloudWatch](https://aws.amazon.com/cloudwatch/) for some of the most important
+internal metrics, combined with a external services like
+[Pingdom](https://www.pingdom.com/) for detecting site-wide outages. Also, none
+of the classical host-based solutions satisfied us. In a world of containers
+and microservices, host-based tools like Icinga,
+[Thruk](https://www.thruk.org/) or Zabbix felt antiquated and not ready for the
+job. When we started to investigate whitebox monitoring, some of us luckily
+attended the Golang Meetup where Julius and Björn announced Prometheus. We
+quickly set up a Prometheus server and started to instrument our Go services
+(we use almost only Go for the backend). It was amazing how easy that was - the
+design felt like being cloud- and service-oriented as a first principle and
+never got in the way.
+
+
+## How did you transition?
+
+Transitioning wasn't that hard, as timing wise, we were lucky enough to go from
+no relevant monitoring directly to Prometheus.
+
+The transition to Prometheus was mostly including the Go client into our apps
+and wrapping the HTTP handlers. We also wrote and deployed several exporters,
+including the [node_exporter](https://github.com/prometheus/node_exporter) and
+several exporters for cloud provider APIs. In our experience monitoring and
+alerting is a project that is never finished, but the bulk of the work was done
+within a few weeks as a side project.
+
+Since the deployment of Prometheus we tend to look into metrics whenever we
+miss something or when we are designing new services from scratch.
+
+It took some time to fully grasp the elegance of PromQL and labels concept
+fully, but the effort really paid off.
+
+
+## What improvements have you seen since switching?
+
+Prometheus enlightened us by making it incredibly easy to reap the benefits
+from whitebox monitoring and label-based canary deployments. The out-of-the-box
+metrics for many Golang aspects (HTTP Handler, Go Runtime) helped us to get to
+a return on investment very quickly - goroutine metrics alone saved the day
+multiple times. The only monitoring component we actually liked before -
+[Grafana](http://grafana.org/) - feels like a natural fit for Prometheus and
+has allowed us to create some very helpful dashboards. We appreciated that
+Prometheus didn't try to reinvent the wheel but rather fit in perfectly with
+the best solution out there. Another huge improvement on predecessors was
+Prometheus's focus on actually getting the math right (percentiles, etc.). In
+other systems, we were never quite sure if the operations offered made sense.
+Especially percentiles are such a natural and necessary way of reasoning about
+microservice performance that it felt great that they get first class
+treatment.
+
+![Database Dashboard](/assets/blog/2016-10-12/prometheus-dashboard-db.jpg)
+
+The integrated service discovery makes it super easy to manage the scrape
+targets. For Kubernetes, everything just works out-of-the-box. For some other
+systems not running on Kubernetes yet, we use a
+[Consul-based](https://www.consul.io/) approach. All it takes to get an
+application monitored by Prometheus is to add the client, expose `/metrics` and
+set one simple annotation on the Container/Pod. This low coupling takes out a
+lot of friction between development and operations - a lot of services are
+built well orchestrated from the beginning, because it's simple and fun.
+
+The combination of time-series and clever functions make for awesome alerting
+super-powers. Aggregations that run on the server and treating both
+time-series, combinations of them and even functions on those combinations as
+first-class citizens makes alerting a breeze - often times after the fact.
+
+## What do you think the future holds for JustWatch and Prometheus?
+
+While we value very much that Prometheus doesn't focus on being shiny but on
+actually working and delivering value while being reasonably easy to deploy and
+operate - especially the Alertmanager leaves a lot to be desired yet. Just some
+simple improvements like simplified interactive alert building and editing in
+the frontend would go a long way in working with alerts being even simpler.
+
+We are really looking forward to the ongoing improvements in the storage layer,
+including remote storage. We also hope for some of the approaches taken in
+[Project Prism](https://github.com/weaveworks/prism) and
+[Vulcan](https://github.com/digitalocean/vulcan) to be backported to core
+Prometheus. The most interesting topics for us right now are GCE Service
+Discovery, easier scaling, and much longer retention periods (even at the cost
+of colder storage and much longer query times for older events).
+
+We are also looking forward to use Prometheus for more non-technical
+departments as well. We’d like to cover most of our KPIs with Prometheus to
+allow everyone to create beautiful dashboards, as well as alerts. We're
+currently even planning to abuse the awesome alert engine for a new, internal
+business project as well - stay tuned!
+

content/community.html

@@ -55,6 +55,7 @@ title: Community
     <ul>
       <li><a href="http://www.robustperception.io">Robust Perception</a></li>
       <li><a href="http://juliusv.com">Julius Volz</a> (independent contractor)</li>
+      <li><a href="https://container-solutions.com">Container Solutions</a></li>
     </ul>
 
     <h1>Acknowledgements</h1>

content/docs/alerting/clients.md

@@ -12,7 +12,7 @@ recommended to configure alerting rules in Prometheus based on time series
 data rather than implementing a direct client.__
 
 The Alertmanager listens for alerts on an API endpoint at `/api/v1/alerts`.
-Clients are expected to continously re-send alerts as long as they are still
+Clients are expected to continuously re-send alerts as long as they are still
 active (usually on the order of 30 seconds to 3 minutes).
 Clients can push a list of alerts to that endpoint via a POST request of
 the following format:

content/docs/alerting/configuration.md

@@ -105,7 +105,7 @@ It then traverses the child nodes. If `continue` is set to false, it stops
 after the first matching child. If `continue` is true on a matching node, the
 alert will continue matching against subsequent siblings.
 If an alert does not match any children of a node (no matching child nodes, or
-none exist), the alert is handled based on the configuration paramters of the
+none exist), the alert is handled based on the configuration parameters of the
 current node.
 
 

content/docs/instrumenting/exporters.md

@@ -40,6 +40,7 @@ wide variety of JVM-based applications, for example [Kafka](http://kafka.apache.
    * [SQL query result set metrics exporter](https://github.com/chop-dbhi/prometheus-sql)
 
 ### Hardware related
+   * [apcupsd exporter](https://github.com/mdlayher/apcupsd_exporter)
    * [IoT Edison exporter](https://github.com/roman-vynar/edison_exporter)
    * [IPMI exporter](https://github.com/lovoo/ipmi_exporter)
    * [knxd exporter](https://github.com/RichiH/knxd_exporter)
@@ -64,7 +65,7 @@ wide variety of JVM-based applications, for example [Kafka](http://kafka.apache.
    * [Passenger exporter](https://github.com/stuartnelson3/passenger_exporter)
    * [Varnish exporter](https://github.com/jonnenauha/prometheus_varnish_exporter)
    * [WebDriver exporter](https://github.com/mattbostock/webdriver_exporter)
-   
+
 ### APIs
    * [Docker Hub exporter](https://github.com/infinityworksltd/docker-hub-exporter)
    * [GitHub exporter](https://github.com/infinityworksltd/github-exporter)
@@ -75,7 +76,7 @@ wide variety of JVM-based applications, for example [Kafka](http://kafka.apache.
 ### Logging
    * [Google's mtail log data extractor](https://github.com/google/mtail)
    * [Grok exporter](https://github.com/fstab/grok_exporter)
-   
+
 ### Other monitoring systems
    * [AWS CloudWatch exporter](https://github.com/prometheus/cloudwatch_exporter) (**official**)
    * [Cloud Foundry Firehose exporter](https://github.com/cloudfoundry-community/firehose_exporter)

content/docs/instrumenting/writing_exporters.md

@@ -127,7 +127,7 @@ prefix on to these if they follow the [matching
 semantics](https://docs.google.com/document/d/1Q0MXWdwp1mdXCzNRak6bW5LLVylVRXhdi7_21Sg15xQ/edit).
 E.g. Prometheus has `scrape_duration_seconds` for how long a scrape took, it’s
 good practice to have e.g. `jmx_scrape_duration_seconds` saying how long the
-JMX collector took to do it’s thing. For process stats where you have access to
+JMX collector took to do its thing. For process stats where you have access to
 the pid, both Go and Python offer collectors that’ll handle this for you (see
 the [haproxy exporter](https://github.com/prometheus/haproxy_exporter) for an
 example).
@@ -172,7 +172,7 @@ it’s easier to use them that way.
 The rule of thumb is that one metric should make sense when summed or averaged.
 There is one other case that comes up with exporters, and that’s where the data
 is fundamentally tabular and doing otherwise would require users to do regexes
-on metric names to be useable. Consider the voltage sensors on your
+on metric names to be usable. Consider the voltage sensors on your
 motherboard, while doing math across them is meaningless, it makes sense to
 have them in one metric rather than having one metric per sensor. All values
 within a metrics should (almost) always have the same unit (consider if fan
@@ -452,7 +452,7 @@ A user may have many exporters and Prometheus components on the same machine,
 so to make that easier each has a unique port number.
 
 [https://github.com/prometheus/prometheus/wiki/Default-port-allocations](https://github.com/prometheus/prometheus/wiki/Default-port-allocations)
-is where we track them, this is publically editable.
+is where we track them, this is publicly editable.
 
 Feel free to grab the next free port number when developing your exporter,
 preferably before publicly announcing it. If you’re not ready to release yet,

content/docs/introduction/faq.md

@@ -121,7 +121,7 @@ Currently, the following external systems are supported:
 
 ### Can I create dashboards?
 
-Yes, we recomend [Grafana](/docs/visualization/grafana/) for production usage. [PromDash](/docs/visualization/promdash/) and [Console templates](/docs/visualization/consoles/) also exist.
+Yes, we recommend [Grafana](/docs/visualization/grafana/) for production usage. [PromDash](/docs/visualization/promdash/) and [Console templates](/docs/visualization/consoles/) also exist.
 
 ### Can I change the timezone? Why is everything in UTC?
 

content/docs/introduction/roadmap.md

@@ -36,7 +36,7 @@ GitHub issue: [#398](https://github.com/prometheus/prometheus/issues/398)
 ### Server-side metric metadata support
 
 At this time, metric types and other metadata are only used in the
-client libaries and in the exposition format, but not persisted or
+client libraries and in the exposition format, but not persisted or
 utilized in the Prometheus server. We plan on making use of this
 metadata in the future. For example, we could suggest automatic rates
 over counters, warn users if they take the rate of a gauge, or display

content/docs/operating/configuration.md

@@ -62,9 +62,10 @@ global:
   external_labels:
     [ <labelname>: <labelvalue> ... ]
 
-# Rule files specifies a list of files from which rules and alerts are read.
+# Rule files specifies a list of globs. Rules and alerts are read from
+# all matching files.
 rule_files:
-  [ - <filepath> ... ]
+  [ - <filepath_glob> ... ]
 
 # A list of scrape configurations.
 scrape_configs:
@@ -185,6 +186,10 @@ ec2_sd_configs:
 file_sd_configs:
   [ - <file_sd_config> ... ]
 
+# List of GCE service discovery configurations.
+gce_sd_configs:
+  [ - <gce_sd_config> ... ]
+
 # List of Kubernetes service discovery configurations.
 kubernetes_sd_configs:
   [ - <kubernetes_sd_config> ... ]
@@ -357,6 +362,7 @@ The following meta labels are available on targets during relabeling:
 
 * `__meta_ec2_availability_zone`: the availability zone in which the instance is running
 * `__meta_ec2_instance_id`: the EC2 instance ID
+* `__meta_ec2_instance_state`: the state of the EC2 instance
 * `__meta_ec2_private_ip`: the private IP address of the instance, if present
 * `__meta_ec2_public_dns_name`: the public DNS name of the instance, if available
 * `__meta_ec2_public_ip`: the public IP address of the instance, if available
@@ -432,6 +438,30 @@ may contain a single `*` that matches any character sequence, e.g. `my/path/tg_*
 
 NOTE: Prior to v0.20, `names:` was used instead of `files:`.
 
+### `<gce_sd_config>`
+
+CAUTION: GCE SD is in beta: breaking changes to configuration are still
+likely in future releases.
+
+[GCE](https://cloud.google.com/compute/) SD configurations allow retrieving scrape targets from GCP GCE instances.
+The private IP address is used by default, but may be changed to the public IP
+address with relabeling.
+
+The following meta labels are available on targets during relabeling:
+
+* `__meta_gce_project`: the GCP project in which the instance is running
+* `__meta_gce_zone`: the GCE zone in which the instance is running
+* `__meta_gce_network`: the network of the instance
+* `__meta_gce_subnetwork`: the subnetwork of the instance
+* `__meta_gce_public_ip`: the public IP address of the instance, if present
+* `__meta_gce_private_ip`: the private IP address of the instance
+* `__meta_gce_instance_name`: the name of the instance
+* `__meta_gce_instance_status`: the lifecycle status of the instance
+* `__meta_gce_instance_tags`: comma separated list of instance tags
+* `__meta_gce_instance_metadata_<metadatakey>`: each metadata value of the instance
+
+
+
 ### `<kubernetes_sd_config>`
 
 CAUTION: Kubernetes SD is in beta: breaking changes to configuration are still
@@ -509,8 +539,16 @@ api_servers:
 # The Kubernetes role of entities that should be discovered.
 role: <role>
 
-# Run in cluster. This will use the automounted CA certificate and bearer
-# token file at /var/run/secrets/kubernetes.io/serviceaccount/ in the pod.
+# When true, Prometheus will assume it is being run inside a Kubernetes pod.
+# This will use the CA certificate and authentication token provided by the
+# Kubernetes service account, mounted at
+# /var/run/secrets/kubernetes.io/serviceaccount/ca.crt and
+# /var/run/secrets/kubernetes.io/serviceaccount/token, respectively.
+#
+# Note that this only handles authentication for service discovery. If the
+# target itself requires authentication to be scraped, that must be
+# configured separately via `tls_config`, `bearer_token`, etc. at the
+# `scrape_config` level.
 [ in_cluster: <boolean> ]
 
 # Optional authentication information used to authenticate to the API server.
@@ -538,9 +576,11 @@ tls_config:
 [ retry_interval: <duration> | default = 1s ]
 ```
 
-
 Where `<role>` must be `endpoint`, `service`, `pod`, `container`, `node`, or `apiserver`.
 
+See [this example Prometheus configuration file](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml)
+for a detailed example of configuring Prometheus for Kubernetes.
+
 ### `<marathon_sd_config>`
 
 CAUTION: Marathon SD is in beta: breaking changes to configuration are still
@@ -702,9 +742,10 @@ prefix is guaranteed to never be used by Prometheus itself.
 [ action: <relabel_action> | default = replace ]
 ```
 
-`<regex>` is any valid [RE2 regular
-expression](https://github.com/google/re2/wiki/Syntax). It is required for
-the `replace`, `keep`, `drop` and `labelmap` actions. The regex is fully anchored.
+`<regex>` is any valid
+[RE2 regular expression](https://github.com/google/re2/wiki/Syntax). It is
+required for the `replace`, `keep`, `drop` and `labelmap` actions. The regex is
+anchored on both ends. To un-anchor the regex, use `.*<regex>.*`.
 
 `<relabel_action>` determines the relabeling action to take:
 

content/docs/practices/rules.md

@@ -33,7 +33,7 @@ doing division, separate the metrics using `_per_` and call the operation
 
 When aggregating up ratios, aggregate up the numerator and denominator
 separately and then divide. Do not take the average of a ratio or average of an
-average as that is not statstically valid.
+average as that is not statistically valid.
 
 When aggregating up the `_count` and `_sum` of a Summary and dividing to
 calculate average observation size, treating it as a ratio would be unwieldy.

content/docs/querying/functions.md

@@ -276,6 +276,11 @@ The special cases are equivalent to those in `ln`.
 `log10(v instant-vector)` calculates the decimal logarithm for all elements in `v`.
 The special cases are equivalent to those in `ln`.
 
+## `minute()`
+
+`minute(v=vector(time()) instant-vector)` returns the minute of the hour for each
+of the given times in UTC. Returned values are from 0 to 59.
+
 ## `month()`
 
 `month(v=vector(time()) instant-vector)` returns the month of the year for each

content/docs/querying/operators.md

@@ -248,4 +248,4 @@ highest to lowest.
 
 Operators on the same precedence level are left-associative. For example,
 `2 * 3 % 2` is equivalent to `(2 * 3) % 2`. However `^` is right associative,
-so `2 * 3 ^ 2` is equivilent to `2 * (3 ^ 2)`.
+so `2 * 3 ^ 2` is equivalent to `2 * (3 ^ 2)`.

lib/helpers/download.rb

@@ -100,19 +100,23 @@ module Downloads
       'Binary'
     end
 
-    # TODO(ts): validate
     def os
-      name.split('.')[3].split('-').first
+      base_name.split('.').last.split('-').first
     end
 
-    # TODO(ts): validate
     def arch
-      name.split('.')[3].split('-').last
+      base_name.split('.').last.split('-').last
     end
 
     def size
       @data['size']
     end
+
+    private
+
+    def base_name
+      name.chomp('.tar.gz').chomp('.zip')
+    end
   end
 
   module Helper

specs/download_spec.rb

+require 'rspec'
+require 'helpers/download'
+
+describe Downloads::Asset do
+  let(:asset) do
+    Downloads::Asset.new({
+      'name' => ' prometheus-1.2.0.freebsd-armv5.tar.gz',
+    })
+  end
+
+  let(:beta) do
+    Downloads::Asset.new({
+      'name' => 'alertmanager-0.5.0-beta.0.darwin-amd64.tar.gz',
+    })
+  end
+
+  describe '#os' do
+    it 'extracts the operating system name' do
+      expect(asset.os).to eql('freebsd')
+      expect(beta.os).to eql('darwin')
+    end
+  end
+
+  describe '#arch' do
+    it 'extracts the architecture' do
+      expect(asset.arch).to eql('armv5')
+      expect(beta.arch).to eql('amd64')
+    end
+  end
+end

static/docs.js

@@ -48,7 +48,7 @@ $(document).ready(function() {
 
   selectDownloads();
 
-  $('.download-selection a').on('click', function() {
+  $('.download-selection a').on('click', function(event) {
     event.preventDefault();
 
     $(this).parents('.btn-group').find('button .caption').text($(this).text());