Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
D
docs
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Administrator
docs
Commits
c1049a73
Commit
c1049a73
authored
Jan 16, 2019
by
Julius Volz
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Update recording rules best practices to 2.x format
Signed-off-by:
Julius Volz
<
julius.volz@gmail.com
>
parent
0c3f7457
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
57 additions
and
42 deletions
+57
-42
rules.md
content/docs/practices/rules.md
+57
-42
No files found.
content/docs/practices/rules.md
View file @
c1049a73
...
...
@@ -50,66 +50,81 @@ conflicts and give you more useful metrics and alerts.
Aggregating up requests per second that has a
`path`
label:
```
instance_path:requests:rate5m =
rate(requests_total{job="myjob"}[5m])
- record: instance_path:requests:rate5m
expr:
rate(requests_total{job="myjob"}[5m])
path:requests:rate5m =
sum without (instance)(instance_path:requests:rate5m{job="myjob"})
- record: path:requests:rate5m
expr:
sum without (instance)(instance_path:requests:rate5m{job="myjob"})
```
Calculating a request failure ratio and aggregating up to the job-level failure ratio:
```
instance_path:request_failures:rate5m =
rate(request_failures_total{job="myjob"}[5m])
instance_path:request_failures_per_requests:ratio_rate5m =
instance_path:request_failures:rate5m{job="myjob"}
/
instance_path:requests:rate5m{job="myjob"}
// Aggregate up numerator and denominator, then divide to get path-level ratio.
path:request_failures_per_requests:ratio_rate5m =
sum without (instance)(instance_path:request_failures:rate5m{job="myjob"})
/
sum without (instance)(instance_path:requests:rate5m{job="myjob"})
// No labels left from instrumentation or distinguishing instances,
// so we use 'job' as the level.
job:request_failures_per_requests:ratio_rate5m =
sum without (instance, path)(instance_path:request_failures:rate5m{job="myjob"})
/
sum without (instance, path)(instance_path:requests:rate5m{job="myjob"})
- record: instance_path:request_failures:rate5m
expr: rate(request_failures_total{job="myjob"}[5m])
- record: instance_path:request_failures_per_requests:ratio_rate5m
expr: |
(
instance_path:request_failures:rate5m{job="myjob"}
/
instance_path:requests:rate5m{job="myjob"}
)
# Aggregate up numerator and denominator, then divide to get path-level ratio.
- record: path:request_failures_per_requests:ratio_rate5m
expr: |
(
sum without (instance)(instance_path:request_failures:rate5m{job="myjob"})
/
sum without (instance)(instance_path:requests:rate5m{job="myjob"})
)
# No labels left from instrumentation or distinguishing instances,
# so we use 'job' as the level.
- record: job:request_failures_per_requests:ratio_rate5m
expr: |
(
sum without (instance, path)(instance_path:request_failures:rate5m{job="myjob"})
/
sum without (instance, path)(instance_path:requests:rate5m{job="myjob"})
)
```
Calculating average latency over a time period from a Summary:
```
instance_path:request_latency_seconds_count:rate5m =
rate(request_latency_seconds_count{job="myjob"}[5m])
instance_path:request_latency_seconds_sum:rate5m =
rate(request_latency_seconds_sum{job="myjob"}[5m])
instance_path:request_latency_seconds:mean5m =
instance_path:request_latency_seconds_sum:rate5m{job="myjob"}
/
instance_path:request_latency_seconds_count:rate5m{job="myjob"}
// Aggregate up numerator and denominator, then divide.
path:request_latency_seconds:mean5m =
sum without (instance)(instance_path:request_latency_seconds_sum:rate5m{job="myjob"})
/
sum without (instance)(instance_path:request_latency_seconds_count:rate5m{job="myjob"})
- record: instance_path:request_latency_seconds_count:rate5m
expr: rate(request_latency_seconds_count{job="myjob"}[5m])
- record: instance_path:request_latency_seconds_sum:rate5m
expr: rate(request_latency_seconds_sum{job="myjob"}[5m])
- record: instance_path:request_latency_seconds:mean5m
expr: |
(
instance_path:request_latency_seconds_sum:rate5m{job="myjob"}
/
instance_path:request_latency_seconds_count:rate5m{job="myjob"}
)
# Aggregate up numerator and denominator, then divide.
- record: path:request_latency_seconds:mean5m
expr: |
(
sum without (instance)(instance_path:request_latency_seconds_sum:rate5m{job="myjob"})
/
sum without (instance)(instance_path:request_latency_seconds_count:rate5m{job="myjob"})
)
```
Calculating the average query rate across instances and paths is done using the
`avg()`
function:
```
job:request_latency_seconds_count:avg_rate5m =
avg without (instance, path)(instance:request_latency_seconds_count:rate5m{job="myjob"})
- record: job:request_latency_seconds_count:avg_rate5m
expr:
avg without (instance, path)(instance:request_latency_seconds_count:rate5m{job="myjob"})
```
Notice that when aggregating that the labels in the
`without`
clause are removed
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment