Skip to main content

prometheus (Source)

This source consumes Prometheus metrics that are exported from a specified URL as Stream App events by sending HTTP requests to the URL. Based on the source configuration, it analyzes metrics from the text response and sends them as Stream App events through key-value mapping.The user can retrieve metrics of the including, counter, gauge, histogram, and summary types. The source retrieves the metrics from a text response of the target. Therefore, it is you need to use string as the attribute type for the attributes that correspond with the Prometheus metric labels. Further, the Prometheus metric value is passed through the event as value. This requires you to include an attribute named value in the stream definition. The supported types for the value attribute are INT, LONG, FLOAT, and DOUBLE.

Syntax

CREATE SOURCE <NAME> WITH (type="prometheus", map.type="<STRING>", target.url="<STRING>", scrape.interval="<INT>", scrape.timeout="<INT>", scheme="<STRING>", metric.name="<STRING>", metric.type="<STRING>", username="<STRING>", password="<STRING>", client.truststore.file="<STRING>", client.truststore.password="<STRING>", headers="<STRING>", job="<STRING>", instance="<STRING>", grouping.key="<STRING>")

Query Parameters

NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
target.urlThis property specifies the target URL to which the Prometheus metrics are exported in the TEXT format.STRINGNoNo
scrape.intervalThis property specifies the time interval in seconds within which the source should send an HTTP request to the specified target URL.60INTYesNo
scrape.timeoutThis property is the time duration in seconds for a scrape request to get timed-out if the server at the URL does not respond.10INTYesNo
schemeThis property specifies the scheme of the target URL. The supported schemes are HTTP and HTTPS.HTTPSTRINGYesNo
metric.nameThis property specifies the name of the metrics that are to be fetched. The metric name must match the regex format, i.e., \[a-zA-Z\_:\]\[a-zA-Z0-9\_:\]\* .Stream nameSTRINGYesNo
metric.typeThis property specifies the type of the Prometheus metric that is required to be fetched. The supported metric types are counter, gauge, histogram, and summary.STRINGNoNo
usernameThis property specifies the username that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and the password to enable basic authentication. If you do not provide a value for one or both of these parameters, an error is logged in the console.STRINGYesNo
passwordThis property specifies the password that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and the password to enable basic authentication. If you do not provide a value for one or both of these parameters, an error is logged in the console.STRINGYesNo
client.truststore.fileThe file path to the location of the truststore to which the client needs to send HTTPS requests via the HTTPS protocol.STRINGYesNo
client.truststore.passwordThe password for the client-truststore. This is required to send HTTPS requests. A custom password can be specified if required.STRINGYesNo
headersHeaders that need to be included as HTTP request headers in the request. The format of the supported input is as follows, "header1:value1,header2:value2"STRINGYesNo
jobThis property defines the job name of the exported Prometheus metrics that needs to be fetched.STRINGYesNo
instanceThis property defines the instance of the exported Prometheus metrics that needs to be fetched.STRINGYesNo
grouping.keyThis parameter specifies the grouping key of the required metrics in key-value pairs. The grouping key is used if the metrics are exported by Prometheus pushGateway in order to distinguish those metrics from already existing metrics. The expected format of the grouping key is as follows: "key1:value1,key2:value2"STRINGYesNo

System Parameters

NameDescriptionDefault ValuePossible Parameters
scrapeIntervalThe default time interval in seconds for the Prometheus source to send HTTP requests to the target URL.60Any integer value
scrapeTimeoutThe default time duration (in seconds) for an HTTP request to time-out if the server at the URL does not respond.10Any integer value
schemeThe scheme of the target for the Prometheus source to send HTTP requests. The supported schemes are HTTP and HTTPS.HTTPHTTP or HTTPS
usernameThe username that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and password to enable basic authentication. If you do not specify a value for one or both of these parameters, an error is logged in the console.Any string
passwordThe password that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and password to enable basic authentication. If you do not specify a value for one or both of these parameters, an error is logged in the console.Any string
trustStoreFileThe default file path to the location of truststore that the client needs to access in order to send HTTPS requests through HTTPS protocol.\${carbon.home}/resources/security/client-truststore.jksAny valid path for the truststore file
trustStorePasswordThe default password for the client-truststore that the client needs to access in order to send HTTPS requests through HTTPS protocol.gdncarbonAny string
headersThe headers that need to be included as HTTP request headers in the scrape request. The format of the supported input is as follows, "header1:value1,header2:value2"Any valid http headers
jobThe default job name of the exported Prometheus metrics that needs to be fetched.Any valid job name
instanceThe default instance of the exported Prometheus metrics that needs to be fetched.Any valid instance name
groupingKeyThe default grouping key of the required Prometheus metrics in key-value pairs. The grouping key is used if the metrics are exported by the Prometheus pushGateway in order to distinguish these metrics from already existing metrics. The expected format of the grouping key is as follows: "key1:value1,key2:value2"Any valid grouping key pairs

Example 1

CREATE SOURCE FooStream1 WITH (type= 'prometheus', target.url= 'http://localhost:9080/metrics', metric.type= 'counter', metric.name= 'sweet_production_counter', map.type='keyvalue') (metric_name string, metric_type string, help string, subtype string, name string, quantity string, value double);

In this example, the Prometheus source sends an HTTP request to the target.url and analyzes the response. From the analyzed response, the source retrieves the Prometheus counter metrics with the sweet_production_counter nameand converts the filtered metrics into Stream App events using the key-value mapper. The generated maps have keys and values as follows:

metric_name -> sweet_production_counter   
metric_type -> counter
help -> help_string_of_metric
subtype -> null
name -> value_of_label_name
quantity -> value_of_label_quantity
value -> value_of_metric

Example 2

CREATE SOURCE FooStream2 WITH (type='prometheus', target.url= 'http://localhost:9080/metrics', metric.type='summary', metric.name='sweet_production_summary', map.type='keyvalue') (metric_name string, metric_type string, help string, subtype string, name string, quantity string, quantile string, value double);

In this example, the Prometheus source sends an HTTP request to the target.url and analyzes the response. From the analysed response, the source retrieves the Prometheus summary metrics with the sweet_production_summary nameand converts the filtered metrics into Stream App events using the key-value mapper. The generated maps have keys and values as follows:

metric_name -> sweet_production_summary   
metric_type -> summary
help -> help_string_of_metric
subtype -> `sum`/`count`/`null`
name -> value_of_label_name
quantity -> value_of_label_quantity
quantile -> value of the quantile
value -> value_of_metric

Example 3

CREATE SOURCE FooStream3 WITH (type= 'prometheus', target.url= 'http://localhost:9080/metrics', metric.type= 'histogram', metric.name= 'sweet_production_histogram', map.type='keyvalue') (metric_name string, metric_type string, help string, subtype string, name string, quantity string, le string, value double);

In this example, the prometheus source sends an HTTP request to the target.url and analyzes the response. From the analyzed response, the source retrieves the Prometheus histogram metrics with the sweet_production_histogram name and converts the filtered metrics into Stream App events using the key-value mapper. The generated maps have keys and values as follows:

metric_name -> sweet_production_histogram   
metric_type -> histogram
help -> <help_string_of_metric>
subtype -> <`sum`/`count`/`bucket`>
name -> <value_of_label_name>
quantity -> <value_of_label_quantity>
le -> <value of the bucket>
value -> <value_of_metric>