Monitoring

Tutorial
If you are deploying using the helm chart with Kubernetes, see the tutorial on setting up Prometheus and Grafana to monitor Pinot.
​https://docs.pinot.apache.org/users/tutorials/monitor-pinot-using-prometheus-and-grafana​
Key Metrics to Watch
Please refer to key metrics documented in monitoring pinot.
Customizing Metrics
Pinot uses yammer MetricsRegistry to collect metrics within our application components. These metrics can be published to a metrics server with the help of MetricsRegistryRegistrationListener interface. By default, metrics are published to JMX using the JmxReporterMetricsRegistryRegistrationListener.
You can write a listener to publish metrics to another metrics server by implementing the MetricsRegistryRegistrationListener interface. This listener can be injected into the controller by setting the fully qualified name of the class in the controller configs for the property pinot.controller.metrics.metricsRegistryRegistrationListeners.
You would have to design your own systems to view and monitor these metrics. A list of all the metrics published for each component can be found in:
​ControllerMeter​
​ControllerGauge​
​BrokerMeter​
​BrokerGauge​
​ServerMeter​
​ServerGauge​
​MinionMeter​
​MinionGauge​
JMX to Prometheus
Metrics published to JMX could also be exposed to Prometheus through tooling like jmx_reporter.
To run as a javaagent, download jmx_prometheus_javaagent jar and pinot.yml run:
ALL_JAVA_OPTS="-javaagent:jmx_prometheus_javaagent-0.12.0.jar=8080:pinot.yml -Xms4G -Xmx4G -XX:MaxDirectMemorySize=30g -Dlog4j2.configurationFile=conf/pinot-admin-log4j2.xml -Dplugins.dir=$BASEDIR/plugins"
bin/pinot-admin.sh ....
This will expose a port at 8080 to dump metrics as Prometheus format for Prometheus scrapper to fetch.

Access Control

Tuning

Last modified 1yr ago