Apache Pinot Docs
latest
Ask or search…
K
Comment on page

Performance Optimization Configurations

Query Option to Enable `AND` Predicate Reordering

An optional optimization for queries with long AND predicate will be to let execution engine reorder the predicates based on cardinality, so as to do minimum scanning for un-indexed operators. To use it, simply add the following option to the original query
SET AndScanReordering = 'True'; SELECT FOO from BAR WHERE predicate1 AND predicate2 AND predicate3 AND predicate4 ...
This feature cannot guarantee optimization for all use cases, but on average it can help. Try with some before/after comparison.

Enabling Server Side Segment Stream Download-Untar with rate limiter:

Pinot server supports segment download-untar with rate limiter, which reduces the bytes written to disk and optimizes disk bandwidth usage https://github.com/apache/pinot/pull/8753. This feature involves the following server level configs
pinot.server.instance.segment.stream.download.untar : true
// enable stream download and untar
pinot.server.instance.segment.stream.download.untar.rate.limit.bytes.per.sec : 100000000
// the max download-untar rate is limited to 100000000 bytes/sec
Another useful config is:
pinot.server.instance.table.level.max.parallel.segment.downloads : 4
This helps to limit the number of max parallel number of segment download per table, at server level

Enabling Netty Native TLS:

Apache Pinot supports using native TLS library https://netty.io/wiki/forked-tomcat-native.html for broker-server transmission, which would potentially boost TLS encryption/decryption performance. This can be enabled by adding the following broker/server config
pinot.broker.tls.ssl.provider : OPENSSL
pinot.server.tls.ssl.provider : OPENSSL

Enabling Netty Native Transport:

Apache Pinot supports using native transport library https://netty.io/wiki/native-transports.html for broker-server transmission, which would potentially boost performance under high QPS. This can be enabled using:
pinot.broker.netty.native.transports.enabled : true
pinot.server.netty.native.transports.enabled : true