Performance Optimization Configurations
Enabling Server Side Segment Stream Download-Untar with rate limiter:
Pinot server supports segment download-untar with rate limiter, which reduces the bytes written to disk and optimizes disk bandwidth usage https://github.com/apache/pinot/pull/8753. This feature involves the following server level configs
pinot.server.instance.segment.stream.download.untar : true
// enable stream download and untar
pinot.server.instance.segment.stream.download.untar.rate.limit.bytes.per.sec : 100000000
// the max download-untar rate is limited to 100000000 bytes/sec
Another useful config is:
pinot.server.instance.table.level.max.parallel.segment.downloads : 4
This helps to limit the number of max parallel number of segment download per table, at server level
Enabling Netty Native TLS:
Apache Pinot supports using native TLS library https://netty.io/wiki/forked-tomcat-native.html for broker-server transmission, which would potentially boost TLS encryption/decryption performance. This can be enabled by adding the following broker/server config
pinot.broker.tls.ssl.provider : OPENSSL
pinot.server.tls.ssl.provider : OPENSSL
Enabling Netty Native Transport:
Apache Pinot supports using native transport library https://netty.io/wiki/native-transports.html for broker-server transmission, which would potentially boost performance under high QPS. This can be enabled using:
pinot.broker.netty.native.transports.enabled : true
pinot.server.netty.native.transports.enabled : true
Last updated
Was this helpful?