Broker

You can set broker properties in a configuration file. The file can be provided during startup time as follows -

bin/pinot-admin.sh StartBroker -configFileName /path/to/broker.conf

broker.conf can have the following properties. All properties are defined in this class.

Property	Default	Description
pinot.broker.delayShutdownTimeMs	10 seconds
pinot.broker.enableTableLevelMetrics	true
pinot.broker.query.response.limit	Integer.MAX_VALUE	When config pinot.broker.enable.query.limit.overrideis enabled, reset limit for selection query if it exceeds this value.
pinot.broker.query.log.length	Integer.MAX_VALUE
pinot.broker.query.log.maxRatePerSecond	10000.0	Maximum queries to be logged per second. Queries with exceptions, or take longer than 1 second are always logged.
pinot.broker.timeoutMs	10 seconds	Timeout for Broker Query in Milliseconds
pinot.broker.startup.minResourcePercent	100	Configuration to consider the broker ServiceStatus as being STARTED if the percent of resources (tables) that are ONLINE for this this broker has crossed the threshold percentage of the total number of tables that it is expected to serve
pinot.broker.enable.query.limit.override	false	Configuration to enable Query LIMIT Override to protect Pinot Broker and Server from fetch too many records back.
pinot.broker.client.queryPort	8099	(Deprecated: use pinot.broker.client.access.protocols.http.port instead.) Legacy port to query broker via http.
pinot.broker.client.access.protocols		Ingress protocols to query broker (http or https or http,https)
pinot.broker.client.access.protocols.http.port		Port to query broker via http
pinot.broker.client.access.protocols.https.port		Port to query broker via https
pinot.broker.netty.enabled	true	Enable unsecured netty connections to pinot-server
pinot.broker.nettytls.enabled	false	Enable secured netty connections to pinot-server
pinot.broker.tls.keystore.path		Path to broker TLS keystore
pinot.broker.tls.keystore.password		keystore password
pinot.broker.tls.truststore.path		Path to broker TLS truststore
pinot.broker.tls.truststore.password		truststore password
pinot.broker.tls.requires_client_auth	false	toggle for requiring TLS client auth
pinot.broker.http.server.thread.pool.corePoolSize	2 * cores	Config for the thread-pool used by pinot-broker's http-server.
pinot.broker.http.server.thread.pool.maxPoolSize	2 * cores	Config for the thread-pool used by pinot-broker's http-server.
pinot.broker.enable.bounded.jersey.threadpool.executor	false	Enable bounded Jersey thread-pool to handle async requests.
pinot.broker.jersey.threadpool.executor.max.pool.size	2 * cores	Config for the bounded Jersey thread-pool to handle async requests.
pinot.broker.jersey.threadpool.executor.core.pool.size	2 * cores	Config for the bounded Jersey thread-pool to handle async requests.
pinot.broker.jersey.threadpool.executor.queue.size	Integer.MAX_VALUE	Config for the bounded Jersey thread-pool to handle async requests.
pinot.broker.max.query.response.size.bytes	Long.MAX_VALUE	Config indicating the maximum serialized response size across all servers for a query. This value is // equally divided across all servers processing the query.
pinot.broker.max.server.response.size.bytes	Long.MAX_VALUE	Config indicating the maximum length of the serialized response per server for a query.
pinot.broker.event.listener.factory.className	org.apache.pinot.spi.eventlistener.query.NoOpBrokerQueryEventListener	Config indicating the event listener class that will be loaded during broker starter and the onQueryCompletion will be called post query completion. It is on the implementation of this method that user can improve query level observability in their system.
pinot.broker.event.listener.request.context.tracked.header.keys		Comma separated string values indicating the request-headers which will be tracked by the event listener class.
pinot.broker.new.segment.expiration.seconds	300 seconds (5 minutes)	Config for broker to consider a new segment as an old segment and start tagging replica-group / instances as unavailable after this time period.
pinot.broker.query.enable.null.handling	false	Config that allows enabling advanced null handling support for all queries by default (see docs)
pinot.broker.multistage.lite.mode.leaf.stage.limit	100000	For MSE Lite Mode, this controls the maximum number of records that a given Leaf Stage instance on a server is allowed to return. Recommended value is 100k records or lower.
pinot.broker.multistage.use.lite.mode	false	Default value of the query option useLiteMode. Only takes effect when usePhysicalOptimizer=true is also set.
pinot.broker.multistage.run.in.broker	true	Whether to run the non-leaf stages in the broker by default. This controls the default value of the query option runInBroker. Only applicable for MSE Lite Mode.
pinot.broker.default.query.limit	10	Default LIMIT applied to queries that don't specify one explicitly.
pinot.broker.enable.query.cancellation	true	Whether to allow queries to be cancelled via the cancel API.
pinot.broker.groupby.trim.threshold	1000000	Threshold for number of groups to trigger trimming at the broker level. Reducing this can lower memory usage at the cost of accuracy.
pinot.broker.min.group.trim.size	5000	Minimum number of groups kept at the broker level after trimming. The actual number kept is max(5 * LIMIT, minGroupTrimSize).
pinot.broker.adaptive.server.selector.type	NO_OP	Type of adaptive server selector for intelligent query routing. Options: NO_OP (round-robin), NUM_INFLIGHT_REQ (least in-flight requests), LATENCY (lowest latency), HYBRID (combines latency and in-flight).
pinot.broker.adaptive.server.selector.enable.stats.collection	false	Enable stats collection for the adaptive server selector. Must be true for LATENCY or HYBRID types.
pinot.broker.adaptive.server.selector.ewma.alpha	0.666	EWMA (Exponentially Weighted Moving Average) alpha value for latency calculation. Higher values weight recent observations more heavily.
pinot.broker.adaptive.server.selector.hybrid.score.exponent	3	Exponent used in the hybrid scoring formula to balance latency and in-flight request count.
pinot.broker.multistage.use.physical.optimizer	false	Enable the physical optimizer for the multi-stage query engine.
pinot.broker.multistage.infer.partition.hint	false	Infer partition hints to optimize data shuffling in the multi-stage query engine.
pinot.broker.multistage.default.hash.function	absHashCode	Default hash function for data partitioning in the multi-stage engine.
pinot.broker.request.handler.type	netty	Type of request handler for broker-to-server communication. Options: netty, grpc, multistage.
pinot.broker.max.reduce.threads.per.query	min(10, cores/2)	Maximum number of threads used to reduce (merge) results from multiple servers for a single query.
pinot.broker.failure.detector.type	NO_OP	Type of failure detector for detecting unhealthy servers. Options: NO_OP, CONNECTION_BASED.
pinot.broker.use.fixed.replica	false	When true, routes queries to a fixed replica group for better cache locality.