Server

Server configuration can be provided as part of the server startup parameters.

bin/pinot-admin.sh StartServer -configFileName /path/to/server.conf

Duplicate Keys in Configuration File

Starting from Apache Pinot 1.3.0, duplicate keys in the configuration file will cause a ConfigurationException to be thrown during startup. Previously, duplicate keys would be silently merged into a list. If you encounter this error, ensure that each configuration property appears only once in your configuration file. The exception will include the exact file path, duplicate key name, and the line numbers where the duplicates were found.

Example error:

ConfigurationException: Duplicate key found in /path/to/server.conf at line 10 and line 15: pinot.server.netty.port

server.conf can have the following properties

Property	Default	Description
pinot.server.netty.port	8098	Port to query Pinot Server
pinot.server.netty.host		Pinot server hostname
pinot.server.adminapi.port	8097	Port for Pinot Server Admin UI
pinot.server.grpc.enable	true	Enable the server-side gRPC query service. This listener is used for gRPC query transport and is separate from the Netty and Admin API listeners.
pinot.server.grpc.port	8090	Port for the server-side gRPC query service.
pinot.server.grpctls.enabled	false	Enable TLS for the server-side gRPC query service. When enabled, configure the TLS material under pinot.server.grpctls.*.
pinot.server.instance.id		By default the server instance id used by Helix is Server_hostname_port where the hostname and port are configured through host and port config values above. This config overwrites the default setting. User can put server id independent of the server's hostname and port.
pinot.server.instance.dataDir	java.io.tmpdir + /PinotServer/index	Directory to hold all the data
pinot.server.instance.consumerDir
pinot.server.instance.segmentTarDir	java.io.tmpdir + /PinotServer/segmentTar	Directory to hold temporary segments downloaded from Controller or Deep Store
pinot.tar.compression.codec.name	gzip	Compression codec used for segment tar archives during upload and download. Supported values: gzip (default), lz4_framed (fast, lower compression ratio), zstandard (good balance of speed and ratio). Introduced in 1.3.0. File extensions: .tar.gz, .tar.lz4, .tar.zst respectively.
pinot.server.instance.readMode	mmap
pinot.server.instance.reload.consumingSegment	true	Specifies if the reload segment API should reload the consuming segments. This is useful when the corresponding schema is updated and we want the changes to be reflected in the consuming segment.
pinot.server.instance.data.manager.class	org.apache.pinot.server. starter.helix.HelixInstanceDataManager
pinot.query.scheduler.name	fcfs	Currently only FCFS (first-come-first-serve) is supported
pinot.query.scheduler.query_runner_threads	CPU cores	Main thread to execute the queries (one thread per query). Can be dynamically updated at runtime via Helix cluster config without requiring a server restart.
pinot.query.scheduler.query_worker_threads	2 * CPU cores	Worker thread to process the segments (multiple threads per query). Can be dynamically updated at runtime via Helix cluster config without requiring a server restart.
queryWorkerThreads	max(2, min(8, 2 * CPU cores))	Top-level server.conf key for the worker pool used by the server-side gRPC query service. This is distinct from pinot.query.scheduler.query_worker_threads, which controls the server query scheduler and Lucene search pool sizing.
maxInboundMessageSizeBytes	134217728 (128 MB)	Top-level server.conf key for the maximum inbound gRPC message size accepted by the server-side gRPC query service.
requestThrottlingMemoryThresholdBytes	17179869184 (16 GiB)	Top-level server.conf key for the direct-memory threshold that triggers request throttling in the server-side gRPC query service.
pinot.query.scheduler.query.log.maxRatePerSecond	unlimited	(Deprecated: use pinot.server.query.log.maxRatePerSecond instead.) Maximum queries to be logged per second. Queries with exceptions and costly queries are always logged.
pinot.server.query.executor.class	org.apache.pinot.core.query. executor.ServerQueryExecutorV1Impl
pinot.server.query.executor.pruner.class	ValidSegmentPruner,DataSchemaSegmentPruner, ColumnValueSegmentPruner,SelectionQuerySegmentPruner
pinot.server.query.executor.timeout	15000	Timeout for Server to process Query in Milliseconds
pinot.server.query.executor.max.execution.threads	-1 (unlimited)	Maximum number of execution threads allowed for a query. Limiting this can prevent a single expensive query from occupying all the execution threads.
pinot.server.query.executor.default.execution.threads	-1 (unset; fall back to max.execution.threads)	Default execution-thread count assigned to queries that do not set SET maxExecutionThreads. When this value is positive and pinot.server.query.executor.max.execution.threads is also positive, the default must be less than or equal to the max. Per-query maxExecutionThreads can still override it, subject to the configured server max.
pinot.server.query.executor.max.init.group.holder.capacity	10000	Initial capacity of the group key holder. Increasing this value can reduce the resizing of the group key holder, but increase the heap usage for small group-by queries.
pinot.server.query.executor.min.init.indexed.table.capacity	128	Minimum initial capacity of the server-side IndexedTable Pinot uses while merging group-by results. Increasing this value can reduce rehashing for high-cardinality group-by queries, but increases memory usage for smaller queries. This value can be overridden per query with SET minInitialIndexedTableCapacity = value.
pinot.server.query.executor.num.groups.limit	100000	Maximum number of groups kept from each segment during query execution. Once this limit is reached, no more groups will be taken (will still aggregate on existing groups).
pinot.server.query.executor.num.groups.warn.limit	150000	Warning threshold for the number of groups accumulated during query execution. When reached, Pinot logs a warning and sets numGroupsWarningLimitReached=true in the response metadata, but does not stop the query.
pinot.server.query.executor.min.segment.group.trim.size	-1 (do not trim)	Minimum number of groups kept at segment level during query execution. If there are enough groups found in a segment, pinot will trim the groups to max(5 * LIMIT, minSegmentgroupTrimSize) groups based on the order-by clause. Increasing this value can increase the accuracy of the results, but also increase the heap usage of the group-by queries.
pinot.server.query.executor.min.server.group.trim.size	5000	Minimum number of groups kept at server level during query execution. If there are enough groups found in a server, pinot will trim the groups to max(5 * LIMIT, minServergroupTrimSize) groups based on the order-by clause. Increasing this value can increase the accuracy of the results, but also increase the heap usage and data transfer cost of the group-by queries.
pinot.server.query.executor.groupby.trim.threshold	1000000	Threshold for number of groups to trigger the server group trimming. Increasing this value can reduce the times of trimming, but also increase the heap usage of the group-by queries.
pinot.server.requestHandlerFactory.class	org.apache.pinot.server. request.SimpleRequestHandlerFactory
pinot.server.instance.segment.format.version
pinot.server.instance.enable.split.commit		(Deprecated) Enable split commit protocol for real-time segment commit.
pinot.server.instance.enable.commitend.metadata		(Deprecated) Enable metadata commit at the end of segment commit.
pinot.server.instance.segment.upload.to.deep.store	false	Default for whether real-time segment commits upload completed segments directly to deep store instead of sending them through the controller. This applies when the table stream config does not set realtime.segment.serverUploadToDeepStore.
pinot.server.instance.offset.lag.tracking.enable	true	Enable tracking of the REALTIME_INGESTION_OFFSET_LAG server gauge for real-time ingestion. Disable this if fetching upstream offsets is too expensive for your stream setup.
pinot.server.instance.offset.lag.tracking.update.interval	60000	Interval in milliseconds between upstream offset refreshes used for REALTIME_INGESTION_OFFSET_LAG. Must be greater than 1000. Increasing this reduces load on the stream metadata provider but makes the lag gauge less fresh.
pinot.server.instance.max.parallel.refresh.threads	1	Number of simultaneous segments that can be refreshed on one server.
pinot.server.instance.realtime.max.parallel.segment.builds	0	Specifies how many parallel real-time segments can be built. Value of <= 0 indicates unlimited.
pinot.server.instance.realtime.alloc.offheap	true	Boolean value to control whether memory for real-time consuming segments should be allocated off-heap.
pinot.server.instance.realtime.alloc.offheap.direct	false	If 'real-time.alloc.offheap' is set to true, this boolean value controls whether the corresponding allocation should be direct or not (false indicate mmap allocation)
pinot.server.startup.minResourcePercent	100	The percentage of tables that need to be in an ONLINE state before the server is marked as STARTED to server queries
pinot.server.startup.timeoutMs	10 minutes	The total amount of time a server will wait for all status checks before server is marked as STARTED to server queries
pinot.server.starter.realtimeConsumptionCatchupWaitMs	0	On it's own, this is a static amount of time servers will wait for consuming segments before server is marked as STARTED to server queries When paired with pinot.server.starter.enableRealtimeOffsetBasedConsumptionStatusChecker or pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker, this is how long those status checkers will wait for segments to turn GOOD This should not be set to more than pinot.server.startup.timeoutMs as it will not be respected
pinot.server.starter.enableRealtimeOffsetBasedConsumptionStatusChecker	false	When true, the server will collect the current, latest offsets for all consuming segments and will mark those segments as GOOD once they have caught up to those offsets
pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker	false	When true, the server will not be marked as STARTED or serve queries until 1) the server has caught up to the latest available offset or the latest consumed event for each segment is at least within pinot.server.starter.realtimeMinFreshnessMs of the current time 2) pinot.server.startup.timeoutMs has elapsed
pinot.server.starter.realtimeMinFreshnessMs	10 seconds	When pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker=true, this configures the minimum freshness (now - last_event_time) for the server status checker to consider a consuming segment GOOD This only applies when pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker=true
pinot.server.starter.realtimeFreshnessIdleTimeoutMs	0	When pinot.server.starter.enableRealtimeFreshnessBasedConsumptionStatusChecker=true, this will mark a segment as GOOD after this timeout if it has not consumed any events.
pinot.server.startup.exitOnServiceStatusCheckFailure	false	When true, the server will shutdown after pinot.server.shutdown.timeoutMs rather than be marked as STARTED if the status has not turned GOOD
pinot.server.startup.enableServiceStatusCheck	true	true - server will run all configured checks false - server will be marked as STARTED without performing an startup checks
pinot.server.startup.serviceStatusCheckIntervalMs	10 seconds	Interval at which server will perform service stats checks
pinot.server.shutdown.timeoutMs	10 minutes
pinot.server.shutdown.enableQueryCheck	true
pinot.server.shutdown.noQueryThresholdMs	15 seconds
pinot.server.shutdown.enableResourceCheck	false
pinot.server.shutdown.resourceCheckIntervalMs	10 seconds
pinot.server.admin.access.control.factory.class	org.apache.pinot.server. api.access.AllowAllAccessFactory
pinot.server.adminapi.access.protocols	http	Ingress protocols to access server admin api (http or https or http,https)
pinot.server.adminapi.access.protocols.http.port		Port to access server admin api via http
pinot.server.adminapi.broker.protocols.https.port		Port to access server admin api via https
pinot.server.tls.keystore.path		Path to server TLS keystore
pinot.server.tls.keystore.password		keystore password
pinot.server.tls.truststore.path		Path to server TLS truststore
pinot.server.tls.truststore.password		truststore password
inot.server.tls.client.auth	false	toggle for requiring TLS client auth
pinot.server.netty.enabled	true	toggle for enabling unsecured netty connections to server
pinot.server.netty.port		Port for accessing pinot server via unsecured netty
pinot.server.nettytls.enabled
pinot.server.nettytls.port		Port for accessing pinot server via TLS-secured netty
pinot.server.http.server.thread.pool.corePoolSize	2 * cores	Config for the thread-pool used by pinot-server's http-server.
pinot.server.http.server.thread.pool.maxPoolSize	2 * cores	Config for the thread-pool used by pinot-server's http-server.
pinot.server.http.server.max.http.header.size	unset (Grizzly default)	Maximum total HTTP header size in bytes accepted by the server admin API's Grizzly HTTP/HTTPS listeners. This does not affect the Netty data plane or gRPC query transport.
pinot.server.http.server.max.request.headers	unset (Grizzly default)	Maximum number of HTTP headers accepted by the server admin API's Grizzly HTTP/HTTPS listeners.
pinot.server.segment.fetcher.http.client.maxConnTotal		Config for the http-client used by HttpSegmentFetcher for downloading segments
pinot.server.segment.fetcher.http.client.maxConnPerRoute		Config for the http-client used by HttpSegmentFetcher for downloading segments
pinot.server.segment.fetcher.http.client.connectionTimeoutMs		Config for the segment fetcher to wait to establish a connection
pinot.server.segment.fetcher.http.request.socketTimeoutMs		Config for the segment fetcher to wait for the segment to download
pinot.server.segment.fetcher.http.request.connectionRequestTimeoutMs		Config for the segment fetcher to wait for a connection from the pool
pinot.server.instance.max.segment.preload.threads	0	Number of threads that should be created to preload the segments in an upsert table. Value should be greater than 0 otherwise preload is synchronous.
pinot.server.instance.dedup.default.ignore.non.default.tiers	false	Default behavior for dedup tables whose dedupConfig.ignoreNonDefaultTiers is DEFAULT. When set to true, Pinot skips immutable segments on non-default tiers when constructing dedup metadata.
pinot.server.instance.dedup.default.allow.dedup.consumption.during.commit	false	Deprecated dedup fallback for dedupConfig.allowDedupConsumptionDuringCommit. When set to true, dedup tables whose table-level flag is still false keep consuming while a previous segment is being finalized, including during segment download and replacement.
pinot.server.instance.upsert.default.allow.partial.upsert.consumption.during.commit	false	Deprecated upsert fallback for upsertConfig.allowPartialUpsertConsumptionDuringCommit. When set to true, partial upsert tables whose table-level flag is still false keep consuming while the previous segment is being finalized, including during segment download and replacement. New deployments should prefer parallelSegmentConsumptionPolicy in streamIngestionConfig.
pinot.server.consuming.segment.consistency.mode	RESTRICTED	Controls how the server handles segment reloads and force commits for partial upsert tables and tables with dropOutOfOrder=true. RESTRICTED: disables reloads and force commits (safest, default). PROTECTED: enables reloads/force commits with metadata reversion; requires ParallelSegmentConsumptionPolicy set to DISALLOW_ALWAYS or ALLOW_DURING_BUILD_ONLY. UNSAFE: allows reloads without metadata reversion.
pinot.server.query.regex.class	JAVA_UTIL	Determine the regex class used for query execution. Valid options are JAVA_UTIL and RE2J
pinot.server.instance.consumer.client.id.suffix		Configures a suffix to differentiate consumer instances. When multiple replicas of a real-time consuming segment have the same client ID (formatted as TableName-TopicName-Partition), this suffix can be used add a unique server level identity.
pinot.server.segment.uploader.protocol	http	Configure the http protocol when server upload segments to controller. This is required
pinot.server.segment.uploader.https.enabled	false	Enable https for server upload protocol, note that you need to configure pinot.server.segment.uploader.protocol=https
pinot.server.segment.uploader.https.ssl.xxxx		All the ssl related configures goes here
pinot.server.segment.uploader.upload.request.timeout.ms	300_000	default is 300 seconds/5mins
pinot.server.segment.uploader.auth.xxx		Auth related configs go here
pinot.server.segment.uploader.http.client.maxConnTotal		uploader http client config
pinot.server.segment.uploader.http.client.maxConnPerRoute		uploader http client config
pinot.server.segment.uploader.http.client.disableDefaultUserAgent		uploader http client config
pinot.server.lucene.min.refresh.interval.ms	10	Wait time before each round of refreshing Lucene indexes config
pinot.server.lucene.max.refresh.threads	1	Number of threads that concurrently refresh Lucene indexes config
pinot.server.shutdown.timeoutMs	600000 (10 minutes)	Timeout in milliseconds for the shutdown checks to complete before the server shuts down regardless.
pinot.server.shutdown.enableQueryCheck	true	When true, the server will wait for in-flight queries to complete before shutting down.
pinot.server.shutdown.noQueryThresholdMs	15000	The server waits until no queries are received for this duration before proceeding with shutdown.
pinot.server.shutdown.enableResourceCheck	false	When true, the server will check that external visibility (e.g., Helix routing tables) has been updated before shutting down.
pinot.server.shutdown.resourceCheckIntervalMs	10000	Interval in milliseconds between resource checks during shutdown.
pinot.server.query.executor.mse.max.execution.threads	-1	Fixed hard limit for concurrent multi-stage executor tasks on this server. When set to a positive value, this overrides the cluster-derived hard limit from pinot.beta.multistage.engine.max.server.query.threads and pinot.beta.multistage.engine.max.server.query.threads.hardlimit.factor. When left non-positive, Pinot uses the cluster-derived hard limit if both cluster values are positive; otherwise hard limiting is disabled.
pinot.server.query.executor.mse.max.execution.threads.exceed.strategy	ERROR	Behavior when the server-side multi-stage executor hard limit is exceeded. Supported values: ERROR (reject immediately) and LOG (allow execution but log a warning).
pinot.query.mse.stats.mode	ALWAYS	Controls whether multi-stage query workers include stage stats in the response. ALWAYS sends stats from every worker. SAFE sends stats only when all brokers and servers report the same Pinot version; use this override during upgrades that still include servers older than Pinot 1.4. NEVER disables stage stats entirely.
pinot.query.multistage.query.server.permit.keep.alive.time.ms	300000	For the MSE QueryServer gRPC server, the minimum interval in milliseconds between broker keep-alive pings that the server accepts. If you reduce pinot.query.multistage.dispatch.channel.keep.alive.time.ms below the default, set this to a value less than or equal to the broker interval. A non-positive value leaves Netty's built-in server default in place.
pinot.query.multistage.query.server.permit.keep.alive.without.calls	false	For the MSE QueryServer gRPC server, whether broker keep-alive pings are permitted while there are no active RPCs. Set this to true if brokers use pinot.query.multistage.dispatch.channel.keep.alive.without.calls=true; otherwise QueryServer will close idle channels with GOAWAY(ENHANCE_YOUR_CALM).
pinot.server.max.segment.preprocess.parallelism	Integer.MAX_VALUE	Maximum number of segments that can be preprocessed (index building on load) in parallel. Reduce this to limit CPU or memory usage during segment loading. Set 0 to block new preprocess work until the value is raised again.
pinot.server.max.segment.download.parallelism	Integer.MAX_VALUE	Maximum number of segments that can be downloaded in parallel from deep store or peers. Set 0 to block new downloads until the value is raised again.
pinot.server.peer.download.enabled	false	When true, PredownloadScheduler retries failed immutable-segment predownloads from peer servers instead of stopping at the deep-store failure. Configure pinot.server.instance.peer.download.scheme on the same server so peers can advertise usable download URLs.
pinot.server.instance.peer.download.scheme		Protocol Pinot should use when downloading segments from peer servers. Valid values are http and https. This is required for PredownloadScheduler peer fallback and other peer-download paths that fetch segments from another server.
pinot.server.predownload.parallelism	numProcessors * 3 when unset or non-positive	Number of threads used by PredownloadScheduler when a replacement server predownloads immutable segments before taking traffic. Set a positive value to override the default.
pinot.server.max.segment.startree.preprocess.parallelism	Integer.MAX_VALUE	Maximum number of star-tree indexes that can be built in parallel during segment preprocessing. Set 0 to block new star-tree preprocess work until the value is raised again.
pinot.server.lucene.max.refresh.threads	1	Maximum number of threads used to refresh Lucene indexes (for text and FST indexes on real-time tables).
pinot.server.lucene.min.refresh.interval.ms	10	Minimum interval in milliseconds between Lucene index refreshes for real-time tables.
pinot.server.consumption.rate.limit	0.0 (no limit)	Maximum number of rows per second that can be consumed across all real-time tables on this server. 0 means no limit.
pinot.server.instance.enableThreadCpuTimeMeasurement	false	Enable per-thread CPU time measurement for query execution. Provides more detailed metrics but adds overhead.
pinot.server.instance.enableThreadAllocatedBytesMeasurement	false	Enable per-thread memory allocation measurement. Provides detailed memory allocation metrics per query but adds overhead.
pinot.server.query.executor.enableThrottlingOnHeapUsage	false	Enable throttling when heap usage is critical. When enabled, Pinot queues query-executor tasks instead of rejecting them immediately under memory pressure.
pinot.server.query.executor.heap.usage.throttle.queue.maxSize	1000	Maximum number of query-executor tasks that can wait in the heap-usage throttle queue. Values less than or equal to 0 make the queue unbounded.
pinot.server.query.executor.heap.usage.throttle.queue.timeoutMs	30000	Default maximum time in milliseconds that a queued query-executor task waits for heap pressure to clear before timing out. Pinot also caps this by the query's remaining deadline when one is present.
pinot.server.query.executor.heap.usage.throttle.monitorIntervalMs	1000	Interval in milliseconds between heap-usage checks that decide when queued query-executor tasks can resume.