Ingestion
The ingestion configuration ('ingestionConfig') is a section of the table configuration that specifies how to ingest streaming data into Pinot.
ingestionConfig
ingestionConfig
streamConfigMaps
streamConfigMaps
The number of rows per segment is computed using the following formula: realtime.segment.flush.threshold.rows / maxPartitionsConsumedByServer
For example, if you set realtime.segment.flush.threshold.rows = 1000
and each server consumes 10 partitions, the rows per segment is 1000/10 = 100
.
Since release-1.2.0
, we introduced realtime.segment.flush.threshold.segment.rows
, which is directly used as the number of rows per segment.
Take the above example, if you set realtime.segment.flush.threshold.segment.rows = 1000
and each server consumes 10 partitions, the rows per segment is 1000
.
Example table config with ingestionConfig
ingestionConfig