BIG_DECIMAL
type is added after release 0.10.0
.group
from the Meetup events Quickstart example.group
is a nested array with elements of object type.group
with complex type to JSON, you can add the following config to table config.transformConfigs
transforms the object group
to a JSON string group_json
, which then creates the JSON indexing with config jsonIndexColumns
. To read the full spec, see json_meetupRsvp_realtime_table_config.json.group
is a reserved keyword in SQL and therefore needs to be quoted in transformFunction
.columnName
can't use the same name as any of the fields in the source JSON data e.g. if our source data contains the field group
and we want to transform the data in that field before persisting it, the destination column name would need to be something different, like group_json
.maxLength
of the field group_json
on the schema, because by default, a string column has a limited length. For example,group
. For the details about the supported JSON function, see guide).JSON_EXTRACT_SCALAR
is needed to extract the values in the GROUP BY and ORDER BY clauses, which invokes the function evaluation.complexTypeConfig
to the ingestionConfig
. For example:complexTypeConfig
, all the map objects will be flattened to direct fields automatically. And with unnestFields
, a record with the nested collection will unnest into multiple records. For instance, the example at the beginning will transform into two rows with this configuration example.group_id
under group
is flattened to group.group_id
. The default value of the delimiter is .
You can choose another delimiter by specifying the configuration delimiter
under complexTypeConfig
. This flattening rule also applies to maps in the collections to be unnested.group_topics
under group
is unnested into the top-level, and converts the output to a collection of two rows. Note the handling of the nested field within group_topics
, and the eventual top-level field of group.group_topics.urlkey
. All the collections to unnest shall be included in the configuration fieldsToUnnest
.fieldsToUnnest
will be serialized into JSON string, except for the array of primitive values, which will be ingested as a multi-value column by default. The behavior is defined by the collectionNotUnnestedToJson
config, which takes the following values:NON_PRIMITIVE
- Converts the array to a multi-value column. (default)ALL
- Converts the array of primitive values to JSON string.NONE
- Does not do any conversion..
is a reserved character in SQL, so you need to quote the flattened columns in the query.fieldsToUnnest
similar to the ones in complexTypeConfig
. And this will simulate the complex-type handling rules on the Avro schema and output the Pinot schema in the file specified in outputDir
.