groupfrom the Meetup events Quickstart example.
groupis a nested array with elements of object type.
groupwith complex type to JSON, you can add the following config to table config.
transformConfigstransforms the object
groupto a JSON string
group_json, which then creates the JSON indexing with config
jsonIndexColumns. To read the full spec, see json_meetupRsvp_realtime_table_config.json.
groupis a reserved keyword in SQL and therefore needs to be quoted in
maxLengthof the field
group_jsonon the schema, because by default, a string column has a limited length. For example,
JSON_EXTRACT_SCALARis needed to extract the values in the GROUP BY and ORDER BY clauses, which invokes the function evaluation.
ingestionConfig. For example:
complexTypeConfig, all the map objects will be flattened to direct fields automatically. And with
unnestFields, a record with the nested collection will unnest into multiple records. For instance, the example at the beginning will transform into two rows with this configuration example.
groupis flattened to
group.group_id. The default value of the delimiter is
.You can choose another delimiter by specifying the configuration
complexTypeConfig. This flattening rule also applies to maps in the collections to be unnested.
groupis unnested into the top-level, and converts the output to a collection of two rows. Note the handling of the nested field within
group_topics, and the eventual top-level field of
group.group_topics.urlkey. All the collections to unnest shall be included in the configuration
fieldsToUnnestwill be serialized into JSON string, except for the array of primitive values, which will be ingested as a multi-value column by default. The behavior is defined by the
collectionNotUnnestedToJsonconfig, which takes the following values:
NON_PRIMITIVE- Converts the array to a multi-value column. (default)
ALL- Converts the array of primitive values to JSON string.
NONE- Does not do any conversion.
fieldsToUnnestsimilar to the ones in
complexTypeConfig. And this will simulate the complex-type handling rules on the Avro schema and output the Pinot schema in the file specified in