To consume in realtime, we simply need to create a table with the same name as the schema and point to the Kafka topic to consume from, using a table definition such as this one:
First, we’ll start a local instance of Kafka and start streaming data into it:Untitled
This will stream one event per second from the Avro file to the Kafka topic. Then, we’ll create a realtime table, which will start consuming from the Kafka topic.
We can then query the table with the following query to see the events stream in:
Repeating the query multiple times should show the events slowly being streamed into the table.