Streaming Data Sources and Sinks

Structured Streaming has built-in support for a number of streaming data sources and sinks (for example, files and Kafka) and programmatic interfaces that allow you to specify arbitrary data writers. These are explored in the articles below.

• Apache Kafka
• Amazon Kinesis
• Optimized S3 File Source with SQS
• Azure Event Hubs
• Delta Lake Tables
• Read and Write Streaming Avro Data with DataFrames
• Write to Arbitrary Data Sinks