Streaming data sources and sinks

Structured Streaming has built-in support for a number of streaming data sources and sinks (for example, files and Kafka) and programmatic interfaces that allow you to specify arbitrary data writers. These are explored in the following articles.

• Apache Kafka
• Amazon Kinesis
• Best practices: Delta Lake Structured Streaming applications with Amazon Kinesis
• Load files from S3 using Auto Loader
• Optimized Amazon S3 Source with Amazon SQS
• Azure Event Hubs
• Delta Lake tables
• Read and write streaming Avro data
• Write to arbitrary data sinks