csv (DataStreamReader)

Loads a CSV file stream and returns the result as a DataFrame. If inferSchema is enabled, the function goes through the input once to determine the schema. To avoid this pass, disable inferSchema or specify the schema explicitly using schema.

Syntax

csv(path, schema=None, **options)

Parameters

Parameter	Type	Description
path	str	Path for the CSV input.
schema	StructType or str, optional	Schema as a StructType or DDL-formatted string (for example, col0 INT, col1 DOUBLE).

Returns

DataFrame

Examples

Load a stream from a temporary CSV file:

Python

import tempfile
import time
with tempfile.TemporaryDirectory(prefix="csv") as d:
    spark.createDataFrame([(1, "2"),]).write.mode("overwrite").format("csv").save(d)
    q = spark.readStream.schema(
        "col0 INT, col1 STRING"
    ).format("csv").load(d).writeStream.format("console").start()
    time.sleep(3)
    q.stop()