load

Loads data from a data source and returns it as a DataFrame.

Syntax

load(path=None, format=None, schema=None, **options)

Parameters

Parameter	Type	Description
path	str or list, optional	One or more paths in a file-system-backed data source.
format	str, optional	The format of the data source. Defaults to 'parquet'.
schema	StructType or str, optional	The input schema as a StructType object or a DDL-formatted string (for example, 'col0 INT, col1 DOUBLE').
**options	dict	Additional string options.

Returns

DataFrame

Examples

Load a CSV file with format, schema, and options specified.

Python

import tempfile
with tempfile.TemporaryDirectory(prefix="load") as d:
    df = spark.createDataFrame([{"age": 100, "name": "Alice"}])
    df.write.option("header", True).mode("overwrite").format("csv").save(d)

    df = spark.read.load(
        d, schema=df.schema, format="csv", nullValue="Alice", header=True)
    df.printSchema()
    # root
    #  |-- age: long (nullable = true)
    #  |-- name: string (nullable = true)
    df.show()
    # +---+----+
    # |age|name|
    # +---+----+
    # |100|NULL|
    # +---+----+