schema_of_json

Parses a JSON string and infers its schema in DDL format.

Syntax

Python

from pyspark.sql import functions as sf

sf.schema_of_json(json, options=None)

Parameters

Parameter	Type	Description
json	pyspark.sql.Column or str	A JSON string or a foldable string column containing a JSON string.
options	dict, optional	Options to control parsing. Accepts the same options as the JSON datasource.

Returns

pyspark.sql.Column: a string representation of a StructType parsed from given JSON.

Examples

Python

import pyspark.sql.functions as sf
parsed1 = sf.schema_of_json(sf.lit('{"a": 0}'))
parsed2 = sf.schema_of_json('{a: 1}', {'allowUnquotedFieldNames':'true'})
spark.range(1).select(parsed1, parsed2).show()

Output

+------------------------+----------------------+
|schema_of_json({"a": 0})|schema_of_json({a: 1})|
+------------------------+----------------------+
|       STRUCT<a: BIGINT>|     STRUCT<a: BIGINT>|
+------------------------+----------------------+