Text files

You can process files with the text format option to parse each line in any text-based file as a row in a DataFrame. This can be useful for a number of operations, including log parsing. It can also be useful if you need to ingest CSV or JSON data as raw strings.

For more information, see text files.

Options

See the following Apache Spark reference articles for supported read and write options.

• Read

  • Python

  • Scala:org.apache.spark.sql.DataFrame)

• Write

  • Python

  • Scala:Unit)