to_timestamp

Converts a column into TimestampType using the optionally specified format. Specify formats according to datetime pattern. By default, it follows casting rules to TimestampType if the format is omitted. Equivalent to col.cast("timestamp").

For the corresponding Databricks SQL function, see to_timestamp function.

Syntax

Python

import pyspark.sql.functions as sf

sf.to_timestamp(col=<col>)

# With format
sf.to_timestamp(col=<col>, format=<format>)

Parameters

Parameter	Type	Description
col	pyspark.sql.Column or str	Column values to convert.
format	str	Optional. Format to use to convert timestamp values.

Returns

pyspark.sql.Column: timestamp value as pyspark.sql.types.TimestampType type.

Examples

Example 1: Convert string to a timestamp.

Python

import pyspark.sql.functions as sf
df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
df.select(sf.to_timestamp(df.t)).show()

Output

+-------------------+
|    to_timestamp(t)|
+-------------------+
|1997-02-28 10:30:00|
+-------------------+

Example 2: Convert string to a timestamp with a format.

Python

import pyspark.sql.functions as sf
df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
df.select(sf.to_timestamp(df.t, 'yyyy-MM-dd HH:mm:ss')).show()

Output

+------------------------------------+
|to_timestamp(t, yyyy-MM-dd HH:mm:ss)|
+------------------------------------+
|                 1997-02-28 10:30:00|
+------------------------------------+