Skip to main content
Last updated on

UserDefinedTableFunction

A user-defined table function in Python.

The constructor of this class is not supposed to be directly called. Use pyspark.sql.functions.udtf to create an instance.

Syntax​

Python
from pyspark.sql.functions import udtf

@udtf(returnType="c1: int, c2: int")
class MyUDTF:
    def eval(self, x: int):
        yield x, x + 1

Properties​

Property

Description

returnType

The return type of the user-defined table function as a StructType, or None if not specified.

Methods​

Method

Description

asDeterministic()

Updates the UserDefinedTableFunction to deterministic.

Notes​

This API is evolving.

Examples​

Python
from pyspark.sql.functions import lit, udtf

@udtf(returnType="c1: int, c2: int")
class PlusOne:
    def eval(self, x: int):
        yield x, x + 1

PlusOne(lit(1)).show()
Output
+---+---+
| c1| c2|
+---+---+
|  1|  2|
+---+---+
Python
_ = spark.udtf.register(name="plus_one", f=PlusOne)
spark.sql("SELECT * FROM plus_one(1)").collect()
Output
[Row(c1=1, c2=2)]
Python
spark.sql("SELECT * FROM VALUES (0, 1), (1, 2) t(x, y), LATERAL plus_one(x)").collect()
Output
[Row(x=0, y=1, c1=0, c2=1), Row(x=1, y=2, c1=1, c2=2)]