Chat model batch inference tasks using PySpark Pandas UDF
This notebook is the partner notebook to the Perform batch inference on a provisioned throughput endpoint notebook. This notebook and the Perform batch inference on a provisioned throughput endpoint notebook must be in the same directory of your workspace for the batch inference workflow to perform successfully.
The following tasks are accomplished in this notebook:
- Read data from the input table and input column
- Construct the requests and send the requests to a Foundation Model APIs endpoint with some kind of concurrency
- Persist input row together with the response data to the output table
Requirements
To run this notebook you need:
The Databricks Runtime for Machine Learning version 14.3 LTS or above
Chat model batch inference tasks using PySpark Pandas UDF
This notebook is the partner notebook to the Perform batch inference on a provisioned throughput endpoint notebook. This notebook and the Perform batch inference on a provisioned throughput endpoint notebook must be in the same directory of your workspace for the batch inference workflow to perform successfully.
The following tasks are accomplished in this notebook:
Requirements
To run this notebook you need:
The Databricks Runtime for Machine Learning version 14.3 LTS or above