ai_mask function
Applies to: 
Databricks SQL Databricks Runtime
This feature is in Public Preview.
During the preview:
- This function is only supported in
us-east1andus-central1. - The underlying language model can handle several languages, but this AI Function is tuned for English.
The ai_mask() function allows you to invoke a state-of-the-art generative AI model to mask specified entities in a given text using SQL. This function uses a chat model serving endpoint made available by Databricks Foundation Model APIs.
Requirements
The underlying models that might be used at this time are licensed under the Apache 2.0 License, Copyright © The Apache Software Foundation or the LLAMA 3.3 Community License Copyright © Meta Platforms, Inc. All rights reserved. Customers are responsible for ensuring compliance with applicable model licenses.
Databricks recommends reviewing these licenses to ensure compliance with any applicable terms. If models emerge in the future that perform better according to Databricks’s internal benchmarks, Databricks might change the model (and the list of applicable licenses provided on this page).
- This function is only available on workspaces in AI Functions using Foundation Model APIs supported regions.
- Check the Databricks SQL pricing page.
- In Databricks Runtime 15.1 and above, this function is supported in Databricks notebooks, including notebooks that are run as a task in a Databricks workflow.
- For batch inference workloads, Databricks recommends Databricks Runtime 15.4 ML LTS for improved performance.
- See limitations.
Syntax
ai_mask(content, labels)
Arguments
content: ASTRINGexpression.labels: AnARRAY<STRING>literal. Each element represents a type of information to be masked.
Returns
A STRING where the specified information is masked.
If content is NULL, the result is NULL.
Examples
> SELECT ai_mask(
'John Doe lives in New York. His email is john.doe@example.com.',
array('person', 'email')
);
"[MASKED] lives in New York. His email is [MASKED]."
> SELECT ai_mask(
'Contact me at 555-1234 or visit us at 123 Main St.',
array('phone', 'address')
);
"Contact me at [MASKED] or visit us at [MASKED]"
Limitations
- This function is not available on Databricks SQL Classic.
- This function can not be used with Spark UDFs.
- This function can not be used with Views.