Vector Search foundational embedding model (GTE) example
This notebook shows how to use the Vector Search Python SDK, which provides a VectorSearchClient as a primary API for working with Vector Search.
This notebook uses Databricks Foundation Model APIs (AWS|Azure) to access the GTE embeddings model to generate embeddings.
Load toy dataset into source Delta table
The following creates the source Delta table.
Chunk sample dataset
Chunking the sample dataset helps you avoid exceeding the context limit of the embedding model. The GTE model supports up to 8192 tokens. However, Databricks recommends that you split the data into smaller context chunks so that you can feed a wider variety of examples into the reasoning model for your RAG application.
Create vector search endpoint
Create vector index
Similarity search
The following cells show how to query the Vector Index to find similar documents.