How to upgrade embedding models in AlloyDB AI with zero downtime

Imagine you've just launched a new semantic search feature for product reviews to help customers quickly research products on your site. Users love it. But the next day, you see an announcement for a new vector embedding model promising higher accuracy, better nuance, and multilingual search out of the box. After testing it in your dev environment, you confirm it's a perfect fit. But now you're left with a common dilemma: How do you upgrade the model without disrupting your users? Upgrading embedding models in production requires re-embedding your entire dataset, rebuilding indexes, and coordinating schema migrations without interrupting user queries. You may encounter several potential problems: Embedding generation: All existing data must be re-embedded using the new embedding model, which typically requires updating complex pipelines and using dual write strategies while the new and old models run side-by-side for a period of time.  Mathematical incompatibility: Embeddings generated by different models inhabit distinct vector spaces and cannot be compared directly. Application query vectors and stored column vectors must switch models in unison to prevent severe search degradation. Index rebuild overhead: Vector indexes must be built on the new vectors before similarity search can perform efficiently at scale. Degraded user experience: Switching embedding models requires multiple DDL changes and other operations that acquire exclusive locks and can stall incoming read and write transactions if not carefully managed. How AlloyDB AI simplifies vector lifecycle management In this blog, we’ll highlight how the rich feature set in AlloyDB AI combines to enable zero-downtime embedding model upgrades, so that your users can benefit from the latest model advances without scheduled downtime maintenance. Model endpoint management gives you native in-database access to any embedding model (first party, third party, or even self-hosted), while auto embeddings rapidly generates new embeddings in the background (up to 3000/second) and keeps them up to date as source data changes. Using AlloyDB’s ScaNN index can also significantly reduce the wall clock time of your upgrade with index builds that are 10x faster than open source HNSW, and native PostgreSQL functionality like concurrent index creation, set-returning functions (SRF), and atomic column renames allow transparent blue/green and canary-style rollouts. Step-by-step migration walkthrough In this example, we demonstrate migrating a production table ecomm.product_reviews from text-embedding-005 (768-d) to gemini-embedding-001 (3072-d) while serving live traffic. Step 0: Prerequisites and preflight checks Verify that the google_ml_integration extension is updated to version 1.5.6 or later (tested on 1.6.1) and confirm model connectivity: code_block <ListValue: [StructValue([('code', "-- Check extension version\r\nSELECT extversion \r\nFROM pg_extension \r\nWHERE extname = 'google_ml_integration';\r\n\r\n-- Create extensions\r\nCREATE EXTENSION IF NOT EXISTS vector;\r\nCREATE EXTENSION IF NOT EXISTS alloydb_scann;\r\n\r\n-- Verify remote endpoint and output dimension (3072-d)\r\nSELECT vector_dims(ai.embedding('gemini-embedding-001', 'hello world')::vector);\r\n\r\n-- Inspect existing auto-embedding settings\r\nSELECT * FROM ai.embedding_settings_view;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f61cd3a94d0>)])]> Step 1: Add the new vector column Adding a nullable column without default values updates catalog metadata without creating new versions of each row. This makes the operation nearly instant. Note that the size of your vector column (3072 in this example) must match the number of dimensions output by your embedding model. We also set the lock_timeout to 2 seconds to prevent the operation from blocking critical production traffic while it waits to acquire an exclusive lock. If the operation fails due to a longer wait for an exclusive lock, simply retry. code_block <ListValue: [StructValue([('code', "SET lock_timeout = '2s';\r\n\r\nALTER TABLE ecomm.product_reviews \r\nADD COLUMN new_embedding_col vector(3072);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f61cd37d710>)])]> Step 2: Initialize auto-embeddings and run backfill Next, we initialize embeddings on our new embedding column using gemini-embedding-001. code_block <ListValue: [StructValue([('code', "CALL ai.initialize_embeddings(\r\n model_id => 'gemini-embedding-001',\r\n table_name => 'ecomm.product_reviews',\r\n content_column => 'review_text',\r\n embedding_column => 'new_embedding_col',\r\n batch_size => 50,\r\n incremental_refresh_mode => 'manual'\r\n);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f61cd367950>)])]> Note that there are two modes available in auto embeddings to keep embeddings up to date as source data changes:   Transactional: When you set incre

Extras afișat conform termenilor RSS ai publicației · 200 cuvinte · textul integral și figurile sunt pe site-ul publicației