Post Syndicated from Dylan Tong original https://aws.amazon.com/blogs/big-data/auto-optimize-your-amazon-opensearch-service-vector-database/

AWS recently announced the general availability of auto-optimize for the Amazon OpenSearch Service vector engine. This feature streamlines vector index optimization by automatically evaluating configuration trade-offs across search quality, speed, and cost savings. You can then run a vector ingestion pipeline to build an optimized index on your desired collection or domain. Previously, optimizing index configurations—including algorithm, compression, and engine settings—required experts and weeks of testing. This process must be repeated because optimizations are unique to specific data characteristics and requirements. You can now auto-optimize vector databases in under an hour without managing infrastructure and acquiring expertise in index tuning.

In this post, we discuss how the auto-optimize feature works, its benefits, and share examples of auto-optimized results.

Overview of vector search and vector indexes

Vector search is a technique that improves search quality and is a cornerstone of generative AI applications. It involves using a type of AI model to convert content into numerical encodings (vectors), enabling content matching by semantic similarity instead of just keywords. You build vector databases by ingesting vectors into OpenSearch to build indexes that enable searches across billions of vectors in milliseconds.

Benefits of optimizing vector indexes and how it works

The OpenSearch vector engine provides a variety of index configurations that help you make favorable trade-offs between search quality (recall), speed (latency), and cost (RAM requirements). There isn’t a universally optimal configuration. Experts must evaluate combinations of index settings such as Hierarchal Navigable Small Worlds (HNSW) algorithm parameters (such as m or ef_construction), quantization techniques (such as scalar, binary, or product), and engine parameters (such as memory-optimized, disk-optimized, or warm-cold storage). The difference between configurations could be a 10% or more difference in search quality, hundreds of milliseconds in search latency, or up to three times in cost savings. For large-scale deployments, cost-optimizations can make or break your budget.

The following figure is a conceptual illustration of trade-offs between index configurations.

Optimizing vector indexes is time-consuming. Experts must build an index; evaluate its speed, quality, and cost; and make appropriate configuration adjustments before repeating this process. Running these experiments at scale can take weeks because building and evaluating large-scale index requires substantial compute power, resulting in hours to days of processing for just one index. Optimizations are unique to specific business requirements and each dataset, and trade-off decisions are subjective. The best trade-offs depend on the use case, such as search for an internal wiki or an e-commerce site. Therefore, this process must be repeated for each index. Lastly, if your application data changes continuously, your vector search quality might degrade, requiring you to rebuild and re-optimize your vector indexes regularly.

Solution overview

With auto-optimize, you can run jobs to produce optimization recommendations, consisting of reports that detail performance measurements and explanations of the recommended configurations. You can configure auto-optimize jobs by simply providing your application’s acceptable search latency and quality requirements. Expertise in k-NN algorithms, quantization techniques, and engine settings aren’t required. It avoids the one-size-fits-all limitations of solutions based on a few pre-configured deployment types, offering a tailored fit for your workloads. It automates the manual labor previously described. You simply run serverless, auto-optimize jobs at a flat rate per job. These jobs don’t consume your collection or domain resources. OpenSearch Service manages a separate multi-tenant warm pool of servers, and parallelizes index evaluations across secure, single-tenant workers to deliver results quickly. Auto-optimize is also integrated with vector ingestion pipelines, so you can quickly build an optimized vector index on a collection or domain from an Amazon Simple Storage Service (Amazon S3) data source.

The following screenshot illustrates how to configure an auto-optimize job on the OpenSearch Service console.

When the job is complete (typically, within 30–60 minutes for million-plus-size datasets), you can review the recommendations and reports, as shown in the following screenshot.

The screenshot illustrates an example where you need to choose the best trade-offs. Do you select the first option, which delivers the highest cost savings (through lower memory requirements)? Or do you select the third option, which delivers a 1.76% search quality improvement, but at higher cost? If you want to understand the details of the configurations used to deliver these results, you can view the sub-tabs on the Details pane, such as the Algorithm parameters tab shown in the preceding screenshot.

After you’ve made your choice, you can build your optimized index on your target OpenSearch Service domain or collection, as shown in the following screenshot. If you’re building the index on a collection or a domain running OpenSearch 3.1+, you can enable GPU-acceleration to increase the build speed up to 10 times faster at a quarter of the indexing cost.

Auto-optimize results

The following table presents a few examples of auto-optimize results. To quantify the value of running auto-optimize, we present gains compared to default settings. The estimated RAM requirements are based on standard domain sizing estimates:

Required RAM = 1.1 x (bytes per dimension x dimensions + hnsw.parameters.m x 8) x vector count

We estimate cost savings by comparing the minimal infrastructure (has just enough RAM) to host an index with the default compared to optimized settings.

Conclusion

You can start building auto-optimized vector databases on the Vector ingestion page of the OpenSearch Service console. Use this feature with GPU-accelerated vector indexes to build optimized, billion-scale vector databases within hours.

Auto-optimize is available for OpenSearch Service vector collections and OpenSearch 2.17+ domains in the US East (N. Virginia, Ohio), US West (Oregon), Asia Pacific (Mumbai, Singapore, Sydney, Tokyo), and Europe (Frankfurt, Ireland, Stockholm) AWS Regions.

About the authors

Post Syndicated from Dylan Tong original https://aws.amazon.com/blogs/big-data/build-billion-scale-vector-databases-in-under-an-hour-with-gpu-acceleration-on-amazon-opensearch-service/

AWS recently announced the general availability of GPU-accelerated vector (k-NN) indexing on Amazon OpenSearch Service. You can now build billion-scale vector databases in under an hour and index vectors up to 10 times faster at a quarter of the cost. This feature dynamically attaches serverless GPUs to boost domains and collections running CPU-based instances. With this feature, you can scale AI apps quickly, innovate faster, and run vector workloads leaner.

In this post, we discuss the benefits of GPU-accelerated vector indexing, explore key use cases, and share performance benchmarks.

Overview of vector search and vector indexes

Vector search is a technique that improves search relevance, and is a cornerstone of generative AI applications. It involves using an embeddings model to convert content into numerical encodings (vectors), enabling content matching by semantic similarity instead of just keywords. You can build vector databases by ingesting vectors into OpenSearch Service to build indexes that enable searches across billions of vectors in milliseconds.

Challenges with scaling vector databases

Customers are increasingly scaling vector databases to multi-billion-scale on OpenSearch Service to power generative AI applications, product catalogs, knowledge bases, and more. Applications are becoming increasingly agentic, integrating AI agents that rely on vector databases for high-quality search results across enterprise data sources to enable chat-based interactions and automation.

However, there are challenges on the way to billion-scale. First, multi-million to billion-scale vector indexes take hours to days to build. These indexes use algorithms like Hierarchal Navigable Small Worlds (HNSW) to enable high-quality, millisecond searches at scale. However, they require more compute power than traditional indexes to build. Furthermore, you have to rebuild your indexes whenever your model changes, such as switching between vendors, versions, or after fine-tuning. Some use cases such as personalized search require models to be fine-tuned daily and adapt to evolving user behaviors. All vectors must be regenerated when the model changes, so the index must be rebuilt. HNSW can also degrade following significant updates and deletes, so indexes must be rebuilt to regain accuracy.

Lastly, as your agentic applications become more dynamic, your vector database must scale for heavy streaming ingestion, updates, and deletes while maintaining low search latency. If search and indexing use the same infrastructure, these intensive processes will compete for limited compute and RAM, so search latency can degrade.

Solution overview

You can overcome these challenges by enabling GPU-accelerated indexing on OpenSearch Service 3.1+ domains or collections. GPU acceleration will dynamically activate, for instance, in response to a reindex command on a million-plus-size index. During activation, index tasks are offloaded to GPU servers that run NVIDIA cuVS to build HNSW graphs. Superior speed and efficiency are achieved through parallelization of vector operations. Inverted indexes will continue using your cluster’s CPU for indexing and search on non-vector data. These indexes operate alongside HNSW to support keyword, hybrid, and filtered vector search. The resources required to build inverted indexes is low compared to HNSW.

GPU acceleration is enabled as a cluster-level configuration, but it can be disabled on individual indexes. This feature is serverless, so you don’t need to manage GPU instances. You simply pay-per-use through OpenSearch Compute Units (OCUs).

The following diagram illustrates how this feature works.

The workflow consists of the following steps:

1. You write vectors into your domain or collection, using the existing APIs: bulk, reindex, index, update, delete, and force merge.
2. GPU acceleration is activated when the indexed vector data surpasses a configured threshold within a refresh interval.
3. This leads to a secure, single-tenant assignment of GPU servers to your cluster from a multi-tenant warm pool of GPUs managed by OpenSearch Service.
4. Within milliseconds, OpenSearch Service initiates and offloads HNSW operations.
5. When the write volume falls below the threshold, GPU servers are scaled down and returned to the warm pool.

This automation is fully managed. You only pay for acceleration time, which you can monitor from Amazon CloudWatch.

This feature isn’t just designed for ease of use. It enables GPU acceleration benefits without economic challenges. For example, a domain sized to host 1 billion (1,024 dimension) vectors compressed 32 times (using binary quantization) takes three r8g.12xlarge.search instances to provide the required 1.15 TBs of RAM. A design that requires running a domain on GPU instances, would need six g6.12xlarge instances to do the same, resulting in 2.4 times higher cost and excessive GPUs. This solution delivers efficiency by providing the right amount of GPUs only when you need them, so you gain speed with cost savings.

Use cases and benefits

This feature has three primary uses and benefits:

• Build large-scale indexes faster, increasing productivity and innovation velocity
• Reduce cost by lowering Amazon OpenSearch Serverless indexing OCU usage, or downsizing domains with write-heavy vector workloads
• Accelerate writes, lower search latency, and improve user experience on your dynamic AI applications

In the following sections, we discuss these use cases in more detail.

Build large-scale indexes faster

We benchmarked index builds for 1M, 10M, 113M, and 1B vector test cases to demonstrate speed gains on both domains and collections. Speed gains ranged from 6.4 to 13.8 times faster. These tests were performed with production configurations (Multi-AZ with replication) and default GPU service limits. All tests were run on right-sized search clusters, and the CPU-only tests had CPU utilization maxed exclusively for indexing. The following chart illustrates the relative speed gains from GPU acceleration on managed domains.

The total index build time on domains includes a force merge to optimize the underlying storage engine for search performance. During normal operation, merges are automatic. However, when benchmarking domains, we perform a manual merge after indexing to make sure merging impact is consistent across tests. The following table summarizes the index build benchmarks and dataset references for domains.

We ran the same performance tests on collections. The performance is different on OpenSearch Serverless because its serverless architecture involves performance trade-offs such as automatic scaling, which introduces a ramp-up to reach peak performance. The following table summarizes these results.

OpenSearch Serverless doesn’t support force merge, so the full benefit from GPU acceleration might be delayed until the automatic background merges complete. The default minimum OCUs had to be increased for tests beyond 1 million vectors to handle higher indexing throughput.

Reduce cost

Our serverless GPU design uniquely delivers speed gains and cost savings. With OpenSearch Serverless, your net indexing costs will be reduced if you have indexing workloads that are significant enough to activate GPU acceleration. The following table presents the OCU usage and cost consumption usage from the previous index build tests.

The vector acceleration OCUs offload and reduce indexing OCUs. The total OCU usage is less with GPU because the index is built more efficiently, resulting in cost savings.

With managed domains, cost savings are situational because search and indexing infrastructure isn’t decoupled like on OpenSearch Serverless. However, if you have a write-heavy, compute-bound vector search application (that is, your domain is sized for vCPUs to sustain write throughput), you could downsize your domain.

The following benchmarks demonstrate the efficiency gains from GPU acceleration. We measure the infrastructure costs during the indexing tasks. GPU acceleration has the additional cost of GPUs at $0.24 per OCU/hour. However, because indexes are built faster and more efficiently, it’s more economical to use GPU to reduce CPU utilization on your domain and downsize it.

*Domains are running a high-availability configuration without any cost-optimizations

Accelerate writes, lower search latency

In experienced hands, domains offer operational control and the ability to achieve great scalability, performance, and cost optimizations. However, operational responsibilities include managing indexing and search workloads on shared infrastructure. If your vector deployment involves heavy, sustained streaming ingestion, updates, and deletes, you might observe higher search times on your domain. As illustrated in the following chart, as you increase vector writes, the CPU utilization increases to support HNSW graph building. Concurrent search latency also increases because of competition for compute and RAM resources.

You could solve the problem by adding data nodes to increase your domain’s compute capacity. However, enabling GPU acceleration is simpler and cheaper. As illustrated in the chart, GPU frees up CPU and RAM on your domain, helping you sustain low and stable search latency under high write throughput.

Get started

Ready to get started? If you already have an OpenSearch Service vector deployment, use the AWS Management Console, AWS Command Line Interface (AWS CLI), or API to enable GPU acceleration on your OpenSearch 3.1+ domain or vector collection. Test it with your existing indexing workloads. If you’re planning to build a new vector database, try out our new vector ingestion feature, which simplifies vector ingestion, indexing, and automates optimizations. Check out this demonstration on YouTube.

Acknowledgments

The authors would like to thank Manas Singh, Nathan Stephens, Jiahong Liu, Ben Gardner, and Zack Meeks from NVIDIA, and Yigit Kiran and Jay Deng from AWS for their contributions to this post.

About the authors

Authors would like to add special thanks to Manas Singh, Nathan Stephens, Jiahong Liu, Ben Gardner, Zack Meeks NVIDIA and Yigit Kiran and Jay Deng from AWS.