aiqbench takeChips & Infra

Elastic reports speedups for Jina models after moving its hosted inference services from NVIDIA L40 to RTX Pro 6000 Blackwell GPUs: up to 8x for the slowest embedding requests, over 4x for reranking, and about 3.7x for OCR. These are vendor-reported figures for a subset of migrated models, with no independent benchmarks or full methodology yet. For readers building retrieval or agentic systems, the practical point is that latency in multi-model pipelines depends heavily on the serving hardware behind managed endpoints.

Sources coveringElastic Security Labs, AWS

Elastic looks to speed up Jina Model Inference by 8x using NVIDIA RTX GPUs

Elastic Security LabsOctober 21, 20264 min read

Elastic is accelerating Jina model inference with NVIDIA RTX Pro 6000 GPUs, delivering up to 8x faster embedding, 4x faster reranking, and 3.7x faster OCR to power search and agentic AI.

Giving AI agents the ability to accurately answer questions grounded in content involves running different kinds of models, each handling a different workload, and every step can add latency that users wait through. Jina AI by Elastic provides small, frontier-grade reader, OCR, embedding, and ranking models that preserve meaning from ingestion to retrieval. Elastic offers them as two managed services: the Jina API for direct calls to Jina models and Elastic Inference Service (EIS), which provides pre-configured, GPU-accelerated endpoints natively integrated with…

Excerpt shown under the publisher's RSS terms · 82 words · full text and figures are on the publisher's site

Read at Elastic Security Labs ↗Ask about this article