Marqo — Tensor Search Engine for AI-Powered Retrieval

Introduction

Marqo is an open-source tensor search engine that combines embedding generation and vector search into a single deployable service. It eliminates the need to manage separate embedding pipelines and vector databases by handling the full chain from raw text or images to ranked search results.

What Marqo Does

• Indexes text and images by automatically generating vector embeddings at ingestion time
• Performs semantic search across documents using natural language queries
• Supports multimodal search combining text and image inputs in a single query
• Provides hybrid search blending keyword BM25 scoring with vector similarity
• Handles filtering, pagination, and faceting alongside semantic ranking

Self-Hosting & Configuration

• Deploy with a single Docker command; no external dependencies required
• Configure the embedding model through the index settings API
• Set inference device to CPU or GPU depending on available hardware
• Adjust index parameters including number of replicas and shard count
• Use environment variables to configure memory limits and batch sizes

Architecture Overview

Marqo bundles a vector store, an embedding inference service, and a REST API into a single container. When documents are added, the service runs them through a configurable embedding model (CLIP, SBERT, or custom ONNX models) and stores both the raw data and resulting vectors. At query time, the query is embedded using the same model and compared against stored vectors using approximate nearest neighbor search.

Key Features

• Zero-config embedding: no separate model serving or preprocessing pipeline needed
• Built-in support for CLIP models enabling text-to-image and image-to-image search
• Hybrid lexical and semantic search with configurable score blending
• Field-level weighting lets you boost specific attributes during search
• GPU-accelerated inference for high-throughput indexing and querying

Comparison with Similar Tools

• Meilisearch — Fast full-text search but lacks native vector embedding; Marqo handles the full semantic pipeline
• Weaviate — Feature-rich vector database; Marqo is simpler to deploy with built-in model inference
• Qdrant — High-performance vector engine; requires external embedding generation unlike Marqo
• Elasticsearch with kNN — Established platform with added vector support; Marqo is purpose-built for tensor search
• Typesense — Typo-tolerant keyword search; Marqo adds semantic understanding through embeddings

FAQ

Q: Do I need a GPU to run Marqo? A: No, it runs on CPU. A GPU significantly speeds up embedding generation for large-scale indexing.

Q: What embedding models does Marqo support? A: It supports CLIP, Sentence Transformers, ONNX models, and OpenAI/Hugging Face hosted models.

Q: Can Marqo handle image search? A: Yes, using CLIP models it supports text-to-image, image-to-text, and image-to-image search.

Q: How does Marqo compare to building a custom RAG pipeline? A: Marqo replaces the embedding generation, vector storage, and retrieval components of a RAG pipeline with a single service.

Sources

• https://github.com/marqo-ai/marqo
• https://docs.marqo.ai/