# Velox — Unified Vectorized Execution Engine by Meta

> High-performance C++ vectorized database execution library designed as a shared backend for data processing systems like Presto, Spark, and PyTorch.

## Install

Save as a script file and run:

# Velox — Unified Vectorized Execution Engine by Meta

## Quick Use
```bash
# Clone and build
git clone https://github.com/facebookincubator/velox.git
cd velox
make setup-ubuntu  # or setup-macos
make release

# Run unit tests
make unittest
```

## Introduction
Velox is an open-source C++ library from Meta that provides a unified, high-performance vectorized execution engine for data processing. Instead of every query engine rebuilding the same operators, Velox offers a shared set of reusable components — type system, vector buffers, expression evaluation, and operators — that any data system can plug into.

## What Velox Does
- Provides a columnar in-memory data format with Apache Arrow compatibility
- Implements vectorized expression evaluation for SQL-like computations
- Offers a library of common operators: filter, project, join, aggregate, sort, and exchange
- Supports adaptive query execution with runtime filter pushdown and dynamic partition pruning
- Enables extensibility through user-defined functions, types, and connectors

## Architecture Overview
Velox operates on columnar vectors using a pull-based iterator model where operators request batches from their children. The expression evaluator compiles SQL expressions into a tree of vector functions that process entire columns at once. Memory management uses arena allocation with spill-to-disk support. Connectors abstract data sources (Hive, Iceberg, S3) behind a unified scan interface.

## Self-Hosting & Configuration
- Build from source on Linux or macOS using `make` with dependency setup scripts provided
- Integrate as a C++ library by linking against libvelox in your query engine
- Configure memory limits, spill directories, and thread pool sizes programmatically
- Register custom scalar and aggregate functions via the function registry API
- Use the built-in benchmarking framework to measure operator performance

## Key Features
- Vectorized processing delivers 3-10x speedups over row-at-a-time execution
- Adaptive execution reorders and prunes operations based on runtime statistics
- Shared codebase used by Presto, Spark (Gluten), and PyTorch (TorchArrow) at Meta
- Codegen support for JIT-compiling hot expressions
- Comprehensive function library covering 300+ Presto-compatible SQL functions

## Comparison with Similar Tools
- **Apache DataFusion** — Rust-based query engine; Velox is a C++ library designed for embedding into existing engines
- **Apache Arrow Acero** — Arrow's built-in execution engine; Velox offers more advanced adaptive execution
- **DuckDB** — Standalone analytical database; Velox is a composable library, not a complete database
- **ClickHouse** — Full OLAP database; Velox provides building blocks rather than a turnkey solution

## FAQ
**Q: Is Velox a database?**
A: No. Velox is an execution library. Query engines like Presto and Spark use Velox as their compute backend.

**Q: What languages can use Velox?**
A: Velox is a C++ library with Python bindings (PyVelox) for experimentation and testing.

**Q: Does Velox support GPU acceleration?**
A: Not natively. Velox focuses on CPU vectorization, though community efforts explore GPU integration.

**Q: Who uses Velox in production?**
A: Meta uses Velox in its Presto deployment. The Gluten project brings Velox to Apache Spark.

## Sources
- https://github.com/facebookincubator/velox
- https://velox-lib.io/

---
Source: https://tokrepo.com/en/workflows/asset-ceb66b2f
Author: Script Depot