q — Run SQL Directly on CSV, TSV, and Log Files

What it is

q is a command-line tool that lets you run full SQLite-compatible SQL queries directly against CSV, TSV, and any delimited text file on disk. Instead of importing data into a database or writing awk/sed pipelines, you write SQL and q handles parsing, column detection, and query execution. It supports JOIN operations across multiple files, aggregations, and subqueries.

q targets data analysts, SREs, and developers who frequently analyze structured text files. It is particularly useful for one-off log analysis, CSV exploration, and quick data investigations where setting up a database would be overkill.

How it saves time or tokens

q eliminates the database import step for ad-hoc queries. Instead of creating a table, loading data, querying, and cleaning up, you write a single SQL statement against the file. For log analysis during incidents, this means getting answers in seconds rather than minutes. JOIN operations across multiple CSV files replace multi-step shell pipelines with a single readable query.

How to use

1. Install q:

brew install q

1. Run a SQL query against a CSV file:

q -H -d , "SELECT SUM(amount) FROM orders.csv WHERE status = 'paid'"

1. Join multiple files:

q -H -d , "SELECT o.id, c.email FROM orders.csv o JOIN customers.csv c ON o.cid = c.id"

Example

Analyze web server access logs with SQL:

# Top 10 IPs by request count
q "SELECT c1 as ip, COUNT(*) as hits FROM access.log GROUP BY c1 ORDER BY hits DESC LIMIT 10"

# 404 errors in the last hour
q "SELECT c7 as path, COUNT(*) FROM access.log WHERE c9 = '404' GROUP BY c7 ORDER BY 2 DESC"

# Join access logs with a blocklist CSV
q -H "SELECT a.c1, b.reason FROM access.log a JOIN blocklist.csv b ON a.c1 = b.ip"

Columns without headers are referenced as c1, c2, c3, etc.

Related on TokRepo

• AI Tools for Database — Database tools including SQL clients and query optimizers
• AI Tools for DevOps — DevOps tools for log analysis and infrastructure management

Common pitfalls

• Without the -H flag, q treats the first row as data, not headers. Always use -H for CSV files with header rows.
• The -d flag sets the delimiter. Forgetting it for CSV files (default is whitespace) produces incorrect column parsing.
• q loads the entire file into memory for query execution. Files larger than available RAM will cause out-of-memory errors. Use streaming tools for very large files.