Cython — Write C Extensions for Python Using Python-Like Syntax

Introduction

Cython is both a programming language and a compiler. The language is a superset of Python that adds optional C-level type declarations. The compiler translates Cython source files into C code, which is then compiled into Python extension modules. Many core scientific Python libraries (NumPy, SciPy, pandas, scikit-learn) use Cython internally for performance-critical code paths.

What Cython Does

• Compiles Python-like code to C for 10-100x speedup on numerical workloads
• Adds static type declarations (cdef int, cdef double) to eliminate Python overhead
• Wraps existing C and C++ libraries for seamless access from Python
• Generates Python extension modules (.so/.pyd) compatible with CPython
• Supports typed memoryviews for efficient NumPy array access without GIL contention

Architecture Overview

Cython reads .pyx source files and translates them into C source files. Each Python-level construct is mapped to CPython C API calls, while typed variables and functions are compiled to direct C operations. The generated C file is compiled by a standard C compiler (gcc, clang, MSVC) into a shared library that CPython loads as a native extension module.

Self-Hosting & Configuration

• Install with pip install cython and ensure a C compiler is available (gcc, clang, or MSVC)
• Write .pyx files with optional cdef type annotations for performance-critical variables
• Compile with cythonize -i module.pyx for quick builds or use a setup.py with setuptools
• Use .pxd declaration files to share typed declarations across Cython modules
• Enable the annotate flag (cython -a module.pyx) to generate an HTML report showing Python/C interaction hotspots

Key Features

• Superset of Python: any valid Python file is valid Cython (gradual optimization)
• Direct C-level memory access via typed memoryviews for NumPy arrays
• nogil blocks release the GIL for true multi-threaded parallelism in C sections
• Wraps C/C++ libraries with cdef extern from declarations
• Pure Python mode allows type hints via decorators without changing .py files

Comparison with Similar Tools

• Numba — JIT compiler with decorators; no compilation step but limited to numerical subset of Python
• PyPy — Alternative Python interpreter with tracing JIT; faster for general code but incompatible with some C extensions
• pybind11 — C++ binding tool; better for wrapping complex C++ APIs but requires writing C++
• CFFI — Foreign function interface for calling C code; simpler for one-off C calls but no optimization of Python code
• Mypyc — Compiles type-annotated Python via mypy; easier adoption but less mature and fewer optimizations

FAQ

Q: Do I need to rewrite my entire codebase in Cython? A: No. Cython is designed for incremental optimization. Profile first, then convert only the hot functions to .pyx files with type annotations.

Q: Does Cython work with Python 3.12+? A: Yes. Cython 3.0+ fully supports Python 3.12 and later, including the latest CPython internals.

Q: How do I distribute Cython extensions to users? A: Include the generated C files in your sdist so users do not need Cython installed. Ship wheels with precompiled binaries for common platforms.

Q: Can Cython compile async/await code? A: Yes. Cython 3.0 supports async def, await, and async for with full type optimization.

Sources

• https://github.com/cython/cython
• https://cython.readthedocs.io/