Development Guide¶

This page explains how to set up a local development environment for dot-explorer so you can edit both the Rust extension and the Python package and run the full test suite.

Prerequisites¶

Install Rust¶

dot-explorer requires a working Rust toolchain (stable channel). Install it with rustup:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
rustup update stable

Verify the installation:

rustc --version
cargo --version

Create a conda environment¶

A minimal conda environment file is provided at environment.yml. It pins Python to 3.14 (free-threaded build) and installs the key Python dependencies.

conda env create -f environment.yml
conda activate dot-explorer

Alternatively, use any Python ≥ 3.12 virtual environment:

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

Fork and clone the repository¶

Fork dot-explorer on GitHub, then clone your fork:

git clone https://github.com/<your-username>/dot-explorer.git
cd dot-explorer
git remote add upstream https://github.com/Adamtaranto/dot-explorer.git

Install in development mode¶

maturin is the build backend for the Rust extension. Install it and then build the package in editable mode:

pip install maturin
# Build Rust extension + install package in editable mode
maturin develop --extras dev,docs

The --extras dev,docs flag installs all optional development and documentation dependencies declared in pyproject.toml.

Verify the install:

import dot_explorer
print(dot_explorer.__version__)

Installing pre-commit hooks¶

dot-explorer uses pre-commit to enforce code quality on every commit. Install the hooks once after cloning:

pip install pre-commit   # already included in the dev extras
pre-commit install                 # install commit-stage hooks
pre-commit install --hook-type pre-push   # install push-stage hooks

The hooks include:

Run all hooks manually at any time:

pre-commit run --all-files

Running Python tests¶

pytest tests/ -v

To run a specific test file or test function:

pytest tests/test_index.py -v
pytest tests/test_index.py::test_get_paf_all_returns_paf_lines -v

Python code style — ruff¶

dot-explorer uses ruff for Python linting and formatting (configured in pyproject.toml).

Check for issues:

ruff check python/ tests/

Auto-fix fixable issues:

ruff check --fix python/ tests/

Format code:

ruff format python/ tests/

Rust code quality¶

Format¶

cargo fmt

Check formatting without modifying files (also used by the pre-commit hook):

cargo fmt --all -- --check

Lint (Clippy)¶

cargo clippy -- -D warnings

Compile check¶

Quickly verify the crate compiles without producing a binary:

cargo check

Rust unit tests¶

Run the Rust-side library tests:

cargo test --lib

Run all Rust tests (including integration tests, if any):

cargo test

Benchmarks¶

dot-explorer ships both Rust and Python benchmarks, run in CI via CodSpeed (.github/workflows/codspeed.yml). Both feed on deterministic, seeded synthetic DNA so results are reproducible; the real assemblies used during development are private and git-ignored.

Rust benchmarks¶

The Rust benches live in benches/ and use codspeed-criterion-compat (a drop-in criterion API), so the same targets run under plain cargo bench and under cargo codspeed:

cargo bench                       # all benches
cargo bench --bench bench_compare # forward/reverse matching only

Python benchmarks¶

The Python benches live in python/benchmarks/ and use pytest-codspeed (declared in the test extra). Run them with:

pip install '.[test]'
pytest python/benchmarks --codspeed

They cover FASTA byte parsing, the CrossIndex build + compute_matches path, contig reordering, dotplot rendering, and HTML report generation.

K-mer memory profile¶

CodSpeed measures instructions, not memory. scripts/mem_profile_index.py builds CrossIndex pairs over synthetic homologous sequences in clean subprocesses and reports peak RSS plus the index's own accounting (SequenceIndex.approx_bytes(): retained sequence bytes, the CSR k-mer tables, and the pairwise coordinate cache):

python scripts/mem_profile_index.py --mb 10 50 100

Measured natively (macOS arm64, k=15, 2 % divergence, per-side sizes; the audit that sized the web app's limits):

Take-aways: match records store coordinates only (no sequence copies); the fixed cost is the CSR tables at ~15 B/bp, the retained raw sequence copies are minor (~1 B/bp), and the rest of the peak is the materialised Python PafRecord objects — min_block_len cuts the record count (39 M → 1.0 M at 25 bp here) without changing the index size. This matches the in-browser measurement (~32 B/bp of combined input at the 2.9 GB peak for a 90 Mb run), which is why the web app's k-mer gate sits at 80 Mb combined: the 4 GB wasm heap minus the Pyodide baseline supports ~110 Mb in theory, and 80 Mb keeps a ~30 % margin.

Building the documentation locally¶

The site is built with Zensical (configured in zensical.toml), the successor to MkDocs + Material by the same authors. Install the docs dependencies (included in maturin develop --extras dev,docs), then render the tutorial notebooks and serve:

python scripts/notebooks_to_md.py   # docs/tutorials/*.ipynb -> *.md
zensical serve

Open http://127.0.0.1:8000 in your browser. The site rebuilds automatically when you save a documentation file.

To build a static site into site/:

zensical build --clean