Build Options
MTL5 provides several CMake options to control the build and enable optional features.
Core Options
Section titled “Core Options”| Option | Default | Description |
|---|---|---|
MTL5_BUILD_TESTS | ON | Build the Catch2 test suite |
MTL5_BUILD_EXAMPLES | ON | Build example programs |
MTL5_BUILD_REGRESSION_TESTS | OFF | Also build the large-scale solver regression tests in tests/regression/ (slow; see Regression tests) |
External Library Bindings
Section titled “External Library Bindings”These options link optional external libraries for hardware-accelerated or specialized solvers. MTL5 has native C++ implementations of all algorithms — these bindings are for users who want to delegate to optimized external libraries.
| Option | Default | Description |
|---|---|---|
MTL5_WITH_BLAS | OFF | Link BLAS library for dense acceleration |
MTL5_WITH_LAPACK | OFF | Link LAPACK library for factorizations |
MTL5_WITH_ZLIB | OFF | Link zlib for transparent gzip (.mtx.gz) Matrix Market reading |
MTL5_WITH_UMFPACK | OFF | Link UMFPACK library (SuiteSparse) |
MTL5_WITH_SUPERLU | OFF | Link SuperLU library |
MTL5_WITH_SUITESPARSE_KLU | OFF | Link KLU sparse solver (SuiteSparse) |
MTL5_WITH_SUITESPARSE_CHOLMOD | OFF | Link CHOLMOD Cholesky (SuiteSparse) |
MTL5_WITH_SUITESPARSE_SPQR | OFF | Link SuiteSparseQR (SuiteSparse) |
Vendor threads inside MTL5 regions
Section titled “Vendor threads inside MTL5 regions”A vendor BLAS/LAPACK, CHOLMOD and SPQR run thread teams of their own. With
MTL5_NUM_THREADS > 1, a vendor call made from inside an MTL5 parallel region
runs on one thread where the vendor allows a per-thread setting: MKL, and BLIS
1.0 or later. A call made outside a region keeps the vendor’s threads. The
configure log names the vendor it found (BLAS/LAPACK vendor: ...); select one
with BLA_VENDOR:
cmake -B build -DMTL5_WITH_BLAS=ON -DBLA_VENDOR=FLAME # BLIScmake -B build -DMTL5_WITH_BLAS=ON -DBLA_VENDOR=Intel10_64lp # threaded MKLcmake -B build -DMTL5_WITH_BLAS=ON -DBLA_VENDOR=OpenBLASOpenBLAS’s thread count can only be set for the whole process, so MTL5 leaves
it alone. With a threaded MTL5, set OPENBLAS_NUM_THREADS=1; with a threaded
OpenBLAS, leave MTL5_NUM_THREADS at 1. The other variables are
BLIS_NUM_THREADS, MKL_NUM_THREADS and OMP_NUM_THREADS. See
On-node threading.
Matrix Market I/O: gzip and large files
Section titled “Matrix Market I/O: gzip and large files”mtl::io::mm_read / mm_read_dense read SuiteSparse .mtx files. Two features
support circuit5M-scale workloads:
Transparent gzip. Built with -DMTL5_WITH_ZLIB=ON (defines MTL5_HAS_ZLIB),
mm_read("matrix.mtx.gz") decompresses on the fly — no need to pre-extract a
downloaded .tar.gz. Without the flag, passing a .gz path throws a clear error
pointing at the option. Plain .mtx reading is unaffected and needs no zlib.
cmake -B build -DCMAKE_CXX_STANDARD=20 -DMTL5_WITH_ZLIB=ONLarge-file load path. The sparse coordinate reader assembles CRS directly from
a single triplet buffer sized from the header nnz and sorts it in place,
avoiding the extra full-size copy that the generic coordinate2D::compress()
makes. This roughly halves transient peak memory on very large matrices while
producing identical output (same (row, col) order, same duplicate
accumulation). The buffer is sized from the header nnz, capped at what the
file’s size can hold (an entry line takes at least 6 bytes), or at 2^22 entries
when the size is unknown (gzip, a pipe), so a header that overstates nnz
doesn’t reserve memory that the data can’t fill. In the same way, mm_read_dense
checks that an array file is long enough for its declared values before it
allocates the dense result. Entries are
parsed with std::from_chars rather than string streams, which roughly halves
the load time: 0.52 s to 0.22 s for bbmat (1.77M nnz), and 1.76 s to 0.77 s
for rajat30 (6.18M nnz), on an x86-64 desktop with GCC 15 at -O2.
What is read, and what is rejected. Both readers take coordinate and array
files with a real, integer or complex field (complex needs a
std::complex value type), and general, symmetric, skew-symmetric or
hermitian symmetry. A mirrored file is expanded to the full matrix: an array
file stores the lower triangle by columns, without the diagonal when
skew-symmetric. pattern files are not supported. The readers throw
std::runtime_error on:
- a banner without all five tokens, or with an unknown token;
- a negative, non-integer or overflowing size (for an array file, or a dense
result, that includes
rows * cols); - a malformed or out-of-range entry, including a value with two signs (
+-5) or a hexadecimal float; - a skew-symmetric diagonal entry;
- too few or too many entries;
- a truncated or corrupt gzip stream.
A coordinate file is not bounded by rows * cols: duplicate entries are summed,
so its nnz may exceed the matrix’s capacity, and mm_read takes a sparse
matrix whose rows * cols overflows size_t (#855). Blank lines are skipped,
\r\n endings are accepted, and the nan and inf that the writers emit read
back. mm_write and mm_write_sparse choose the
banner’s field from the value type, complex, integer or real, and write
enough digits for each value to read back exactly.
circuit5M (Freescale/circuit5M, ~5.56M × 5.56M, ~59.5M nnz). The loader is
built to handle this scale, but it is an opt-in large run (multi-GB file and RAM):
the matrix is not downloaded or factored in CI. To validate locally, fetch the
matrix, then load and factor it with native KLU in double, recording wall-clock
load/factor time and peak RSS. Expect the load to be dominated by I/O and the
single triplet buffer (~nnz × 24 bytes) plus the CRS output; factor cost is
governed by the per-block ordering (see the native-KLU notes).
Native Performance & Development
Section titled “Native Performance & Development”These tune or instrument in-tree builds (tests, benchmarks). They are
BUILD_INTERFACE-only and never leak into the installed/exported package.
| Option | Default | Description |
|---|---|---|
MTL5_WITH_HIGHWAY | OFF | Use Google Highway for SIMD-accelerated native kernels (found or fetched at a pinned tag); scalar fallback otherwise |
MTL5_NATIVE_ARCH | OFF | Tune for the host CPU (-march=native); lets the SIMD layer pick the widest ISA (non-portable binaries) |
MTL5_NATIVE_ARCH_FLAGS | -march=native | The flags MTL5_NATIVE_ARCH adds, space-separated. Set it where the compiler under-detects the host: GCC 13 resolves -march=native to baseline Armv8.0-a on the Arduino Ventuno Q (Cortex-A78C + A55), which loses DotProd, so that board’s profiles pass -march=armv8.2-a+dotprod+fp16+rcpc -mtune=cortex-a78c (#806). On AArch64 Linux, configure warns when the flags miss a feature /proc/cpuinfo lists (DotProd, I8MM) |
MTL5_NATIVE_FAST_GEMM | OFF | Route mtl::mult through the native blocked GEMM / SIMD GEMV path instead of the generic loop |
MTL5_INTERLEAVE_ROWS | (empty) | Rows the generic gemv, trmv and forward substitution sum per pass, 1–64, for every type. Empty keeps the per-type default. For float and double it is measured per ISA class: 8 on x86-64, and 4 on AArch64 (#795 removed GCC’s exception of 2 for float trmv and forward substitution). Every target caps it at half its vector registers; any other target, 32-bit x86 included, is assumed to have 8, so it gets 4. Every other type defaults to 4. See Interleave width. Results are bit-identical at any width; bench_interleave_rows reports the fastest. Outside the tree, pass -DMTL5_INTERLEAVE_ROWS=W, or _F32 / _F64 for one type, or specialize mtl::interface::interleave_rows (see include/mtl/interface/interleave_rows.hpp) |
MTL5_INTERLEAVE_RUNTIME | OFF | Choose the interleave width at run time instead (#778). The generic gemv, trmv and forward substitution are compiled at W = 2, 4, 6, 8 and 12 for float and double, and on first use each takes the width a table of the interleave-width measurements lists for this CPU and compiler. A CPU or compiler the table doesn’t list, an x86-64 build with AVX, and any MTL5_INTERLEAVE_ROWS* override or interleave_rows specialization keep the compile-time width. Results are bit-identical whichever width runs. It costs compile time: each file that calls these kernels instantiates five widths. Outside the tree, define MTL5_INTERLEAVE_RUNTIME; mtl::interface::force_interleave_rows(w) fixes the width for tests and benchmarks (see include/mtl/interface/interleave_runtime.hpp) |
MTL5_SANITIZE | (empty) | Comma-separated sanitizers for in-tree builds, e.g. -DMTL5_SANITIZE=address,undefined (GCC/Clang) |
# Native fast dense path, host-tuned (what benchmarks/run_sweeps.sh builds):cmake -B build -DMTL5_NATIVE_FAST_GEMM=ON -DMTL5_WITH_HIGHWAY=ON -DMTL5_NATIVE_ARCH=ON
# ASan + UBSan over the kernels:cmake -B build-asan -DMTL5_SANITIZE=address,undefined -DMTL5_WITH_HIGHWAY=ONMultithreaded GEMM
Section titled “Multithreaded GEMM”The native blocked GEMM parallelizes its row (ic) loop with the C++ standard
concurrency runtime (std::thread) — no OpenMP dependency. It is controlled
at runtime by the MTL5_NUM_THREADS environment variable (clamped to the
hardware concurrency); unset or 1 keeps the single-thread path unchanged.
MTL5_NUM_THREADS=8 ./your_program # use up to 8 threads for native GEMMFor a caller in a real-time loop, define MTL5_POOL_SPIN (the same in every
translation unit). MTL5’s kernels then run on mtl::parallel::spin_pool, which
dispatches and joins without a mutex: its workers spin, then sleep on a futex.
The environment variable MTL5_POOL_SPIN sets how many iterations they spin
(default 4096; 0 sleeps at once). See
On-node threading.
Build Presets
Section titled “Build Presets”The project includes CMake presets for common configurations:
# Development (Debug, tests ON)cmake --preset dev
# Or configure manuallycmake -B build -DCMAKE_CXX_STANDARD=20 -DMTL5_WITH_BLAS=ONcmake --build build -j$(nproc)Running Tests
Section titled “Running Tests”# Run all testsctest --test-dir build
# Run a specific testctest --test-dir build -R test_concepts
# Run with verbose outputctest --test-dir build --output-on-failureRegression tests
Section titled “Regression tests”The unit tests (tests/unit/) check each operation on small inputs, in seconds.
The regression tests (tests/regression/) check the solvers on problems large
enough to expose what small ones hide: error growth with n, ill-conditioning,
fill-in, and iteration counts. Every test case runs at several sizes, from 100
to 50,000 unknowns, and asserts a residual or backward-error bound that scales
with n.
They are off by default. MTL5_BUILD_REGRESSION_TESTS=ON builds them, and the
ci-regression preset turns it on together with a Release build:
cmake --preset ci-regressioncmake --build build-ci-regression --parallelctest --test-dir build-ci-regression -L regression --output-on-failureEach regression test carries the CTest label regression, so one build directory
serves both tiers:
ctest --test-dir build-ci-regression -L regression # regression tests onlyctest --test-dir build-ci-regression -LE regression # unit tests onlyctest --test-dir build-ci-regression # bothA test that fails prints its residual, its bound, and the problem size, so the failing configuration is visible without rerunning it.
What they cover
Section titled “What they cover”There are 14 test programs, one per solver. Sizes in n are unknowns; k × k is a 2-D grid, so n = k².
Dense factorizations (dense/). Each solves Ax = b and asserts the backward
error ‖Ax − b‖ / (‖A‖_F ‖x‖) against a bound that scales with n: n·ε for the
well-conditioned matrices, rising to 1000·n·ε for Moler, 10⁶·n·ε for
Vandermonde, 10⁸·n·ε for Pascal and 10¹⁰·n·ε for Hilbert.
| Test | Matrices | n |
|---|---|---|
test_lu | Frank, Moler, Lehmer; Pascal and Hilbert (ill-conditioned) | 50 to 1000 |
test_cholesky | random SPD (κ = 100), Lehmer, Moler, Minij | 100 to 1000 |
test_ldlt | random SPD (κ = 100), Lehmer, Moler, Minij | 100 to 1000 |
test_ldlt_bk | Bunch–Kaufman: random SPD, Lehmer, Moler, an indefinite tridiagonal | 100 to 1000 |
test_qr | Frank, Kahan, Vandermonde, and a QR solve. Checks ‖QR − A‖/‖A‖ and ‖QᵀQ − I‖ | 100 to 1000 |
test_eigenvalue | Symmetric eigenvalues of a random symmetric and a Wilkinson matrix. Checks that the eigenvalues sum to tr(A) | 100 to 501 |
There is no SVD regression test. The native SVD’s accuracy at these sizes
doesn’t yet meet a bound worth asserting (see the note in test_eigenvalue.cpp).
Sparse direct solvers (sparse/). Each asserts the relative residual
‖Ax − b‖ / ‖b‖ against n·ε, or n²·ε for the 1-D problems.
| Test | Problems | Size |
|---|---|---|
test_sparse_lu | 1-D convection–diffusion (unsymmetric), 2-D Laplacian, 2-D Poisson | n up to 10,000; k up to 100 |
test_sparse_cholesky | 1-D Laplacian, 2-D Laplacian and Poisson with AMD ordering | n up to 50,000; k up to 100 |
test_sparse_ldlt | the same problems as Cholesky | n up to 50,000; k up to 100 |
test_sparse_qr | 1-D and 2-D Laplacian, and a tall-skinny least-squares problem (two stacked 1-D Laplacians, 2n × n) | n up to 5000; k up to 50 |
test_klu_scaling | native KLU on 2-D Poisson at 64², 128², 256² | see below |
test_klu_scaling is a timing test, not an accuracy test. Each refinement
quadruples n. An O(n²) factorization would take about 16 times as long per step,
an O(n^1.5) one about 8 times, and native KLU measures 5 to 7 times. The test
fails if any step’s time ratio exceeds 13, between the two hypotheses, so it
catches an asymptotic regression rather than a slow machine. It takes the best
of 3 runs at each size.
Iterative solvers (itl/). Each asserts a relative residual below 1e−8.
| Test | Problems and preconditioners | Size |
|---|---|---|
test_cg | 2-D Laplacian with identity and Jacobi; 2-D Poisson with IC(0); 1-D Laplacian with Jacobi | k up to 100; n = 1000 |
test_bicgstab | 2-D Laplacian with identity and Jacobi; 1-D convection–diffusion with ILU(0) | k up to 70; n up to 20,000 |
test_gmres | 2-D Laplacian with identity and Jacobi; 1-D convection–diffusion with ILU(0) | k up to 158; n up to 50,000 |
Time limits
Section titled “Time limits”Each regression test has a 300 s limit. test_cg, test_bicgstab and
test_gmres have 600 s, set in tests/regression/CMakeLists.txt. GMRES needs
it: it took 486 s on a Xeon E5-2420 v2, so 300 s made the tier unrunnable on
older machines. The whole tier takes about 7 minutes on a GitHub runner, and
longer locally on slower hardware. Build in Release, since a Debug build is many
times slower.
CI runs the tests in two tiers, both in .github/workflows/ci.yml:
| Tier | Job | Runs on | What it runs |
|---|---|---|---|
| 1 | build | every PR, draft or not; every push to main; release tags | the unit tests on 8 platforms: Linux GCC and Clang on x64 and ARM64, macOS (two Apple Clang versions), Windows MSVC and Clang-CL |
| 1 | lapack, threaded, fp-contract, tsan, highway, benchmarks | the same events, alongside build | unit tests in special configurations: with LAPACK, multithreaded, -march=x86-64-v3, ThreadSanitizer, Highway SIMD, and a benchmark build |
| 2 | regression | non-draft PRs only, once build has passed | the regression tests, on Linux x64 with GCC 13 |
The regression job:
- configures with
cmake --preset ci-regressionand GCC 13, with a compiler cache (sccache) except on pull requests from forks, so a fork can’t write to the shared cache; - builds with
--parallel 2; - runs
ctest -L regression --output-on-failure --timeout 600. The per-test limits above take precedence over--timeout.
Some consequences of when it runs:
- Tier 2 waits for
buildonly, not for the other Tier 1 jobs, so it can start while those are still running. - A draft PR never runs Tier 2. The job shows as skipped. Marking the PR
ready (
gh pr ready <n>) starts a new run that includes it. - Pushes to
mainand release tags run Tier 1 only. Code reachesmainthrough a PR that has already passed Tier 2. - A docs-only change runs neither tier.
ci.ymlignores commits that touch onlydocs/,docs-site/, Markdown, images,LICENSE,.coderabbit.yamlor.nojekyll. - A new commit on a PR cancels the PR’s run in progress, Tier 2 included, so a
cancelled
regressionjob on an older commit is expected.