retroSoC/scripts/README.md

Build and Quality Scripts

This directory contains the Python implementation of setup, dependency locking, build manifests, flow execution, simulation checks, regression orchestration, quality checks, metrics, packaging, and cleanup.

apu_mcasm.py, apu_isa.py, apu_interpreter.py, and apu_primitives.py are the frozen APU microcode V1 assembler/ABI model, P3/P4/P5 target interpreters, and bit-accurate P4 primitive model. They accept only the APU instruction language and deliberately have no C, ELF, RV32, dynamic linking, or runtime code-generation path.

setup_apu_reference.py installs the checksum-pinned libFLAC and official FLAC corpus as host-only P5 verification inputs and builds libFLAC with Ogg disabled below the selected build variant. These inputs never enter firmware, RTL, or shipped APU microcode. qualify_apu_p5_corpus.py enumerates that exact corpus, records source and decoded-PCM hashes, and classifies each file against the frozen P5 profile. run_apu_p5_corpus_rtl.py then runs every classified file through one shared, verification-only apb4_apu fixture compiled for Icarus and Verilator. It records exact production result/accounting and PCM hashes in the same manifest; per-case results are resumable and neither the fixture nor corpus enters a product filelist.

Scripts are part of the build contract. Prefer existing helpers over ad-hoc shell behavior, preserve structured JSON results, and keep setup/download behavior controlled by dependencies/dependencies.lock.json.

setup_npu_reference.py prepares the locked KWS/VWW inputs and the existing TensorFlow/gemmlowp reference sources. npu_framework_reference.py builds the host-only adapter in tests/cpp/npu_framework_reference.cc; its integer kernels consume original tensor constants and quantization, never NPU packed artifacts. Only raw model parsing is shared with the NPU compiler. Run make CONFIG=configs/ci/ihp130.mk npu-p0-qualify after make CONFIG=configs/ci/ihp130.mk setup-npu-reference to compare both complete 1000-input corpora against the Python graph reference and compiled executor. The selected variant's npu/p0/ directory retains oracle build provenance, per-input golden tensors, three-way hashes, logs and qualification-p0.json. The runner uses at most 16 workers (JOBS); missing sources/tools, numerical differences and incomplete runs cannot pass. A direct --limit invocation is debug-only and returns a nonzero status even when its selected cases match.

npu_compiler.py is the P5 production entry point. It accepts one static batch-one INT8 TFLite v3 subgraph, rejects unsupported placement with an operator-specific diagnostic, and emits ABI-1 descriptors, packed constants, npu.json, and a model-prefixed freestanding C plan. The generated plan uses caller-owned aligned descriptor/arena storage, checked physical relocation, bounded HAL waits, 64-byte Zicbom maintenance when compiled for HP, and the model-derived terminal integer Softmax. Generated files remain below the selected build variant. make CONFIG=configs/ci/ihp130.mk npu-p5-deployments builds the locked KWS and VWW packages reproducibly.

qualify_npu_p5.py builds one production NPU fixture per simulator and runs the frozen ten-input sets from both manifests. Every accepted output byte is checked in actual tile-write order, so later arena reuse cannot hide an intermediate mismatch. Invoke it through make CONFIG=configs/ci/ihp130.mk npu-p5-rtl; missing tools, inputs, terminal markers, or any layer write fail the required run.

npu_p5_report.py fails closed while assembling P5 evidence. It verifies P0, compiler/parity and embedded-C/HAL results; exact dual-simulator case counts; source, trace, log and deployment hashes; LP/HP build manifests and firmware; cross-profile KWS package identity; and zero skipped/rejected cases. Invoke it through npu-p5-report with the retained LP/HP variant roots and config digests. The generated report records its fully expanded command.

development_environment.py is the shared Docker, Nix, and manual bootstrap entry point. It installs only the checksum-verified open-source tool bundles and hash-pinned Python dependencies; project-local PDK and source setup remains under the existing Make targets.

publish_fatfs_artifact.sh is the manual release helper for the lock-pinned FatFs R0.16 archive. It verifies the archive, GitHub authentication, release absence, and the published asset checksum before reporting the checksum-pinned URL.

parse_performance_log.py converts the APP=benchmark UART PERF records into meta/performance.json. The parser requires the terminal PERF_BENCHMARK_PASS marker. parse_coremark_log.py converts the fixed SRAM CoreMark quick report into meta/coremark.json; it requires one valid COREMARK_RESULT record and COREMARK_PASS. Both reports complement, rather than replace, the common SIM_TEST_PASS simulation verdict.

generate_vexiiriscv.py verifies the locked VexiiRiscv revision and generates the fixed HP core below build/; generated RTL is never tracked. setup_hp_linux.py installs the locked OpenSBI, Linux, and Buildroot revisions and can resume a checkout left without HEAD by an interrupted fetch. build_hp_linux.py builds the RV32 ilp32d image set with the repo-owned external OpenSBI platform. package_hp_boot.py creates the v1 LP/HP flash bundle and its SHA-256/CRC manifest. check_lp_hp_performance.py applies the measured 2.5x CoreMark/MHz gate.

run_debug_session.py is the local Hazard3 debug acceptance driver. It starts the Verilator remote-bitbang endpoint, the lock-pinned OpenOCD binary, and RISC-V GDB, then records their logs and a structured result. Invoke it through make CONFIG=configs/ci/ihp130-debug.mk SIMU=VERILATOR debug-sim; see ../docs/hazard3-debug.md for its scope and limitations.

program_xpi_flash.py drives the SRAM-resident XPI loader through an existing OpenOCD GDB endpoint. It validates and splits a BIN at 4 KiB sector boundaries, generates a reviewable GDB script, and performs erase/program only when --execute is explicit. See ../docs/ip/xpi.md.

check_rtl_style.py applies the ownership-aware RTL style rules from ../rtl/rtl_style_manifest.json. The CI target checks changed self-owned RTL for positional module connections, legacy constructs, and the staged naming contract. rtl-style-check-all retains the historical full-tree structural baseline while naming debt is migrated in module-sized batches. Existing findings must not be expanded by a new change.

migrate_rtl_connections.py is the conservative migration helper for legacy positional instances. It discovers ANSI-style module declarations in the production RTL and locked technology/IP trees, rewrites only exact positional parameter and port lists to named connections, and reports ambiguous instances without modifying them. Run make rtl-migrate-connections, then make rtl-format rtl-style-check; review the generated diff before committing.

migrate_rtl_names.py shortens only local identifiers beginning with s_ or r_ (plus local automatic variables such as read_request). Public module ports and interface fields are intentionally unchanged. Run make rtl-migrate-names, then format and lint the resulting diff.

Update or add tests in ../tests for script behavior. Run ruff check . and python3 -m pytest -q; build-flow changes also require the relevant dry-run or regression profile from AGENTS.md.