retroSoC/docs/engineering.md

Engineering Workflow

Supported Profiles

Tier	Profile	Automated coverage
Pull request	configs/ci/ihp130-tiny.mk	Tiny firmware, Verilator/Icarus, lint; local Yosys, netlist and OpenSTA
Smoke	configs/ci/ihp130.mk	strict Verilator RTL lint, firmware, Verilator SVA compilation, Icarus assembly self-test
Pull request	configs/ci/ihp130.mk	strict Verilator RTL lint, firmware, Verilator, Icarus
Pull request	configs/ci/ihp130-debug.mk	Verilator remote-bitbang JTAG DTM, Debug Module, OpenOCD, and GDB acceptance
Pull request	configs/ci/gf180.mk	strict Verilator RTL lint, firmware, Verilator, Icarus
Pull request	configs/ci/ics55.mk	strict Verilator RTL lint, firmware, Verilator, Icarus
Pull request	configs/ci/sky130.mk	strict Verilator RTL lint, firmware, Verilator, Icarus
Nightly	configs/ci/ihp130.mk	repeated behavioral IHP130 regression and CoreMark
Cluster	configs/cluster/ics55.mk	compatibility profile for site-specific ICS55 runs

Regression Verilator firmware simulations override the profiles' manual bringup default with APP=ci_smoke. The IHP130 PR simulation also uses the 32 KiB ld2_all_sram layout, the explicit Verilator --fast-flash backend, and an 1800-second wall-clock budget. GF180, ICS55, and SKY130 use the common ld2_sdram execution layout, the same fast-flash backend, and an 1800-second budget for the GA2D acceptance workload while LP instruction traffic shares the SDRAM target. The broad smoke therefore does not execute from the serial PSRAM model. The full serial XPI and PSRAM path remains covered by the Icarus assembly self-test and directed memory-model tests. This application verifies UART output, archinfo APB readback, on-chip SRAM first/last-word and 8/16/32-bit access, Fabric Monitor collection through the PCLK-to-HP APB path, truthful extension capabilities, RNG integration, SDRAM 8/16/32-bit access in a reserved tail scratch area, and test-status completion within the CI time budget; bringup retains the full automatic application-information report for manual runs.

ARCHINFO ABI exposes immutable build, configuration, topology, feature, and technology identity. Its register specification, driver, standalone tests, and formal properties are delivered by the archinfo IP repository. The integration generates matching RTL and C metadata headers from the source revision, dependency lock, and canonical configuration digest. Normal builds clear the release flag; an audited release flow sets BUILD_RELEASE=YES, which is part of the canonical configuration digest.

RNG is a management-only APB entropy-controller integration. Its register specification, portable driver, standalone simulation, synthesis, formal properties, security boundary, and delivery checklist are owned by the RNG IP repository. The SoC currently feeds it from an explicitly unqualified deterministic source for reproducible bring-up and regression. Consequently rs_rng_read_entropy() fails closed with RS_ENOTSUP; only the diagnostic API returns words. Production use requires replacing that source with a PDK-qualified entropy macro and completing the source-specific characterization and NIST SP 800-90B evidence.

The local full flow defines a reproducible OpenSTA core baseline for every CI PDK profile: IHP130 uses slow_1p08V_125C, GF180 uses ss_125C_4v50, ICS55 uses the H7CR ss_1p08_125C view, and SKY130 uses ss_100C_1v40. Enabled SRAM macros use matching slow-corner Liberty views; the SKY130 SRAM view is analytically generated by the locked OpenRAM artifact. The SDC is generated from rtl/mini/integration/clock_reset_domains.json, applies the external, audio, and DVP clocks at their technology-buffer observation pins, and groups unrelated clocks asynchronously. It intentionally excludes package, board, pad-ring, extraction, and PLL timing, so it is a regression-quality core analysis rather than physical signoff.

VCS flows remain cluster-only because public runners do not have the licensed simulator. ICS55 uses the same variant layout, manifests, result JSON, warning scanner, and cleanup rules as the other open-source CI PDKs.

Reproducible Inputs

dependencies/dependencies.lock.json is the source of truth for external Git repositories, downloaded archives, OCI container base images, Nix inputs, and Ubuntu 22.04 toolchain bundles. Git checkouts use full 40-character revisions. Archive downloads are accepted only after SHA-256 verification. The Docker image uses an immutable OCI digest. The flake lock and dependency lock must agree on each Nix input revision and NAR hash. CI actions are pinned to commit IDs.

The development environment guide defines the Docker, Nix, and manual entry points. All three use scripts/development_environment.py to install the same checksum-verified open-source tools. PDK and managed-source setup deliberately remains a checkout operation through make setup or make setup-regression, rather than a container or Nix image layer.

To update a dependency:

1. Update its revision, version, URL, and checksum in the lock.
2. Run python3 scripts/dependency_lock.py.
3. Run the affected setup target with --update directly, then run make doctor.
4. Run python3 -m pytest -q and the affected regression profile.
5. Review the generated manifest, warning delta, metrics, and SBOM before merging.

Do not reuse an old checksum with a new URL or version. The lock digest is part of every variant ID, so changing the lock creates a new output directory without contaminating prior results.

For every downloaded archive, calculate the checksum from a fresh download of the exact URL in the lock; never derive it from an already-installed local tool or a separately cached archive. Before committing a toolchain lock update, install every affected tool into an empty temporary cache and check its reported version. For application archives, run the affected setup script with --update against an empty cache. This is the same checksum and safe-extraction path exercised by CI:

python3 scripts/install_toolchain.py \
  --lock dependencies/dependencies.lock.json --platform ubuntu-22.04 \
  --cache /tmp/retrosoc-toolchain-verify \
  --tool verilator --tool sv2v --tool iverilog \
  --tool yosys --tool opensta --tool riscv_gnu

MISRA Governance

Self-owned crt/ and app/ C/header code follows the MISRA C:2012 Amendment 2 policy. The policy defines its scope, exclusions, Mandatory/Required/Advisory handling, partial automated enforcement, and required deviation process. The executable exclusion list in quality/embedded_c_policy.json remains authoritative.

This repository does not claim complete MISRA certification from its current format, policy, compiler-warning, unit-test, or regression gates. Required-rule deviations must be reviewed and recorded in quality/misra/deviations.md with the affected implementation change.

Build Layout

The variant identifier is the profile name, the local creation timestamp, and a hash over effective hardware/software build settings and the dependency lock. The timestamp has the YYYY-MM-DD-HH-MM format and does not alter the hash. Runtime-only settings such as wave capture and timeout do not alter the hash either. BUILD_TIMESTAMP defaults to the current local minute; set it explicitly when separate Make commands must share one variant. The regression runner exports one timestamp to all of its child Make commands.

build/<variant>/
  generated/mpw/<simulator>/
  generated/archinfo/
  generated/pin_map/
  sw/
  sim/<simulator>/
  formal/<proof>/
  syn/yosys/         (balanced)
  syn/yosys-area/    (SYNTH_RECIPE=area)
  syn/yosys-speed/   (SYNTH_RECIPE=speed)
  sta/opensta/       (balanced)
  sta/opensta-area/  (SYNTH_RECIPE=area)
  sta/opensta-speed/ (SYNTH_RECIPE=speed)
  meta/manifest.json
  meta/warnings.json
  meta/metrics.json          (balanced)
  meta/metrics-area.json     (SYNTH_RECIPE=area)
  meta/metrics-speed.json    (SYNTH_RECIPE=speed)

SYNTH_RECIPE is a flow selector and does not change the configuration hash. The selected recipe determines the synthesis netlist consumed by OpenSTA and netlist simulation, and determines which metrics file is written. balanced is the default full-flow recipe. The nightly IHP130 command set also defines area and speed synth, STA, and metrics stages; hosted CI currently filters them through --behavioral-only, while local execution remains available. Netlist simulation is an explicit, on-demand validation because of its runtime.

Use the same commit, PDK, BUILD_TIMESTAMP, and YOSYS_TARGET_PERIOD_PS for all recipes in a QoR comparison:

make CONFIG=configs/ci/ihp130.mk SYNTH=YOSYS SYNTH_RECIPE=balanced synth
make CONFIG=configs/ci/ihp130.mk SYNTH=YOSYS SYNTH_RECIPE=area synth
make CONFIG=configs/ci/ihp130.mk SYNTH=YOSYS SYNTH_RECIPE=speed synth
make CONFIG=configs/ci/ihp130.mk SYNTH_RECIPE=area STA=OPENSTA sta
make CONFIG=configs/ci/ihp130.mk SYNTH_RECIPE=area metrics

Recipe optimization is evaluated using top area, cell count, WNS/TNS, flow duration, warning signatures, and netlist simulation verdict together. The area recipe is area-first and the speed recipe is timing-first; a single improved metric is not sufficient to replace balanced. Metrics policy remains in observe mode while recipe baselines are collected and reviewed.

Every Yosys run writes yosys-perf.json, <top>_pre_memory.rpt, and <top>_pre_memory.json immediately before memory lowering. These reports are the authoritative pass-timing and inferred-memory inventory for memories that would otherwise be expanded into registers and muxes. Structured flow results also record peak_rss_kib, the operating system's maximum resident-set observation for the child tool. Keep both artifacts when a synthesis run times out or is interrupted; a missing final netlist is never a successful synthesis result. The full and block Yosys flows enforce YOSYS_TIMEOUT=10800 seconds by default. Override it only for an explicitly reviewed diagnostic run; timeout exit 124 remains a failed synthesis verdict.

Generated filelists and MPW output are flow-local. Make depfiles track expanded RTL sources and included headers. The shared MPW generator is protected by a file lock. Default tool parallelism is capped at 16 and can be set with JOBS=<n> or MAX_JOBS=<n>.

Result Policy

Every EDA and simulation flow command is run through scripts/run_flow.py, which streams a log and writes a result JSON containing the command, timestamps, duration, exit code, and status. Automated firmware signals a terminal result by writing SYSCTRL TEST_STATUS at offset 0x84: bit 31 is VALID, bit 0 is PASS, and bits 15:8 are an implementation-defined result code. The first valid full-word write after reset is sticky. The testbench and Verilator harness translate the valid/pass bits into SIM_TEST_PASS or SIM_TEST_FAIL; the Verilator harness additionally prints the result code. A simulation passes only when its command succeeds, the pass marker is present, and no fatal, failure, or timeout marker is present. UART startup text is diagnostic only; it is not a verdict. The local netsim-boot shortcut remains explicitly scoped to stopping Icarus assembly netlist simulation at Hello retroSoC!; CI uses the terminal software status. Icarus regressions use the assembly self-test as retrosoc_asm and cover both the SDRAM and PSRAM mapped windows, while the normal retrosoc_fw image remains available for firmware size tracking and Verilator regressions.

Formal Protocol Proofs

make CONFIG=configs/ci/ihp130.mk formal proves selected protocol invariants with SymbiYosys, Yosys, sv2v, and Bitwuzla. The current targets are bus, rib_adapter, rib2apb, sysctrl, pll_rcu, gpio, ws2812, uart, i2c, timer, clint, dvp, i2s, psram, onchip_ram, opipsram, dma, and sdio; each uses an SBY bounded proof task plus a cover task. The default depth is 20; ws2812 uses depth 120 and a 120-second per-task limit to cover a complete serialized word and reset path. i2c uses depth 80 and a 300-second per-task limit to reach a complete receive path and export its witness. opipsram uses depth 40 for prove, cover, and BMC; prove and cover retain a 300-second limit, while BMC uses 600 seconds to preserve the full bound on variable-speed hosted runners. dma uses a depth-24 BMC task, depth-32 covers, and a 120-second per-task limit for the narrowest supported four-channel, 32-bit/4-beat/4-word dma_core configuration. Its constrained native AXI4 responder and AXI4-Stream environment prove source stability under backpressure, FIFO bounds, transaction-owner stability, abort draining, channel accounting and terminal isolation, and observable AXI burst rules; covers include a complete MM-to-MM burst, an I2S-stream abort, and an AXI read error. sysctrl checks register side effects, sticky terminal-test status, PLL request handling, and fault reporting. pll_rcu checks the clock-switch controller state machine. gpio checks dual-window access isolation, fixed user-GPIO ownership, locks, handoff, open-drain safety, and mux behavior. rib_adapter checks both single-word compatibility adapters under backpressure. ws2812 checks APB4 response stability, FIFO bounds, waveform idle safety, sticky error and interrupt propagation, abort, and underflow preconditions. uart checks FIFO bounds, interrupt composition, DMA gating, and idle/break line safety. i2c checks FIFO bounds, interrupt composition, valid command consumption, open-drain idle safety, and recovery state consistency. Direct Common utility proofs, including spill_register, are maintained in the Common repository. FORMAL_DEPTH and FORMAL_TIMEOUT set the proof bound and per-task wall-clock limit. Use make CONFIG=<profile> formal-doctor before a local proof to verify that the required tools are available.

The I2C and DMA cover tasks store the compact Yosys witness, constraints, and Verilog testbench but disable direct VCD output. Expanding the longer I2C receive or DMA burst covers to VCD produces large artifacts and can exceed the task limit on networked filesystems; the witness remains available for an explicit trace conversion when waveform inspection is required.

The locked Bitwuzla 0.9.1 CLI uses --lang smt2, while the selected Yosys version invokes legacy Bitwuzla options. scripts/bitwuzla_smt2.py is the single compatibility adapter used by SBY; it translates only solver startup arguments and does not modify the generated SMT2 model or proof properties.

Proof environments instantiate the production protocol RTL and the shared ClusterIP interfaces, registers, and utility cells. The generated conversion, SBY configuration, SMT2 model, logs, traces, task status, and structured verdict are stored below build/<variant>/formal/<proof>/ and build/<variant>/meta/formal.json. The IHP130 smoke workflow enables formal-doctor so hosted CI checks that the locked SBY and Bitwuzla tools are present. It does not run make formal: the GA2D cover bound is 199 steps with a 7200s task budget, which does not fit the 60-minute smoke job. Full PDK regressions also skip those PDK-independent protocol proofs. Run make CONFIG=<profile> formal locally or on a longer job when the complete proof suite is required.

The pll_rcu proof uses one formal clock for its system and external-clock inputs to verify the controller protocol independently of analogue clock behavior. It is not asynchronous CDC/RDC signoff; implementation timing, clock-tree checks, and the technology PLL model remain separate signoff work.

make check-warnings normalizes tool messages and compares them with quality/warnings/<profile>/<tool>.json. New signatures and increased counts make the standalone command fail; resolved warnings are reported but do not fail. Regression suites run warning and metric checks as non-blocking observations so their JSON reports are generated and uploaded even when a baseline delta is present. Compilation, simulation, synthesis, and STA failures remain blocking. To refresh one baseline after review:

python3 scripts/analyze_warnings.py baseline \
  --root . --profile ihp130 --tool verilator \
  --log build/<variant>/sim/verilator/verilating.log \
  --output quality/warnings/ihp130/verilator.json

Metrics Promotion

make check-metrics currently uses quality/metrics/policy.json in observe mode. Regression artifacts retain the metrics from every completed regression run. After ten successful main runs for the same profile and locked tools, create the median baseline:

python3 scripts/metrics.py baseline \
  --input run-01/metrics.json --input run-02/metrics.json \
  --input run-03/metrics.json --input run-04/metrics.json \
  --input run-05/metrics.json --input run-06/metrics.json \
  --input run-07/metrics.json --input run-08/metrics.json \
  --input run-09/metrics.json --input run-10/metrics.json \
  --output quality/metrics/baseline.json

Review the medians, commit the baseline, then change policy mode to gate. The committed future limits are 5% firmware growth, 3% top area/cell growth, and 0.05 ns WNS regression. TNS is recorded for diagnosis but is initially non-blocking.

RTL Maturity and Synthesis Intent

rtl/rtl_readiness.json is the machine-readable status record for self-owned RTL. prototype, verified, rtl-freeze, and tapeout-ready describe increasing evidence requirements; the readiness checker validates the status, configuration digest, baseline revision, evidence paths, synthesis intent, and waivers. The checker does not replace the actual flows.

For a freeze or release review, compare the recorded synthesis intent with the Yosys/library reports and OpenSTA result: Common register or inferred register type, reset and enable precedence, RAM/FIFO or macro mapping, pipeline boundary, area/cell count, and WNS/TNS. A change that affects an interface, register map, reset, timing contract, or CDC is not a formatting-only change. Logical-equivalent changes after freeze must retain the baseline revision and equivalence evidence.

CI And Releases

quality.yml validates C, Makefile, and self-owned RTL formatting as well as Python, YAML, GitHub Actions, the dependency lock, and the complete Pytest suite. The test environment restores the locked IHP130 PDK and APU reference corpus plus the NPU models, inputs, and oracle sources, installs the locked third-party simulation models, and installs the locked Verilator, Icarus, sv2v, and Yosys tools before Pytest so required RTL, netlist, timing-model, and corpus fixtures execute. regression-smoke.yml provides fast IHP130 feedback; the four PDK regression workflows remain required PR coverage and do not repeat the format checks.

Self-owned RTL also passes rtl-style-check, which applies the ownership-aware rules in rtl/rtl_style_manifest.json. New positional module connections, legacy always @ blocks, and unjustified explicit nets are rejected in changed self-owned RTL. Formal/DV, PDK, generated, and managed RTL use separate validation profiles and are not mechanically rewritten by the root formatter. Quality CI also runs rtl-readiness-check-all to validate the maturity record. GitHub-hosted regression currently passes --behavioral-only, so it provides behavioral and warning evidence but skips SoC Yosys synthesis, netlist simulation, OpenSTA, and synthesis-recipe metrics. The locked tools are still installed and checked. This temporary policy avoids the unresolved JPEG synthesis-memory peak. Local full regressions retain those stages and remain the source of synthesis and timing evidence until hosted coverage is restored. development-environment.yml independently builds Docker and installs Nix, checks each resulting tool environment, and runs the same hosted IHP130 PR command set. It verifies environment reproducibility but does not expand the behavioral-only evidence boundary. nightly.yml repeats the fixed IHP130 architecture as two parallel behavioral jobs: the PR IHP130 matrix and the extra CoreMark coverage. Source dependencies, locked tool archives, and Verilator ccache use separate cache keys.

Tags matching v* run release.yml. The release contains a flattened SystemVerilog export, a source tarball, build manifest, dependency lock, CycloneDX SBOM, and SHA256SUMS. make package creates the same files locally below dist/<variant>/.