From 991ae606f9aefc8c1185e045739d74febb138248 Mon Sep 17 00:00:00 2001 From: maksyuki Date: Wed, 21 Oct 2026 11:30:42 +0800 Subject: [PATCH] fix(hp): order OpenC906 write bursts behind their pad AW issue The vendored BIU streams a writeback's W data as soon as the victim buffer offers it (AXI-legal W-before-AW), while the AW still waits in awbuf for the serializing 128->64 downsizer to finish the previous write (the ~200-cycle SDRAM write response latency). The Mini HP fabric behind the pad is single-outstanding (downsizer + MMIO demux + target guard, no WID), so the early W beats are absorbed into the previous write's burst; each write's AW then arrives to find its W already consumed by the previous write's stream. The cascade persists until the LSU freezes (its fills block behind the SDRAM write channel, which is parked waiting for the orphaned write's W data), leaving a final orphaned AW whose W beats never come: the target guard parks in WriteStream, the SDRAM core parks in its write-data state and blocks the pending read fill, and the C906 stalls at the next LSU-serializing instruction. Event-level tracing pinned the mechanism: with the guard parked, the C906 write side is fully idle (victim buffer idle, store buffer idle, BIU write-selection FIFO empty, zero W beats presented at the 128-bit port for the wedged write) while the guard and the SDRAM controller both hold the orphaned write's AW. The rtl/mini/ip_overrides/aq_biu_write_channel.v override gates the W stream on the burst's AW having been issued on the pad (pad-issued-AW vs completed-W-burst order counters), restoring the AW-before-W order the fabric requires without touching the vendored checkout. The generator registers the override under the pinned sha256 guard, and the Makefile stamps it as a dependency. Evidence: the deterministic wedge payload and the no-fence control run unbounded (previously wedged at ~22 rounds); hp-smoke-sim and hp-rtthread-sim pass (SIM_TEST_PASS code=0); hp-linux-sim completes userspace acceptance (Run /init as init process, HP_LINUX_INIT_PASS, retroSoC HP Linux ready, HP_LINUX_READY, SIM_TEST_PASS code=0, 1.63 G emulator cycles, status: passed with no missing markers); pytest tests/test_hp_platform.py passes 20/20. docs/hp-rv64-validation.md records the mechanism and the acceptance. --- Makefile | 1 + docs/hp-rv64-validation.md | 68 +- rtl/mini/ip_overrides/aq_biu_write_channel.v | 842 +++++++++++++++++++ scripts/generate_openc906.py | 13 + 4 files changed, 896 insertions(+), 28 deletions(-) create mode 100644 rtl/mini/ip_overrides/aq_biu_write_channel.v diff --git a/Makefile b/Makefile index 15466355..21462830 100644 --- a/Makefile +++ b/Makefile @@ -1124,6 +1124,7 @@ npu-p5-lp-sim: firmware sim ifeq ($(HAVE_HP),YES) $(HP_GENERATED_STAMP): $(ROOT_PATH)/scripts/generate_openc906.py \ $(ROOT_PATH)/rtl/mini/ip_overrides/aq_sysio_kid.v \ + $(ROOT_PATH)/rtl/mini/ip_overrides/aq_biu_write_channel.v \ $(ROOT_PATH)/rtl/mini/ip_overrides/sysmap.h \ $(ROOT_PATH)/rtl/mini/ip_overrides/aq_spsram_1024x16.v \ $(ROOT_PATH)/rtl/mini/ip_overrides/aq_spsram_2048x32.v \ diff --git a/docs/hp-rv64-validation.md b/docs/hp-rv64-validation.md index a4a5542b..4eb1e5db 100644 --- a/docs/hp-rv64-validation.md +++ b/docs/hp-rv64-validation.md @@ -56,34 +56,46 @@ qualification. Current evidence requires, at minimum: then probes the internal PLIC successfully at runtime (`riscv-plic: interrupt-controller@8000000: mapped 31 interrupts`, validating the OpenSBI `fdt_irqchip_init` hook) and completes the initramfs unpack - (`Unpacking initramfs...` / `Freeing initrd memory: 292K`). Userspace - acceptance (`HP_LINUX_READY`) is not yet reached: see the write-wedge - frontier entry below. The target stays unqualified until that acceptance - completes. -- The remaining boot frontier is a write-channel wedge, reproduced - deterministically by a bare-metal payload (dcache-dirty eviction traffic - interleaved with fenced MMIO polls wedges it at ~22 rounds in ~1.5 M HP - cycles) and by the kernel late in boot: the C906 issues a write AW whose W - data never leaves the core (zero W beats presented at the 128-bit port, and - at the stuck point the LSU store-buffer and victim-buffer write-valids and - the BIU write buffers are all empty, so the data vanished inside the core - after its AW was accepted), so the demux write FSM is parked in Active, the - SDRAM core sits in its write-data state waiting for the missing W stream, - the concurrent read fill's R never returns behind it, and the core freezes - at the next fenced MMIO poll (`retrosoc_hp_console_putc`). This is - independent of the fence instructions (the no-fence control payload wedges - identically), of the dcache write mode (write-back, write-through, and - dcache-off all wedge), and of the MAEE/cacheability work above. The - integration manual's write-channel contract (INCR only, len 1/4, 16 - outstanding with one per ID, no write interleave, bufferable responses) is - honored by our fabric; the violation is inside the core's LSU data release, - exposed by the slow-SDRAM window. A guard write timeout does not cover the - source-stalled W stream, so the wedge is permanent. Root-causing it needs - C906 LSU debugging (store-buffer/victim-buffer data release) or an - interconnect that buffers a full write burst before engaging the SDRAM - command engine; both are out of this round's scope. An sdram_axi4-side - gate-the-command-on-full-burst prototype recovered the blocked read but - not the C906's withheld W, so it is not a fix and was not kept. + (`Unpacking initramfs...` / `Freeing initrd memory: 292K`). With the + write-channel fix below applied, the full `hp-linux-sim` run now reaches + userspace acceptance: `Run /init as init process`, busybox init, + `HP_LINUX_INIT_PASS`, `retroSoC HP Linux ready`, `HP_LINUX_READY`, and + `SIM_TEST_PASS code=0` (1.63 G emulator cycles in ~40.5 h wall; the result + checker reports `status: passed` with no missing markers). The wedge + payload and heartbeat probes below document the mechanism and the fix. +- The write-channel wedge is root-caused and fixed. Event-level tracing of + the C906 write channel showed the failure is a systematic AW/W misordering, + not data lost inside the LSU: the BIU streams a writeback's W beats to the + pad as soon as the victim buffer offers them, while the AW still waits in + `awbuf` for the serializing 128->64 downsizer to finish the previous write + (the ~200-cycle SDRAM write response latency). The early W beats are + AXI-legal (W-before-AW), but the Mini HP fabric behind the pad is + single-outstanding (downsizer + MMIO demux + target guard, no WID), so they + are absorbed into the previous write's burst; each write's AW then arrives + to find its W already consumed by the previous write's stream. The cascade + persists until the LSU freezes (its fills block behind the SDRAM write + channel, which is parked waiting for the orphaned write's W data), leaving + a final orphaned AW whose W beats never come: the target guard parks in + WriteStream, the SDRAM core parks in its write-data state and blocks the + pending read fill, and the C906 stalls at the next LSU-serializing + instruction (`retrosoc_hp_console_putc`). Evidence: with the guard parked, + the C906 write side is fully idle (victim buffer idle, store buffer idle, + BIU write-selection FIFO empty, and zero W beats ever presented at the + 128-bit port for the wedged write) while the guard and the SDRAM controller + both hold the orphaned write's AW. The fix is the + `rtl/mini/ip_overrides/aq_biu_write_channel.v` override, which gates the W + stream on the burst's AW having been issued on the pad (pad-issued-AW vs + completed-W-burst order counters), restoring the AW-before-W order the + fabric requires without touching the vendored checkout. Validated: the + deterministic wedge payload runs unbounded (previously wedged at ~22 + rounds/~1.83 M HP cycles, now clean past 10.5 M emulator cycles), the + no-fence control payload is clean, `hp-smoke-sim`/`hp-rtthread-sim` pass on + the committed IHP130 profile, and the full `hp-linux-sim` reaches + `HP_LINUX_READY` with `SIM_TEST_PASS code=0` (see the status entry above). + The earlier note that a guard write timeout cannot recover a source-stalled + W stream remains true and is a separate hardening gap (the guard's + read/write timeouts are gated by source readiness), not the wedge's cause; + it is out of this round's scope. - The Linux boot blocker was a doc/RTL mismatch in OpenC906, root-caused with bare-metal MMU payloads and locked by simulation evidence: the user manual states `mxstatus.MAEE` resets to 0, but the RTL (`aq_cp0_ext_csr.v`) resets diff --git a/rtl/mini/ip_overrides/aq_biu_write_channel.v b/rtl/mini/ip_overrides/aq_biu_write_channel.v new file mode 100644 index 00000000..2fb63232 --- /dev/null +++ b/rtl/mini/ip_overrides/aq_biu_write_channel.v @@ -0,0 +1,842 @@ +// retroSoC controlled override of the vendored OpenC906 aq_biu_write_channel.v. +// +// Source: openc906 dependency (dependencies.lock.json "openc906"), +// C906_RTL_FACTORY/gen_rtl/biu/rtl/aq_biu_write_channel.v +// revision b0c06eb1f8b3bae663bd8b87eac89ff48e68a57f +// License: Apache-2.0 (T-Head Semiconductor) +// +// retroSoC change (Mini HP fabric contract): upstream streams a write's W +// data as soon as the LSU offers it, even while the AW still waits in awbuf +// for a busy downstream slave (AXI-legal W-before-AW). The Mini HP fabric +// behind the pad is single-outstanding (serializing 128->64 downsizer + MMIO +// demux + target guard) and cannot attribute early W beats without a WID, so +// the early beats land in the prior write's burst, the queued AW is orphaned, +// and the write channel wedges. The override gates the W stream on the +// burst's AW having been issued on the pad (retro_aw_issued_q / +// retro_wburst_done_q below). Everything else is byte-identical to upstream. +// Keep in sync when the lock revision changes. + +/*Copyright 2020-2021 T-Head Semiconductor Co., Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// &Depend("cpu_cfig.h"); @17 + +// &ModuleBeg; @19 +module aq_biu_write_channel( + ar_hit_wtable, + araddr, + arcache, + aw_hit_wtable, + awaddr, + awburst, + awcache, + awlen, + awlock, + awprot, + awready, + awsize, + awvalid, + axim_clk_en, + biu_clk, + biu_pad_awaddr, + biu_pad_awburst, + biu_pad_awcache, + biu_pad_awid, + biu_pad_awlen, + biu_pad_awlock, + biu_pad_awprot, + biu_pad_awsize, + biu_pad_awvalid, + biu_pad_bready, + biu_pad_wdata, + biu_pad_wlast, + biu_pad_wstrb, + biu_pad_wvalid, + cp0_biu_icg_en, + cpurst_b, + forever_cpuclk, + pad_biu_awready, + pad_biu_bid, + pad_biu_bresp, + pad_biu_bvalid, + pad_biu_wready, + pad_yy_icg_scan_en, + wdata, + wlast, + wready, + write_channel_clk_en, + wstrb, + wtable_no_op, + wvalid +); + +// &Ports; @20 +input [39 :0] araddr; +input [3 :0] arcache; +input [39 :0] awaddr; +input [1 :0] awburst; +input [3 :0] awcache; +input [1 :0] awlen; +input awlock; +input [2 :0] awprot; +input [2 :0] awsize; +input awvalid; +input axim_clk_en; +input biu_clk; +input cp0_biu_icg_en; +input cpurst_b; +input forever_cpuclk; +input pad_biu_awready; +input [7 :0] pad_biu_bid; +input [1 :0] pad_biu_bresp; +input pad_biu_bvalid; +input pad_biu_wready; +input pad_yy_icg_scan_en; +input [127:0] wdata; +input wlast; +input [15 :0] wstrb; +input wvalid; +output ar_hit_wtable; +output aw_hit_wtable; +output awready; +output [39 :0] biu_pad_awaddr; +output [1 :0] biu_pad_awburst; +output [3 :0] biu_pad_awcache; +output [7 :0] biu_pad_awid; +output [7 :0] biu_pad_awlen; +output biu_pad_awlock; +output [2 :0] biu_pad_awprot; +output [2 :0] biu_pad_awsize; +output biu_pad_awvalid; +output biu_pad_bready; +output [127:0] biu_pad_wdata; +output biu_pad_wlast; +output [15 :0] biu_pad_wstrb; +output biu_pad_wvalid; +output wready; +output write_channel_clk_en; +output wtable_no_op; + +// &Regs; @21 +reg awbuf_vld; +reg bbuf_vld; +reg [3 :0] bid; +reg [39 :0] biu_pad_awaddr; +reg [1 :0] biu_pad_awburst; +reg [3 :0] biu_pad_awcache; +reg [7 :0] biu_pad_awid; +reg [7 :0] biu_pad_awlen; +reg biu_pad_awlock; +reg [2 :0] biu_pad_awprot; +reg [2 :0] biu_pad_awsize; +reg [127:0] biu_pad_wdata; +reg biu_pad_wlast; +reg [15 :0] biu_pad_wstrb; +reg wbuf_vld; +reg [3 :0] wtable_create_ptr; +// retroSoC override: pad-order counters for the write W-after-AW gate. +reg [3 :0] retro_aw_issued_q; +reg [3 :0] retro_wburst_done_q; + +// &Wires; @22 +wire ar_hit_wtable; +wire [15 :0] ar_hit_wtable_addr; +wire [15 :0] ar_hit_wtable_so; +wire [39 :0] araddr; +wire [3 :0] arcache; +wire aw_hit_wtable; +wire [15 :0] aw_hit_wtable_addr; +wire [15 :0] aw_hit_wtable_so; +wire [39 :0] awaddr; +wire awbuf_create_en; +wire awbuf_pop_en; +wire [1 :0] awburst; +wire [3 :0] awcache; +wire awclk; +wire [1 :0] awlen; +wire awlock; +wire [2 :0] awprot; +wire awready; +wire [2 :0] awsize; +wire awvalid; +wire axim_clk_en; +wire bbuf_create_en; +wire bbuf_pop_en; +wire biu_clk; +wire biu_pad_awvalid; +wire biu_pad_bready; +wire biu_pad_wvalid; +wire cp0_biu_icg_en; +wire cpurst_b; +wire forever_cpuclk; +wire pad_biu_awready; +wire [7 :0] pad_biu_bid; +wire pad_biu_bvalid; +wire pad_biu_wready; +wire pad_yy_icg_scan_en; +wire wbuf_create_en; +wire wbuf_pop_en; +wire wclk; +wire [127:0] wdata; +wire wlast; +wire wready; +wire retro_w_ordered; +wire retro_w_burst_end; +wire write_channel_clk_en; +wire [15 :0] wstrb; +wire [9 :0] wtable_create_addr; +wire wtable_create_burst; +wire [15 :0] wtable_create_en; +wire [15 :0] wtable_create_sel; +wire wtable_create_so; +wire [15 :0] wtable_entry_vld; +wire wtable_full; +wire wtable_no_op; +wire [15 :0] wtable_pop_en; +wire wvalid; + + +parameter PADDR = 40; +parameter DEPTH = 16; + +//========================================================== +// address and so check +//========================================================== + +assign ar_hit_wtable = (|ar_hit_wtable_addr[DEPTH-1:0]) | + (|ar_hit_wtable_so[DEPTH-1:0]); + +assign aw_hit_wtable = (|aw_hit_wtable_addr[DEPTH-1:0]) | + (|aw_hit_wtable_so[DEPTH-1:0]); + +//============================================================================== +// AW channel +//============================================================================== +// &Force("bus","awid",3,0); @40 +// &Force("output","awready"); @41 + +assign awready = (!awbuf_vld | pad_biu_awready) & !wtable_full & axim_clk_en; + +assign awbuf_create_en = awvalid & awready; +assign awbuf_pop_en = awbuf_vld & pad_biu_awready & axim_clk_en; + +always @(posedge biu_clk or negedge cpurst_b) +begin + if (!cpurst_b) + awbuf_vld <= 1'b0; + else if (axim_clk_en) begin + if (awbuf_create_en) + awbuf_vld <= 1'b1; + else if (awbuf_pop_en) + awbuf_vld <= 1'b0; + else + awbuf_vld <= awbuf_vld; + end +end + +assign biu_pad_awvalid = awbuf_vld; + +always @(posedge awclk or negedge cpurst_b) +begin + if (!cpurst_b)begin + biu_pad_awaddr[PADDR-1:0] <= {PADDR{1'b0}}; + biu_pad_awsize[2:0] <= 3'b0; + biu_pad_awlen[7:0] <= 8'b0; + biu_pad_awburst[1:0] <= 2'b0; + biu_pad_awcache[3:0] <= 4'b0; + biu_pad_awprot[2:0] <= 3'b0; + biu_pad_awlock <= 1'b0; + biu_pad_awid[7:0] <= 8'b0; + + end + else if (awbuf_create_en & axim_clk_en) begin + biu_pad_awaddr[PADDR-1:0] <= awaddr[PADDR-1:0]; + biu_pad_awsize[2:0] <= awsize[2:0]; + biu_pad_awlen[7:0] <= {6'b0, awlen[1:0]}; + biu_pad_awburst[1:0] <= awburst[1:0]; + biu_pad_awcache[3:0] <= awcache[3:0]; + biu_pad_awprot[2:0] <= awprot[2:0]; + biu_pad_awlock <= awlock; + biu_pad_awid[7:0] <= {4'b0,wtable_create_ptr[3:0]}; + end +end + + +//============================================================================== +// W channel +//============================================================================== +// retroSoC override: hold a write burst's W data until its AW has been issued +// on the pad. The vendored BIU streams W as soon as the LSU offers it while +// the AW can still wait in awbuf for a busy downstream slave; on a +// single-outstanding fabric the early W beats are absorbed into the prior +// write's burst, orphaning this write's AW downstream and wedging the write +// channel. Count pad-issued AWs against completed W bursts: a W burst may +// stream only once its AW is out. In-order AW/W keeps the wrapped comparison +// exact (the counters never drift apart by more than the wsel depth). +assign retro_w_ordered = (retro_aw_issued_q != retro_wburst_done_q); +assign retro_w_burst_end = wvalid & wready & wlast; + +always @(posedge biu_clk or negedge cpurst_b) +begin + if (!cpurst_b) begin + retro_aw_issued_q <= 4'b0; + retro_wburst_done_q <= 4'b0; + end + else if (axim_clk_en) begin + if (awbuf_pop_en) + retro_aw_issued_q <= retro_aw_issued_q + 4'b1; + if (retro_w_burst_end) + retro_wburst_done_q <= retro_wburst_done_q + 4'b1; + end +end + +assign wready = (!wbuf_vld | pad_biu_wready) & axim_clk_en & retro_w_ordered; + +assign wbuf_create_en = wvalid & (!wbuf_vld | pad_biu_wready) & retro_w_ordered; +assign wbuf_pop_en = wbuf_vld & pad_biu_wready & axim_clk_en; + +always @(posedge biu_clk or negedge cpurst_b) +begin + if (!cpurst_b) + wbuf_vld <= 1'b0; + else if (axim_clk_en) begin + if (wbuf_create_en) + wbuf_vld <= 1'b1; + else if (wbuf_pop_en) + wbuf_vld <= 1'b0; + else + wbuf_vld <= wbuf_vld; + end +end + +assign biu_pad_wvalid = wbuf_vld; + +always @(posedge wclk or negedge cpurst_b) +begin + if (!cpurst_b) begin + biu_pad_wstrb[15:0] <= 16'b0; + biu_pad_wdata[127:0] <= 128'b0; + biu_pad_wlast <= 1'b0; + end + else if (wbuf_create_en & axim_clk_en) begin + biu_pad_wstrb[15:0] <= wstrb[15:0]; + biu_pad_wdata[127:0] <= wdata[127:0]; + biu_pad_wlast <= wlast; + end +end + +//============================================================================== +// B channel +//============================================================================== +// &Force("bus", "pad_biu_bid",7,0); @131 +// &Force("input","pad_biu_bresp"); @132 +// &Force("bus","pad_biu_bresp",1,0); @133 + +assign biu_pad_bready = 1'b1; + +assign bbuf_create_en = axim_clk_en & pad_biu_bvalid; +assign bbuf_pop_en = bbuf_vld; + +always @(posedge biu_clk or negedge cpurst_b) +begin + if (!cpurst_b) + bbuf_vld <= 1'b0; + else if (bbuf_create_en) + bbuf_vld <= 1'b1; + else if (bbuf_pop_en) + bbuf_vld <= 1'b0; +end + +always @(posedge biu_clk or negedge cpurst_b) +begin + if (!cpurst_b) + bid[3:0] <= 4'b0; + else if (bbuf_create_en) + bid[3:0] <= pad_biu_bid[3:0]; +end + +assign wtable_pop_en[DEPTH-1:0] = (16'b1 << bid[3:0]) & {DEPTH{bbuf_vld}}; + +//============================================================================== +// write table +//============================================================================== + +always @(posedge awclk or negedge cpurst_b) +begin + if (!cpurst_b) + wtable_create_ptr[3:0] <= 4'b0; + else if (awbuf_create_en) + wtable_create_ptr[3:0] <= wtable_create_ptr[3:0] + 4'b1; +end + +assign wtable_full = wtable_entry_vld[wtable_create_ptr[3:0]]; + +assign wtable_create_sel[DEPTH-1:0] = 16'b1 << wtable_create_ptr[3:0] ; +assign wtable_create_en[DEPTH-1:0] = wtable_create_sel[DEPTH-1:0] & {16{awbuf_create_en}}; +assign wtable_create_addr[9:0] = awaddr[13:4]; +assign wtable_create_so = !awcache[1]; +assign wtable_create_burst = awlen[1:0] == 2'b11; +//no_op +assign wtable_no_op = ~(|wtable_entry_vld[DEPTH-1:0]); + +//============================================================================== +// ICG +//============================================================================== +assign write_channel_clk_en = bbuf_create_en | bbuf_pop_en | + wbuf_create_en | wbuf_pop_en | + awbuf_create_en | awbuf_pop_en; + +// &Instance("gated_clk_cell", "x_aq_biu_aw_gated_clk"); @189 +gated_clk_cell x_aq_biu_aw_gated_clk ( + .clk_in (forever_cpuclk ), + .clk_out (awclk ), + .external_en (1'b0 ), + .global_en (1'b1 ), + .local_en (awbuf_create_en ), + .module_en (cp0_biu_icg_en ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en) +); + +// &Connect(.clk_in (forever_cpuclk), @190 +// .external_en (1'b0), @191 +// .global_en (1'b1), @192 +// .module_en (cp0_biu_icg_en), @193 +// .local_en (awbuf_create_en), @194 +// .clk_out (awclk)); @195 + +// &Instance("gated_clk_cell", "x_aq_biu_w_gated_clk"); @197 +gated_clk_cell x_aq_biu_w_gated_clk ( + .clk_in (forever_cpuclk ), + .clk_out (wclk ), + .external_en (1'b0 ), + .global_en (1'b1 ), + .local_en (wbuf_create_en ), + .module_en (cp0_biu_icg_en ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en) +); + +// &Connect(.clk_in (forever_cpuclk), @198 +// .external_en (1'b0), @199 +// .global_en (1'b1), @200 +// .module_en (cp0_biu_icg_en), @201 +// .local_en (wbuf_create_en), @202 +// .clk_out (wclk)); @203 +//============================================================================== +// write table Instance +//============================================================================== +// &ConnRule(s/_x$/[0]/); @207 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_0"); @208 +aq_biu_wt_entry x_aq_biu_wt_entry_0 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[0]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[0] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[0]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[0] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[0] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[0] ), + .wtable_pop_en_x (wtable_pop_en[0] ) +); + + +// &ConnRule(s/_x$/[1]/); @210 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_1"); @211 +aq_biu_wt_entry x_aq_biu_wt_entry_1 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[1]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[1] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[1]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[1] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[1] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[1] ), + .wtable_pop_en_x (wtable_pop_en[1] ) +); + + +// &ConnRule(s/_x$/[2]/); @213 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_2"); @214 +aq_biu_wt_entry x_aq_biu_wt_entry_2 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[2]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[2] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[2]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[2] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[2] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[2] ), + .wtable_pop_en_x (wtable_pop_en[2] ) +); + + +// &ConnRule(s/_x$/[3]/); @216 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_3"); @217 +aq_biu_wt_entry x_aq_biu_wt_entry_3 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[3]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[3] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[3]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[3] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[3] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[3] ), + .wtable_pop_en_x (wtable_pop_en[3] ) +); + + +// &ConnRule(s/_x$/[4]/); @219 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_4"); @220 +aq_biu_wt_entry x_aq_biu_wt_entry_4 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[4]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[4] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[4]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[4] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[4] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[4] ), + .wtable_pop_en_x (wtable_pop_en[4] ) +); + + +// &ConnRule(s/_x$/[5]/); @222 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_5"); @223 +aq_biu_wt_entry x_aq_biu_wt_entry_5 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[5]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[5] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[5]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[5] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[5] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[5] ), + .wtable_pop_en_x (wtable_pop_en[5] ) +); + + +// &ConnRule(s/_x$/[6]/); @225 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_6"); @226 +aq_biu_wt_entry x_aq_biu_wt_entry_6 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[6]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[6] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[6]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[6] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[6] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[6] ), + .wtable_pop_en_x (wtable_pop_en[6] ) +); + + +// &ConnRule(s/_x$/[7]/); @228 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_7"); @229 +aq_biu_wt_entry x_aq_biu_wt_entry_7 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[7]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[7] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[7]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[7] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[7] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[7] ), + .wtable_pop_en_x (wtable_pop_en[7] ) +); + + +// &ConnRule(s/_x$/[8]/); @231 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_8"); @232 +aq_biu_wt_entry x_aq_biu_wt_entry_8 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[8]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[8] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[8]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[8] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[8] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[8] ), + .wtable_pop_en_x (wtable_pop_en[8] ) +); + + +// &ConnRule(s/_x$/[9]/); @234 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_9"); @235 +aq_biu_wt_entry x_aq_biu_wt_entry_9 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[9]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[9] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[9]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[9] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[9] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[9] ), + .wtable_pop_en_x (wtable_pop_en[9] ) +); + + +// &ConnRule(s/_x$/[10]/); @237 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_10"); @238 +aq_biu_wt_entry x_aq_biu_wt_entry_10 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[10]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[10] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[10]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[10] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[10] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[10] ), + .wtable_pop_en_x (wtable_pop_en[10] ) +); + + +// &ConnRule(s/_x$/[11]/); @240 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_11"); @241 +aq_biu_wt_entry x_aq_biu_wt_entry_11 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[11]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[11] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[11]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[11] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[11] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[11] ), + .wtable_pop_en_x (wtable_pop_en[11] ) +); + + +// &ConnRule(s/_x$/[12]/); @243 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_12"); @244 +aq_biu_wt_entry x_aq_biu_wt_entry_12 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[12]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[12] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[12]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[12] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[12] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[12] ), + .wtable_pop_en_x (wtable_pop_en[12] ) +); + + +// &ConnRule(s/_x$/[13]/); @246 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_13"); @247 +aq_biu_wt_entry x_aq_biu_wt_entry_13 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[13]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[13] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[13]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[13] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[13] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[13] ), + .wtable_pop_en_x (wtable_pop_en[13] ) +); + + +// &ConnRule(s/_x$/[14]/); @249 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_14"); @250 +aq_biu_wt_entry x_aq_biu_wt_entry_14 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[14]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[14] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[14]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[14] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[14] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[14] ), + .wtable_pop_en_x (wtable_pop_en[14] ) +); + + +// &ConnRule(s/_x$/[15]/); @252 +// &Instance("aq_biu_wt_entry","x_aq_biu_wt_entry_15"); @253 +aq_biu_wt_entry x_aq_biu_wt_entry_15 ( + .ar_hit_wtable_addr_x (ar_hit_wtable_addr[15]), + .ar_hit_wtable_so_x (ar_hit_wtable_so[15] ), + .araddr (araddr ), + .arcache (arcache ), + .aw_hit_wtable_addr_x (aw_hit_wtable_addr[15]), + .aw_hit_wtable_so_x (aw_hit_wtable_so[15] ), + .awaddr (awaddr ), + .awcache (awcache ), + .awlen (awlen ), + .cp0_biu_icg_en (cp0_biu_icg_en ), + .cpurst_b (cpurst_b ), + .forever_cpuclk (forever_cpuclk ), + .pad_yy_icg_scan_en (pad_yy_icg_scan_en ), + .wtable_create_addr (wtable_create_addr ), + .wtable_create_burst (wtable_create_burst ), + .wtable_create_en_x (wtable_create_en[15] ), + .wtable_create_so (wtable_create_so ), + .wtable_entry_vld_x (wtable_entry_vld[15] ), + .wtable_pop_en_x (wtable_pop_en[15] ) +); + + +// &ModuleEnd; @255 +endmodule + + diff --git a/scripts/generate_openc906.py b/scripts/generate_openc906.py index 87736308..aa8b4306 100644 --- a/scripts/generate_openc906.py +++ b/scripts/generate_openc906.py @@ -10,6 +10,15 @@ for their vendored counterparts: - ``aq_sysio_kid.v``: upstream hardwires mhartid to 0; the Mini LP/HP contract requires the HP core to report hart 1. +- ``aq_biu_write_channel.v``: the vendored BIU streams a write's W data as + soon as the LSU offers it, even while the AW still waits in ``awbuf`` for a + busy downstream slave (AXI-legal W-before-AW). The Mini HP fabric behind the + pad is single-outstanding (serializing 128->64 downsizer + MMIO demux + + target guard) and cannot attribute early W beats without a WID, so the + beats land in the prior write's burst, the queued AW is orphaned, and the + channel wedges (observed as a permanent guard WriteStream stall with the + SDRAM controller parked waiting for write data). The override gates the W + stream on the burst's AW having been issued on the pad. - ``sysmap.h``: the upstream default region table marks the Mini peripheral window cacheable/bufferable; the override maps the Mini address map with the MMIO window (including the core-internal CLINT/PLIC window) as @@ -64,6 +73,10 @@ OVERRIDES = { "sysmap.h", "0389089a42ef762768e57bbb4fdcbe6b95f33ab650b5bacc0de8122e0d8711d5", ), + "C906_RTL_FACTORY/gen_rtl/biu/rtl/aq_biu_write_channel.v": ( + "aq_biu_write_channel.v", + "7d98d231287f4a8807593990411d96879deb68cfda764c2fb92bc93c2b10e204", + ), "C906_RTL_FACTORY/gen_rtl/ifu/rtl/aq_spsram_1024x16.v": ( "aq_spsram_1024x16.v", "8969cdfd73933ff4ca105d562e31283e16433f90e4ceaa4339759e6df7ba34d9",