Native CCSD occupied-pair ladder (#806)¶
The first CCSD migration is the real FP64 occupied-pair ladder in the native closed-shell solver:
R2[i,j,a,b] += sum(m,n) W[m,n,i,j] tau[m,n,a,b]
The method still builds W and tau and owns every other residual term, denominator, spin coefficient, pair symmetrization, DIIS operation and energy reduction. The equations are the existing closed-shell spin integration of Stanton et al., JCP 94, 4334 (1991), equations (2), (6) and (10). The alpha-beta spatial convention already used in the method has unit coefficient for this ladder. It is not replaced by the spin-orbital one-half coefficient. DF integral construction remains the existing DePrince and Sherrill route.
Binding and execution¶
OccupiedLadderPlan uses the existing View, SourceStamp, IndexProgram,
Budget, BorrowScope and Lease contracts. The checked IR is
mnij,mnab -> ijab, with all six orbital labels explicit. A flat pair extent
is insufficient: each pair has ordered occupied or virtual axes and the
actual reference frame, frozen offset and space identity. The method binds
the assembled integral owner and orthonormal MO frame, and revises the
iteration stamp when amplitudes change. W and tau receive independent value
epochs and read leases. The residual requires a writable, nonaliasing owner.
Stale values, sources, domains, expired owners and conflicting leases refuse.
The three native matrices are column-major. W has rows m*nocc+n and
columns i*nocc+j; tau and R2 have virtual columns a*nvir+b. The registered
cpu.eigen.occupied-ladder.fp64.v1 path lowers W-transpose times tau directly
into R2 with beta one. It reuses the supplied-pack Eigen GEMM boundary from
the transform slice, in one synchronous worker, with no second residual
array and no new four-index intermediate. The bounded internal packs are at
most 64 by 64 each, plus alignment. Ordered mode sums ascending flattened
mn; the method explicitly selects reassociated FP64. There is no symmetry
shortcut, complex conjugation, reduced precision, online tuning or mutation
of global thread settings.
Only the full native closed-shell CCSD iteration enters this operation. The local/target-pair residual APIs, other CC variants, UCCSD and triples retain their existing implementations. Their broader migration remains open.
Admission boundary¶
CCSDOptions.ccsd_contraction_memory_bytes caps the tracked storage live at
this operation. Zero retains the uncapped native default. It is separate
from requested_memory_bytes, whose existing triples-phase meaning is
unchanged. This is not a whole-calculation or whole-iteration cap: integral
preparation, earlier/later residual construction and DIIS extrapolation can
have different peaks. A refusal may occur after those earlier phases.
The method imports the reference coefficients, Fock/energy owners, integral blocks and retained DF factors, amplitudes and residuals, every residual intermediate live at the ladder, and the existing DIIS history. Numeric owners are counted once; W, tau and R2 borrows spend credits in that imported inventory. IR, view and lease metadata and actual pack allocation are then charged by the engine. DIIS maps/partly filled blocks have a conservative 8 KiB allowance per maximum history slot, plus two slots; numeric history vectors are counted separately by actual size. This is a modeled storage bound, not process RSS or allocator bookkeeping telemetry.
All kernel admission, leases and pack allocations precede the first ladder
write. The output is already the method-owned residual; failure at this
boundary leaves it unchanged by the ladder. Earlier residual terms may have
written it. ccsd_contraction_peak_bytes and ccsd_contraction_calls report
the realized operation-phase inventory and call count. The new finite cap
refuses on unsupported closed-shell variants, UCCSD and the Python A-CCSD(T)
route instead of silently applying it to a different implementation.
Focused checks¶
The existing classified tensor test entry compiles the new native fixture. Six checks cover the transform and MP2 regressions plus the ladder. The ladder uses asymmetric W/tau and nonzero initial residuals, occupied/virtual sizes (2,5), (5,2), (9,11), (3,3), (1,7) and empty spaces, in ordered and reassociated modes. Every output is compared with an independent explicit mn sum at 2e-11 tolerance. Controls exercise exact/one-byte-below caps, private input/output alias rejection, stale value/source/domain revisions, expired scopes, output writer conflicts, missing storage, overflow and injected pack allocation failure. The numeric leaf runs with Eigen allocation disabled even in a BLAS build. ASan/UBSan also pass.
A source-linked full-residual control compares the existing path and the migrated path on arbitrary asymmetric integral blocks and fixed amplitudes at (nocc,nvir)=(2,5),(4,3),(1,7). Maximum R1/R2 difference is 4.45e-16; exactly one migrated call is observed. It also checks the UCCSD cap refusal.
Fixed-input performance¶
benchmarks/tensor_ccsd_ladder.cpp compares the actual predecessor
R2.noalias() += W.transpose()*tau, explicit transpose-pack plus GEMM,
the checked path and the raw supplied-pack leaf. It uses 21 rotated samples
with fixed inputs and resets R2 outside timing. The arm64 runs used one
OpenMP/BLAS thread and Eigen 5.0.1, separately compiled with and without
Accelerate dispatch. Median times are microseconds; raw execution excludes
plan, leases and pack allocation.
Occupied, virtual |
Eigen old |
Accelerate old |
Checked |
Raw |
Packed baseline |
Explicit packing |
Pack bytes |
Tracked benchmark peak |
|---|---|---|---|---|---|---|---|---|
2, 2 |
0.042 |
0.042 |
0.292 |
0.125 |
0.167 |
0.042 |
288 |
6,488 |
4, 19 |
8.71 |
5.00 |
11.54 |
11.33 |
5.08 |
0.25 |
10,272 |
109,440 |
8, 31 |
356.00 |
135.13 |
385.75 |
390.46 |
144.04 |
5.38 |
65,568 |
1,087,104 |
12, 40 |
3,580.46 |
1,121.25 |
4,547.50 |
4,745.79 |
1,075.17 |
24.63 |
65,568 |
3,922,560 |
First planning cost was 1.33-4.96 us and repeated planning 0.63-6.54 us in the BLAS profile. Peak bytes include both resident inputs and the complete resident residual; the smallest benchmark’s peak includes a simultaneously live timing-only second plan. Kernel packing is included in raw execution; there is no output conversion. Each execution visits each of two affine input bindings once logically; internal matrix blocks are reread/packed. Maximum fixed-input difference is 1.96e-14. The explicit-workspace kernel is slower than the vendor baseline. Vendor workspace remains unqualified for strict admission, and no speedup is claimed.
Complete calculation evidence¶
The source/runtime identities, vq receipts and residual traces are recorded on #806. Complete controls use neutral singlet H4/STO-3G without frozen orbitals and water/def2-SVP with one frozen occupied orbital. DF cases use def2-SVP-RIFIT; no case includes triples. H4 coordinates in bohr are (0,0,0), (0,0,1.43), (0.2,3.8,0.15), (0.4,3.85,1.70). Water has O at the origin and H at (0,+/-1.43,1.11) bohr. Native RHF energy/gradient thresholds are 1e-13/1e-10, and CCSD energy/residual thresholds are 1e-12/1e-10. ORCA 6.1.1 uses VeryTightSCF and MDCI STol=1e-10, with both TrafoType=trafo_ri and KCOpt=KC_RI for fully fitted integrals; the conventional control uses trafo_full and KC_MO. Native and ORCA DIIS/damping procedures differ, so iteration counts are compared only against the same-reference predecessor.
The preset limits are 2e-11 for the synthetic ladder/full residual, 2e-10 for same-reference amplitudes and residual traces, 1e-10 Eh for same-reference energy, and 2e-6 Eh for matched ORCA HF, CCSD correlation and total. A containing native executable is required; an older installed Python native module supplies only the upstream converged HF reference. Packaged Python extension qualification is separate from the source and executable controls.
The complete DF H4 and frozen-core water calls reproduce predecessor energies, T1 amplitudes, T1/T2 norms and every recorded residual norm within those limits. The conventional H4 control also passes. Measured energies in Eh:
Case |
Native HF |
Native correlation |
Native total |
ORCA correlation |
ORCA total |
|---|---|---|---|---|---|
H4, DF |
-2.209069888186 |
-0.047046741015 |
-2.256116629201 |
-0.047046741 |
-2.256116629340 |
H4, conventional |
-2.209069888186 |
-0.047058666810 |
-2.256128554996 |
-0.047058667 |
-2.256128554990 |
Water, DF, frozen core |
-75.960973265765 |
-0.210839668208 |
-76.171812933974 |
-0.210839518 |
-76.171812783900 |
ORCA HF energies are -2.209069888184 and -75.960973265673 Eh. All HF, correlation and total differences pass the preset 2e-6 Eh limit; the largest is 1.51e-7 Eh. This is evidence for the listed fixtures, not general CCSD acceptance. H4 DF/conventional tracked phase peaks are 80,504/76,920 bytes, and water DF is 2,625,168 bytes. The complete calls converge at their exact measured peak and refuse one byte below it. H4 uses 17 ladder calls; water uses 52. Forcing the existing blocked virtual-integral route in water gives the same energies, amplitude norms and residual trace. Its tracked ladder peak is 1,582,600 bytes, exactly 1,042,568 bytes lower: the released 361 by 361 virtual integral array. Exact-cap success and one-byte-below refusal also pass in that route.
Five-repeat complete native CCSD calls include integral/factor preparation and the whole solve, on fixed upstream HF inputs. The initial old/new medians were 234.77/438.64 ms for DF H4 and 22,719.88/4,050.53 ms for water. A second H4 cohort was 1,378.77/531.55 ms; conventional H4 was 2.90/1.65 ms and blocked-integral water was 28,824.92/14,500.26 ms. These non-interleaved shared-machine measurements show severe scheduling variability and cannot establish a complete-method speedup or regression. The rotated fixed-input kernel comparison above is the useful performance control; its bounded path loses to vendor GEMM.
The migration does not qualify other residual contractions, UCCSD, local CC, triples, precision variants, a global CC memory limit or faster bounded BLAS. The parent remains open.
Virtual-pair ladder¶
The next full closed-shell CCSD migration accumulates
R2[ij,ab] += sum_ef tau[ij,ef] W[ab,ef]. The method still builds
M[mn,ef]=(me|nf), initializes W from the (ae|bf) panel and subtracts its
two singles terms. The spatial W update has coefficient one half, and
the final product has coefficient one. These are the existing spatial
spin integration of Stanton et al., Eqs. (2), (7) and (10); there is no new
symmetry, precision, approximation or changed orbital selection.
The column-major tau/residual pair pitch is nocc squared. W has rows
local_a*nvir+b and columns e*nvir+f. A checked four-axis view must
bind a to the indicated subrange of the same virtual space, with the
same parent frame, coordinate and step. The plan checks the pitch and
order of every axis, input epochs, source stamp and exclusive residual
access; it derives the writable window from the full residual and offset.
The resident call spans all a. The blocked call regenerates (ae|bf) from
the original row-major B_vv into one panel and W tile, then releases them.
Row limits still apply; for nvir > 1 a blocked tile covers at most half
the virtuals (rounded down, with a one-orbital floor). No full W is needed
in this route. Target-only, PNO/extended-domain and skipped-ladder calls
retain their original behavior.
The existing phase cap imports the same retained inventory as the occupied ladder, reserves M before its construction, and reserves W and, if needed, one regenerated panel before either payload is allocated. A resident integral panel is a borrow of the already-counted integral block. The two workspace- sensitive preparation products use the existing sequential supplied-pack Eigen leaf with the original factor order and scale. Preparation scratch is released before planning/executing the final checked product. OpenMP fill loops retain their original ownership and have no per-worker matrix copy. The packed GEMMs own one worker; no thread settings are changed. Ordered final execution reduces ascending e,f, while production permits reassociated FP64. No unknown vendor workspace is charged as zero.
The method cap starts with the tracked storage live at these ladder phases; it does not move the integral, other residual, DIIS or triples admission boundaries. Checked metadata, full tau/residual and packs are included. M and the current tile reservations are independent of the number of tiles. A budget or allocation failure can occur before a private preparation buffer is created or before a tile’s first residual write. Earlier tiles may already have written columns. The method propagates failure and discards the entire iteration; no energy or partial residual is published and there is no undo copy. The phase peak spans preparation and execution; calls count completed occupied and virtual products, excluding W preparation.
Entrywise tests use asymmetric W/tau, nonzero initial R2, nonzero offsets, short final tiles, unequal/equal and empty domains, both numerical modes, source/value/domain mismatches, swapped axes, overlapping buffers, expired and busy leases, overflow, allocation failure and exact/below pack budgets. The two private preparation products are separately compared with explicit sums, with Eigen numeric allocation disabled. At arbitrary fixed T1/T2 and (nocc,nvir)=(2,5),(4,3),(1,7),(4,19), every full R1/R2 entry agrees with the predecessor within 3.56e-15 (limit 2e-11). Regenerated and resident panels agree; a skipped ladder does not consult B_vv, and a missing required factor source fails. The six focused regression cases and BLAS-profile ASan/UBSan check pass.
The following single-thread arm64/Accelerate fixed-input medians are in microseconds. The benchmark rotates 21 samples and resets the same residual outside the clock. Checked execution includes leases, pack allocation and GEMM; raw uses preallocated packs. The packed baseline includes explicit transpose time. Tracked peak covers the final contraction’s resident W, tau, full R2, metadata and packs, excluding benchmark comparison buffers.
nocc,nvir; a begin,count |
Cold/warm plan |
Old GEMM |
Checked |
Raw |
Packed (transpose) |
Pack bytes / tracked peak |
|---|---|---|---|---|---|---|
2,2; 0,2 |
67.08/1.46 |
0.08 |
0.38 |
0.13 |
0.25 (0.08) |
288 / 6,560 |
4,19; 0,19 |
8.17/1.46 |
85.17 |
362.92 |
364.50 |
284.58 (140.79) |
40,992 / 1,181,864 |
4,19; 4,4 |
19.71/1.25 |
18.08 |
77.63 |
76.50 |
46.33 (21.04) |
40,992 / 358,784 |
4,19; 16,3 |
9.88/1.21 |
14.50 |
56.79 |
56.50 |
36.17 (16.92) |
37,408 / 300,328 |
8,31; 7,7 |
13.21/1.25 |
630.04 |
2,203.67 |
2,218.04 |
1,452.38 (661.42) |
65,568 / 2,723,816 |
At (4,19; 4,4), isolated old/new panel regeneration took 98.46/328.21 us, and the half-weight W update 47.50/76.58 us. The largest difference across all timed kernel/preparation comparisons was 8.71e-14. The bounded kernel loses to vendor GEMM; no speedup is claimed. There is no output conversion.
A separate source-linked profile times the actual predecessor (including its checked occupied ladder) and new full residual on the same asymmetric 4-occupied, 19-virtual inputs, 76 auxiliary functions, one thread. In the blocked case five a tiles have counts 4,4,4,4,3. Values are medians of seven calls in microseconds; M, W fill/update and panel regeneration times are sums over each call. The W timer excludes allocation; the full timer includes it. These CPU timings are distinct from complete physical CCSD runs.
Storage |
M old/new |
W fill old/new |
Regenerate old/new |
Full residual old/new |
|---|---|---|---|---|
Resident |
7.00/6.83 |
810.33/802.38 |
0/0 |
5,872.92/6,260.46 |
Blocked |
6.75/7.04 |
680.13/801.96 |
293.13/958.04 |
5,810.96/7,911.96 |
Complete native CCSD through vq used the same RHF, orbital/auxiliary basis, spin, frozen core, thresholds, geometries and matched ORCA controls listed above. The affected executable was rebuilt; the installed native Python module supplied only the converged RHF input. Source/binary identities and receipts are on #806, separately from packaged extension delivery. Both implementations used the same reference in each pair. The comparison included HF, correlation and total energy, T1 entries, T1/T2 norms, iteration counts, and each energy and R1/R2 norm in the residual trace. The largest T1-entry and trace differences were 1.13e-17 and 7.12e-16, respectively. Converged HF/correlation/total values match the native table above to the shown precision. Maximum absolute difference from ORCA is 1.51e-7 Eh (preset 2e-6 Eh). All four calculations converge at the exact measured peak and refuse at one byte below it.
Complete calculation |
Iterations / ladder products |
Current phase peak (bytes) |
CCSD old/new (ms) |
|---|---|---|---|
DF H4 |
17 / 34 |
81,784 |
10.77/146.51 |
Conventional H4 |
17 / 34 |
78,200 |
1.72/1.03 |
Frozen-core DF water, resident |
52 / 104 |
3,745,144 |
3,531.01/2,717.82 |
Frozen-core DF water, five tiles |
52 / 312 |
2,118,752 |
3,222.42/4,853.60 |
The new peaks include virtual preparation/execution; historical occupied-only peak numbers above describe a narrower phase. The four-orbital row window uses 4,4,4,4,3 a values in blocked water. Complete timings are medians of three calls on fixed RHF inputs, including CCSD preparation and solve. Their variability and the blocked regression preclude an overall speedup claim. Other residual terms, local CC, UCCSD, triples, whole-method admission and qualification of a faster bounded backend remain open.
Same-spin W2 particle-hole ring¶
Only the following term of the full native closed-shell FP64 doubles equation
moves into checked execution:
half[i,j,a,b] += sum(m,e) T2[i,m,a,e] W2[m,e,j,b].
The method still forms W2 from the same-spin spatial block of the Stanton
et al. intermediate (p. 4335, Eqs. (2), (8), permutation convention (11)).
Neither W1’s direct-minus-exchanged amplitudes nor WX’s different occupied
index are folded into W2. The two singles products stay in the method loop.
The method still adds half(i,j,a,b) and half(j,i,b,a) to R2 exactly once.
No new spin factor, orbital approximation or symmetry of either input is
assumed. The isolated W2 sum is now added after the other half terms, a
declared FP64 reassociation, not a promise of bitwise iteration histories.
W2RingPlan binds the actual occupied/virtual domains, frame, source stamp
and T2/W2 value epochs. IndexProgram uses imae,mejb -> ijab; the T2 and
half matrices have physical OO rows and VV columns, W2 has OV rows and OV
columns. Equal flattened extents cannot replace the axis/stride checks.
Only a nonaliasing, read-write half owner is allowed. The registered leaf
cpu.eigen.ccsd-ring-w2.fp64.v1 in reassociated mode
gathers at most 32 (i,a) by 64 (m,e) entries, contracts against a pitched
W2 column panel in a single worker with supplied Eigen packs, and scatters
at most 32 by 64 values to the corresponding half entries. Ordered execution
sums ascending m,e without packs. Both policies accumulate with unit weight.
The operation-phase budget imports the retained CC inventory used by the ladders, including all three live ring intermediates. It reserves the existing full half residual before allocation. The virtual ladder’s private M/panel/W and packs are already dead. T2, W2 and half consume imported owner credits once. Views, checked IR, synchronous leases and all gather, result and kernel packs enter the same ledger. The tested build uses at most 81,952 scratch bytes for the reassociated kernel, independent of OV size; the ordered path uses zero numeric scratch. Admission, input epoch/lease checks and scratch allocation precede the first W2 write; no rollback copy is made. On failure the solver publishes no incomplete iteration. This is still a tracked contraction-phase limit, not a cap on W construction, the entire residual, DIIS, triples, external runtime overhead or RSS.
The target-only and local/extended-domain entry points still use their original fused scalar expression. Skipping ring evaluation still skips W2 even when the caller requests an export. Source-linked controls check target/skip behavior including intermediate exports. On arbitrary amplitudes at (nocc,nvir)=(2,5),(4,3),(1,7),(4,19), the maximum difference in any full R1/R2 entry versus the original scalar loop is 3.56e-15 (limit 2e-11). A separate full-residual control at (4,19) suppresses the virtual ladder: the measured peak is then the ring’s 1,875,882 bytes including that fixture’s retained state. It passes exactly there and refuses one byte below, after the occupied ladder and before the ring writes. Seven focused native cases pass, including the transform, MP2 and previous ladders. The isolated ring oracle uses independent asymmetric FP64 inputs, nonzero initial output, unequal/equal/empty spaces and short final tiles. Both numerical modes, wrong revisions/domains/strides, overflow, aliases, expired/conflicting leases, pack allocation failure and exact/below boundaries are checked. The BLAS-profile ASan/UBSan run with Eigen allocation disabled during execution passes.
Fixed-input performance is measured on arm64 with one BLAS/CPU worker, 15 rotated samples and resets outside the clock. The scalar comparison isolates the actual predecessor’s m,e loop; the complete residual also measures the effect of removing it from the fused loop. The strong comparison gathers a whole OV-by-OV amplitude matrix, uses Accelerate GEMM and scatters its whole product. This needs two additional complete matrices; neither is used by the bounded plan. Times below are median microseconds; cold planning is a single observation. The checked peak includes input/output owners, controls and scratch, excluding benchmark-only comparison arrays.
Occupied, virtual |
Cold/warm plan |
Scalar loop |
Checked |
Whole-layout total (gather/GEMM/scatter) |
Scratch / peak bytes |
|---|---|---|---|---|---|
2,2 |
16.75/0.71 |
0.08 |
0.29 |
0.38 (0.08/0.08/0.04) |
544 / 5,992 |
4,19 |
2.33/0.63 |
331.08 |
69.46 |
22.46 (3.00/14.38/4.50) |
81,952 / 225,640 |
9,11 |
2.25/0.71 |
813.71 |
143.63 |
36.21 (5.92/24.79/5.38) |
81,952 / 322,240 |
8,31 |
3.50/0.67 |
16,257.20 |
2,235.62 |
649.42 (124.29/434.96/73.13) |
81,952 / 1,563,112 |
12,40 |
82.58/0.67 |
140,220.00 |
15,605.30 |
4,471.33 (492.79/3,644.33/285.46) |
81,952 / 5,616,616 |
Maximum benchmark error was 6.04e-14. The checked path regresses for the tiny case, improves on the scalar loop for the four larger shapes, and loses to the whole-layout vendor path on those shapes.
A separate source-linked profile measures the actual predecessor and changed residual on the same 4-occupied, 19-virtual, 76-auxiliary fixed-amplitude fixture, one worker and seven calls per route. The predecessor includes both previously migrated ladders. The half timer includes its allocation and all scalar half terms, plus the changed ring admission/execution; the ring timer isolates binding/execution within that half. Times are median us:
Virtual integral storage |
Whole half old/new |
Checked W2 binding + execution |
Full residual old/new |
|---|---|---|---|
Resident |
3,070.42/2,584.92 |
86.00 |
6,047.38/5,662.92 |
Five a tiles |
3,036.42/2,589.12 |
84.29 |
6,715.17/6,304.67 |
The containing native executable repeated the four vq controls described above on identical upstream RHF references, with no triples. The external ORCA 6.1.1 inputs, results and tolerances were unchanged and reused. The largest difference in HF, correlation or total against ORCA was 1.51e-7 Eh (preset 2e-6 Eh). Total energies were -2.256116629201248 Eh for DF H4, -2.256128554995736 Eh for conventional H4 and -76.17181293397357 Eh for frozen-core DF water in both routes. Compared with the previous ladder implementation, each HF/correlation/total difference was below 1e-10 Eh. Every T1 entry and each iteration’s energy/R1/R2 norm agreed within 5.21e-18 and 2.28e-14 respectively (preset 2e-10); final T1/T2 norms also passed. Source/binary identities and receipts are recorded on #806. The executable qualification is independent of packaged delivery.
Complete calculation |
Iterations / all checked products |
Peak (bytes) |
CCSD old/new (ms) |
|---|---|---|---|
DF H4 |
17 / 51 |
81,784 |
16.77/74.56 |
Conventional H4 |
17 / 51 |
78,200 |
0.32/0.35 |
Frozen-core water, resident |
52 / 156 |
3,745,144 |
15,024.65/3,636.98 |
Frozen-core water, five tiles |
52 / 364 |
2,118,752 |
10,762.19/6,550.37 |
Each calculation reached its measured cap and refused one byte below it. These four whole-method phase maxima were unchanged: the largest ladder phase already exceeded the W2 phase. The fixed-amplitude ring-dominant control above tests the new cap independently. Complete timings are medians of three CCSD calls on fixed RHF inputs, with predecessor and new runs in separate batches on a shared machine. Both H4 timings regress. The changes in these complete times cannot be attributed solely to the W2 kernel. Other ring flavours, W construction, local CC, UCCSD and triples remain outside this increment.