vibe-qcDocumentation

Complete Fock and eigensolver unification programme

Parent #472. Reconciled against main ab5a8b01, the construction inventory, source definitions and current GitLab records on 2026-11-02. This is the execution plan for the whole programme, not a release prerequisite. #557 is closed; its bounded acceptance must not be reopened by unfinished architecture.

What fully unified means

One semantic model, with representation-specific implementations, must connect physical construction, numerical solution and consumers. It is not one global mutable matrix, one integral engine, or one universal energy formula.

  1. A source declares the actual finite Hamiltonian: basis/ECP, geometry/cell, kernel/range coefficients, image/support domains, fitting factors/rank, exchange zero mode, grid/XC and corrections. Provider receipts are distinct from scientific accuracy certificates and from orbital frames.

  2. A frame declares AO order, overlap, retained projector, coordinate gauge, spin/k/symmetry labels and occupation partitions. Equal dimensions do not imply equal frames. Independent spins retain independent orbital spaces.

  3. A physical evaluated build binds the complete coupled density, physical Fock channels and method-owned energy record to one immutable identity.

  4. Effective ROHF/ROKS and solver-modified operators name their physical parent. DIIS, damping and level shifts cannot overwrite the physical state.

  5. A solver result identifies its operator/frame, residuals, metric orthogonality and completeness. Convergence is not inferred from a backend boolean or an energy change alone. A stopped SCF state is not stationary.

  6. Correlation, properties, derivatives and output consume an admitted state; they cannot reconstruct a different fit, remove an Ewald exchange term or combine old energy with next-step density. A Fock object does not certify correlation factors or derivatives.

FockOperator, OperatorFrame, SourceProvenance, EnergyComponents and FockBuild remain the starting vocabulary. Reuse them; do not create parallel libint, MANTA, BIPOLE or symmetry state classes with competing semantics. A matrix element is meaningful only with its frame and physical source.

Actual gap, not a percentage estimate

The shared Python/native eigensolution boundary and selected Gamma evaluated states exist. The recent #472 admission fixes are useful prerequisites, but are not an all-method construction migration. FockBuild.from_dense owns immutable D/F snapshots; it currently requires dense owned operators. Native hermitian_operator.hpp borrows matrices synchronously for solves; that alone is not the common native evaluated-state bridge in #594.

The existing family inventory in operator_consumers.md is the route authority. Of #594-#616, #602 is closed and 22 remain open at this snapshot. Open does not mean absent: #603 has Gamma external-RKS composition, #604 has bare augmented RHF, and #608 has explicit symmetry producer bridges. Recent #580 PAO/PNO and source-admission increments also remain reusable. Do not use closed-issue counts as percent complete.

The material missing work is:

  • Native lifetime-safe evaluated-state capture shared by libint and MANTA.

  • Molecular HF/KS/open-shell construction and result migration beyond solves.

  • Complete spin/k/grid/fitting and BIPOLE/MANTA periodic state adapters.

  • Sparse, compact, streamed or borrowed state ownership without densification.

  • Remaining correlation, derivative/response and restart/output associations.

  • A route-by-route closure ledger with numerical and resource evidence.

Literature comparison and mathematical contract

The primary papers below were read from the configured literature library. Equations here use project notation, not literal transcriptions. These papers justify the numerical invariants; none prescribes a particular software class. The existing operator contract remains authoritative.

Primary source and passage

Consequence for this programme

Current gap or action

Lowdin 1950, orthogonalization and matrix transformations, Eqs. 15-27

Solve F C = S C e, C^H S C = I; use X^H S X = I, C = X Y in retained space

Carry actual retained X/projector and cutoff, not just dimension; rectangular-X and complex transports remain mandatory controls. Rank truncation policy is a numerical extension and stays explicit

Roothaan 1960, open-shell variational equations and coupling operators, Eqs. 17-29

Physical spin Focks, constrained effective operators and occupation blocks have distinct meanings

Finish molecular #597 and remaining periodic RO routes; do not impose UHF stationarity or compare convention-dependent RO orbital energies as invariants

Pulay 1982, Eqs. 2-4 and algorithm steps 1-3

Residual uses the density that constructed F; R = F D S - S D F, optionally transformed with X; UHF has separate spin residuals

Keep physical and extrapolated operators separate, and preserve each method’s residual norm and aggregation

Davidson 1975, sections I-II, residual correction and projected subspace method

Iterative eigenpairs require residual and subspace checks; partial roots are not complete spectra

Bounded solver parity work under #615, including callbacks, complex types, degeneracies and retained rank; shared code alone is not acceptance

VandeVondele et al. 2005, sections 2.1-2.4, GPW energy, electrostatics and XC discretization

Energy and KS matrix must derive from the same discrete density/grid functional

Complete #603-#605 with joint energy/potential and augmentation terms; changing the quadrature or support is a source change

Mermin 1965, Eqs. 1-4 and finite-temperature variational functional

Energy, entropy, free energy, occupations and chemical potential are distinct state quantities

Preserve admitted smearing conventions and enough spectrum for occupations; fractional densities need not be idempotent

Gygi and Baldereschi 1986, Eqs. 3-5, exchange singularity treatment

Periodic exchange corrections depend on the defined sampling/kernel convention

Preserve zero-mode and mesh source identity through #598/#601/#607 and correlation; never remove the correction to force agreement

For unit-occupation spin densities, molecular HF obeys F_s = h + J[D_a + D_b] - K[D_s] and `E_HF = E_nuc + Tr(D_total h) + 1/2 Tr(D_total J)

  • 1/2 sum_s Tr(D_s K_s). Restricted total-density conventions need their own factors. For KS, use the method's E_xc[D]and derivative potential, not half the trace ofD V_xc. With fixed source/frame and allowed density perturbation, test dE = sum_s Re Tr(F_s dD_s); periodic sums include the declared k weights. This identity does not include geometry/Pulay terms or entropy derivatives. For a retained solve, check X^H(F C - S C e)` as well as reporting the AO residual; discarded-space residuals need not vanish. Compare occupied and degenerate-subspace projectors, not arbitrary eigenvector phases.

Ordered implementation packages

Read the current issue notes before claiming. Existing owners retain their files and jobs. Dependencies below mean a specific delivered interface, not closure of an umbrella. Each row may land in several small increments.

Stage

Existing issues

Deliverable and first slice

Dependency / acceptance focus

0

#615, #472

Freeze per-route completion rows; classify physical Fock, effective operator, metric, response and non-Hermitian bypasses; bound remaining legacy solver parity

Current inventory and accepted evidence; no numerical rewrite or duplicate audit campaign

1

#594

Native borrowed evaluated view plus owned terminal export; first native RHF converged/capped/NOITER state

Define lifetime, source/frame/density identity and copy budget before changing method loops

2

#595

Molecular libint RHF then UHF, stored/direct/fitted variants, PATOM and terminal confirmation

Stage 1 seam; correct factors, independent spins, no extra integral work on exact reusable terminal state

3

#596, #597

RKS then UKS joint XC; separately ROHF then ROKS physical/effective association

Stage 1 plus matching HF semantics; pure/global/range-separated/+U/smearing only where already admitted

4

#616, #608

Same molecular bridge for MANTA direct/RI and qualified quartet/RI symmetry producers

Stages 1-3 interfaces as needed; preserve #619/#635 provider capability and exact-source receipts, including ordered fallback

5

#598, #599, #600, #601

Multi-k Ewald; Gamma GDF open shell; fitted/source/projected variants; multi-k GDF and restricted open shell

Coupled spin/k evaluated state, weights, support/fit rank, full/IBZ transport; no new source equivalence

6

#603, #604, #605

Finish real external-XC acceptance; augmented GAPW temperature/+U/KS/spin; multi-k GPW/GAPW and fixed-density bands

Reuse accepted Gamma GPW and #602; keep parent SCF density separate from band sampling and augmentation energy

7

#606, #607, #577

BIPOLE terminal adapter and native periodic MANTA bridge with source-qualified symmetry

Provider-owned finite Hamiltonians; preserve BIPOLE residual aggregation, Bloch/mesh and Ewald conventions; #629 substitution separately qualified

8

#609, #610

DFTB then PM6/OMx/GFN/MSINDO effective Hamiltonian/state adapters

Reuse frame/state contract, retain model-specific SCC, multipoles, repulsive/dispersion energy and experimental boundaries

9

#580, #611

Finish DLPNO and other canonical/local/multireference Fock consumers

Actual stationary reference, independent spin/frozen partitions, semicanonical blocks and immutable admitted factor source; #581 remains factor authority

10

#612, #613

Properties/gradients/stress/response and public result/QVF/restart completion for each migrated family

Same source and density; derivative capability separately admitted; serialize physical versus solver/effective roles and stopped-state flags

11

#614, #519

Remaining experimental chi and real-Gamma operator/state coverage

Preserve finite-character/torus conventions and experimental labels; canonical controls do not prove periodic correlation accuracy

12

#615, #472

Final route coverage and compatibility closure; remove only proven redundant dispatch

Every applicable row satisfies the validation plan; non-Fock exclusions justified; no silent default or release-policy change

Stages 5-8 and consumer slices can proceed independently once their particular state seam exists. Stage 10 travels with each producer migration, not as a single end-of-project serialization rewrite. #602 is accepted and is a control, not a new work package. All 23 existing family issues are accounted for; #580/#577/#519 are reused rather than duplicated.

First executable milestone: native RHF vertical slice

Implement under #594 with #595 and #613 as consumers. Start in cpp/src/rhf.cpp, native result/binding definitions, cpp/include/vibeqc/hermitian_operator.hpp, python/vibeqc/fock_state.py and its focused tests. Resolve current result/binding paths before editing.

  1. Trace one currently admitted libint RHF build through terminal result and identify which D built F/E, which orbitals define returned D, and what the stopped-state policy currently guarantees. Record the seam on #594.

  2. Specify synchronous borrowed view lifetime and explicit owned export. Borrowed views cannot survive an iteration or provider mutation; owning capture survives later builds. Reuse frame/source objects where immutable.

  3. Capture the existing physical F/D/energy at the same evaluation. Preserve algorithm, convergence thresholds and finalization; if the existing terminal data are inconsistent, fix that demonstrated defect before declaring parity.

  4. Adapt to the existing shared state vocabulary and public result/runner seam. Do not add a Python callback or per-iteration snapshot to a native loop.

  5. Run converged, capped and NOITER RHF plus one rank-deficient frame control; check mutable-input refusal, independent exported lifetime and resource cost.

  6. Land checked source with CI skipped, then complete one containing-runtime vq calculation against the matching accepted reference or fresh ORCA case. State source/native identities, limitations and exact remaining variants.

  7. Reuse this seam for UHF and an admitted MANTA RHF source. This is the first cross-provider milestone; it does not wait for all MANTA performance work.

The first trace found a concrete native RHF prerequisite: on a capped run, the result density and energy were evaluated before DIIS/Fock mixing, while the returned Fock was copied afterward. The native result now captures F at the physical-build point, reusing the existing per-iteration result copy. This preserves SCF updates and converged finalization; it is not completion of the borrowed/owned bridge or density-defining orbital/restart transport. COSX terminal correction and its iterated energy surface remain distinct and must not be admitted as one physical evaluated state without a named policy.

The first native export is now explicit: RHFOptions.capture_evaluated_state retains one terminal OwnedEvaluatedFock from a synchronous EvaluatedFockView. RHFResult.export_evaluated_state() creates the existing immutable FockBuild and a natural-orbital restart view. The first source admission is conventional, all-electron, unfitted libint RHF with the native optimizer. Other sources, including MANTA/external JK, refuse export pending their producer receipts; their calculation APIs are unchanged. The captured actual S/X retain native rank reduction. Native storage is 8*(4*n*n+n*r) bytes per opted-in result, with no per-iteration copy or extra source evaluation. Python export owns a second dense state. See the SCF modes guide. Containing-runtime calculation acceptance is recorded on #594; source checks alone do not close this milestone or the broader bridge issue.

Do not continue an indefinite sequence of isolated legacy guards as the main programme. Any demonstrated guard defect still gets a bounded fix, but the next architectural critical path is stages 1-2.

Resource and API acceptance

Keep synchronous native views allocation-free where existing kernels borrow matrices. Measure persistent bytes, peak extra workspace, bytes copied, source calls and setup/iteration/terminal timings before and after. Export copies may cost O(n squared) per channel, but must not accumulate an iteration history. For streamed/compact routes, no full ERI, global factor or all-k dense snapshot may appear as an adapter side effect. A compact owning state design is a named #594/#607 increment; the current dense FockBuild is not a bounded-memory claim.

Keep public result fields and QVF compatibility while adding explicit state access. Legacy accessors delegate to the physical state only when that mapping is valid; otherwise refuse or expose a clearly named effective/solver field. No automatic provider selection, silent fit replacement or tolerance changes. Performance regressions need measured explanation and a bounded remedy, not new production defaults. Numerical acceptance and resource evidence are separate columns in the completion ledger.

Programme completion

Use fock_unification_validation.md for the acceptance matrix. #472 can close only when every applicable route is accepted or explicitly scoped out by the maintainer, every downstream dependency is resolved for its advertised capability, and compatibility/resource checks pass. A mathematical exclusion (CI Hamiltonian, Hessian, auxiliary metric, full RPA, non-Hermitian CC Jacobian) remains a separate role, not a failed Fock migration. No implementation estimate or completion percentage is credible until stage 0 and the first native slice establish the remaining route count and cost.