Experimental local tensor operations from Python¶
The first TENS-09 interface exposes synchronous, single-caller CPU operations
through vibeqc.tensor. It requires a native runtime built with the local
binding. It does not initialize MPI. The existing Python
DensityFitting.build_J(D) uses this interface for native FP64 factors and
FP64 densities, with the same RI equations and a writable NumPy return.
Its non-FP64 and complex-density cases retain the previous NumPy path.
The FP64 method explicitly permits reassociation and uses the dense-layout
GEMV lowering described below; it does not promise bitwise NumPy equivalence.
Bind, snapshot, execute¶
Supply native float64 or complex128 NumPy arrays. Convert other types
explicitly before import. A finite budget is mandatory for Session.
import numpy as np
from vibeqc.tensor import Domain, Session, Source, Space
session = Session(2 * 1024 * 1024)
source = Source(Space.create(), Space.create(), Space.create())
p = Domain.range(Space.create(), 5)
a = Domain.range(Space.create(), 2)
b = Domain.range(Space.create(), 3)
left = np.arange(10, dtype=np.float64).reshape(5, 2).astype(np.complex128)
right = (1j + np.arange(15, dtype=np.float64)).reshape(5, 3)
l = session.snapshot(left, [p, a], source)
r = session.snapshot(right, [p, b], source)
plan = session.pair(l, r, conjugate_left=True)
result = plan.execute()
value = result.array # Immutable [a,b], retains the native result owner.
pair computes the ascending-p compensated complex sum. Either input can be
conjugated explicitly; neither is conjugated by default. Bilinear and
Hermitian products are different operations. There is no real projection,
phase adjustment, normalization, q/-q identification or scientific source
qualification in this API.
For RI-J, snapshot real factors with domains [p,m,n] and
layout="per_p", snapshot the density with [m,n], then call
session.ri_j(factors, density).execute(). The checked native resident provider
uses the registered single-worker per-P kernel. It computes
gamma[p] = sum(m,n,B[p,m,n]*D[m,n]) followed by
J[m,n] = sum(p,B[p,m,n]*gamma[p]). Rectangles are allowed. There is no implicit
symmetry or conjugation. Numerical ordering follows the registered native
kernel, so roundoff need not be bitwise identical to NumPy/BLAS.
For a resident dense FP64 factor snapshot that avoids unnecessary conversion,
use layout="ri_j" and explicitly bind
session.ri_j(factors, density, numerical_mode="reassociated_fp64").
The snapshot preserves C, F and auxiliary-contiguous producer order where
possible. Sliced, reversed or broadcast arrays are copied directly into a
supported dense execution order. The same checked RI-J plan uses two
single-thread allocation-audited Eigen BuiltIn GEMVs, with no factor pack or
opaque BLAS workspace. Density/output reorder buffers, when needed, are
admitted. Unsupported older Eigen uses a deterministic scalar kernel; a
non-finite BuiltIn result retries the deterministic scalar IEEE path.
The default numerical_mode="preserve_order" still requires the per_p
snapshot capability and uses the registered ordered kernel. There is no
runtime timing calibration, changed global BLAS thread setting or numeric
factor cache. The physical Python method uses a coarse native entry point to
create these same snapshots and bindings once per call, avoiding repeated
Python handle/dictionary round trips. Explicit prepared inputs/plans remain
available for callers who intend immutable reuse.
Ownership, revisions and supported layouts¶
Snapshots copy into admitted native storage. Subsequent mutations through any Python alias do not change bound input, and deleting Python inputs is safe. The caller must exclude concurrent native writes during the snapshot itself; NumPy’s read-only flag cannot prevent such writes. Zero-copy import is not available. Python is never called from an element loop or native worker.
The importer supports rank zero through three, C/F layouts, transposes, slices, reversals, empty axes and read-only broadcast mappings. Scalar and empty snapshots are legal; operation ranks are still checked independently. It verifies byte alignment, strides and the complete accessible span against a contiguous NumPy-owned root, following at most 32 ndarray bases. Arbitrary buffer exporters, stride-trick proxy bases, non-native byte order, other dtypes and rank above three are refused. Make an explicit safe NumPy copy first when an unsupported exporter owns the data. There is no hidden force-cast.
Domain.range(space, extent, begin=0, step=1, parent=..., parent_coordinate=...)
uses the existing native domain. Reuse the same domain objects for indices
that mean the same thing. Equal dimensions do not establish compatibility;
space revision, parent identity/coordinate and coordinate map matter.
Space.revised() returns a revised token. Source, metric, frame and partition
must also agree. Tokens are assertions by the owning method, not proof of a
physical Gaussian source, whitener or gauge. Selected non-affine maps and
physical local-provider submission remain later bindings.
Plans retain their immutable snapshots and may execute repeatedly. Each
execution publishes a distinct result only on success. session.invalidate()
invalidates old inputs and plans; take new snapshots and bind again. Earlier
results remain valid. Concurrent calls on a busy session are refused. Input
leases and admission preflight precede GIL release; native execution rechecks
its own tickets and leases. Exceptions return with the GIL restored and
private allocations released.
result.array is read-only and retains its native owner, even when the plan,
session and result handle are deleted. result.copy_array() makes an admitted
writable NumPy copy. That returned copy becomes caller-owned and leaves the
engine inventory. Handles, owners and process-local revisions are not a wire
or pickle format. Reconstruct metadata and rebind in another process.
Accounting and diagnostics¶
session.live and session.peak report the shared native Budget ledger.
The ledger includes snapshots, retained native owners, fixed native controls,
registered execution workspace and output. During import it counts the root
allocation once, even for a sliced or broadcast view, plus the new copy.
Imports are sequential; the engine does not retain external roots afterwards.
Separate snapshots intentionally own separate copies, even if their Python
inputs alias. Complex execution previews the existing conservative stream
control envelope. This is a tracked-byte ceiling, not a process RSS limit.
Python/NumPy object headers, allocator overhead and unrelated caller-owned
arrays or method state are outside it. Concurrent export may cause a safe
execution refusal despite successful earlier preflight.
Input diagnostics report metadata/import time, copy time and copied bytes. Plan diagnostics separate binding and checked planning. Result diagnostics report additional execution planning, native execution, export time/copy bytes, native factor-packing time, density-packing bytes, kernel, tile visits and tracked bytes, factor/density/output packing bytes, backend workspace bytes and reduction policy. Density/output reorder time is included in execution. The coarse method event includes input and plan phases; explicit handles expose those phases separately. Add the plan and result planning times for the total. Export time is cumulative for that result.
The legacy DensityFitting.build_J signature has no memory-cap parameter.
Like the native legacy adapter, it uses the largest representable ceiling;
this is not a finite-memory promise for the full DF method. It snapshots both
public mutable inputs on each call and makes the writable export explicitly.
It emits tensor_execution through vibeqc.output.record, including these
costs. Use Session directly for finite admission and immutable plan reuse.
The source ADR records tests, measured regressions and pending complete-method scientific acceptance. The layout-aware increment reduces measured production-call costs but large calls still cost more than NumPy with equivalent snapshots in the recorded runs. This interface does not promise a universal speedup. Async execution, DLPack/devices, serialized plans, free-threaded Python qualification and a second physical local-method consumer remain outside this increment.