Lodestar.Gpu 0.1.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

DeviceEmbeddingMatrix

A row-major embedding matrix held on the accelerator across many queries.

public sealed class DeviceEmbeddingMatrix : IDisposable

Example — the shape a caller writes.

using Lodestar.Gpu.Compute;

using var context = GpuContext.Create(preferCpu: true);
float[] rows = [1f, 0f, 0f, 1f, 1f, 0f];
using var matrix = DeviceEmbeddingMatrix.Upload(context, rows, count: 3, dimension: 2);

int held = matrix.Count;  // => 3
int wide = matrix.Dimension;  // => 2

Members — one page each.

Member	What it does
DeviceEmbeddingMatrix.Upload	Uploads a row-major block of vectors
DeviceEmbeddingMatrix.Dispose	Frees the device memory the matrix holds

Properties — Count is how many rows, Dimension how many values each holds.

Remarks — residency is the point. A GPU package that uploads its corpus per call is slower than the SIMD path it replaces: measured at 9.25× slower for a hundred thousand documents answering one query, against 6.6× faster once the same matrix is resident. The crossing point is around a hundred queries per corpus.

This is residency only. Decision 0102 defers the chainable device-resident types until three kernels exist, because chainability is a claim about two operations sharing a residency — DeviceDenseBlock is that half, and it carries doubles rather than the floats an embedding matrix holds.

Rows are L2-normalized on upload by default, which is what turns cosine similarity into a dot product. A zero row is left alone rather than divided by zero.

Applies to — net10.0, netstandard2.1.

See also — TiledCosineTopK, the namespace index.