Module_GPU_Policy - vorcyc/Vorcyc.Mathematics GitHub Wiki

Location: Home / GPU policy

GPU / CUDA policy

Conclusion (0.9+): The main Vorcyc.Mathematics NuGet package does not include and does not plan to ship built-in GPU / CUDA acceleration. Historical ILGPU dependencies and experimental CudaFFT code have been removed. All numerical computation runs on the CPU, with execution policy provided by ComputingContext (scalar / SIMD / multithreaded parallel).

:ledger:Contents

Removed items

Item Description
NuGet dependencies ILGPU and ILGPU.Algorithms removed from dotNET10_Portable
Source code CudaFFT, CudaFastFourierTransform, CUDA_FFT deleted (long unmaintained, incorrect results)
Public API Main package no longer provides CudaFFT or other CUDA FFT types
Tests CuFFT_test deleted

These capabilities were never released as stable public features; documentation and README wording about “GPU compute (incomplete)” has been retired.

Why the main package drops GPU

This is an active product and technical roadmap decision, not a temporary deferral.

1. Aligns with library positioning

The core library targets a pure .NET, portable, native-driver-free high-performance math stack. By default, users should be able to:

  • Work on machines without NVIDIA GPUs
  • Run on servers / CI / edge devices
  • Use machines with only the .NET runtime installed

and dotnet add package Vorcyc.Mathematics without friction. Bundling CUDA into the main package would conflict with that goal.

2. Current workloads fit the CPU better

Typical 0.9 scenarios include:

  • Tabular machine learning (trees, KNN, Softmax, etc.) — no GPU benefit
  • Small-scale custom DL (XOR, small CNN, MFCC classification) — small batches; PCIe transfer often exceeds compute gains
  • Signal processing and MFCC — CPU + SIMD / parallel is sufficient
  • Examples and tests — correctness and reproducibility first

At this scale, GPU maintenance cost is lower ROI than refining ComputingContext and CPU hot paths.

3. Historical GPU code was not production quality

CudaFFT source once noted “results are wrong; do not use for now.” Keeping ILGPU would:

  • Increase package size and transitive dependencies
  • Create false expectations that GPU is supported
  • Split focus away from CPU path testing and unified documentation

The choice is to remove experimental code and document clearly, rather than keep a “incomplete” label.

4. GPU training on the custom DL stack is too costly

Full GPU training on the existing Trainer + hand-written Backward would require:

  • A device memory model (DeviceTensor)
  • Forward / backward kernels or cuDNN integration
  • CPU/GPU numerical alignment tests

That is close to a separate product line. Without clear large-model training demand, it is not a main-package goal.

Current performance path

The main package uses a unified CPU execution strategy:

Mechanism Description
ComputingContext Normal / Simd / Parallel / Auto
ComputingScope Block-level default policy
Hot-path coverage FFT, statistical reductions, matrix multiply, VectorSpan, batch DL forward, parallel MFCC, training entry points, etc.

Recommended usage:

using (ComputingScope.Enter(ComputingContext.Parallel))
{
    // APIs with context:null inside the scope use Parallel
}

values.Sum(ComputingContext.Simd);
MatrixFp32.Multiply(a, b, ComputingContext.Parallel);
trainer.FitBatchSequential(..., computingContext: ComputingContext.Parallel);

See the ComputingContext manual and the “Verification and tests” section.

If you still need more compute

Choose solutions outside the main package rather than expecting built-in CUDA:

Need Recommendation
Faster FFT (CPU native library) Optional package Vorcyc.Mathematics.Extensions.FFTW
Large-model / large-batch DL inference Export to ONNX or similar; use ONNX Runtime (including CUDA EP)
Large-batch training Use mature frameworks such as PyTorch / TensorFlow; this library’s DL module targets teaching and small-scale differentiable experiments
NVIDIA acceleration inside this solution Wait for or implement an optional extension package (see below); do not assume the main package enables GPU by default

Possible future direction

If clear product demand appears later (e.g. large-batch inference, customer-mandated NVIDIA), GPU capability would ship as an optional extension package, following the FFTW extension pattern:

Vorcyc.Mathematics              ← main package, CPU only, zero CUDA dependency
Vorcyc.Mathematics.Extensions.FFTW
Vorcyc.Mathematics.Extensions.*   ← future: CUDA / other native backends (separate install)

Principles:

  • Default main-package installs do not pull CUDA runtime, cufft/cublas DLLs, or driver version issues
  • Extension packages are explicit references with explicit APIs or ComputingContext Backend=Gpu (not implemented yet)
  • Main-package APIs and tests use CPU as the sole baseline

No GPU extension package exists in the current version; this document states architectural principles only and is not a release commitment.

Impact on users

Upgrading from early internal / documentation versions

  • Do not reference CudaFFT or ILGPU-related types; builds will fail.
  • For FFT, use FastFourierTransform + ComputingContext, or the FFTW extension.
  • README “Dependencies” no longer lists ILGPU.

API design

  • ComputingContext describes CPU modes only (CpuExecutionMode).
  • Enums such as ComputingBackend.GPU remain placeholders; the main library does not use them at runtime.
  • “GPU / CUDA” in documentation refers to outside the main package; see this document.

Feedback

If you believe a specific scenario must have GPU support within the Vorcyc ecosystem, include: data scale, batch size, latency targets, and inference-only vs training. That helps evaluate an extension package vs restoring main-package dependencies.

Return to Home · ComputingContext manual