Hey everyone,

I’m a systems engineer with 40+ years in the trenches, and I’m heads-down building a 100% offline, plug-and-play OS appliance called NodeAI OS (working title, pending final brand alignment before release to clear an enterprise trademark overlap).

The core feature I’m testing is our Asymmetric Acceleration Profile Selector. It dynamically orchestrates mismatched architectures (NVIDIA CUDA, AMD ROCm, Intel SYCL, and standard Vulkan) across concurrent compute lanes to aggregate a unified VRAM pool. If no dedicated hardware acceleration is detected, the pipeline automatically switches to an optimized CPU execution fallback path to guarantee system availability.

The user can toggle between Faster Performance (routing compute strictly to matching high-speed silicon lanes) or Maximum Capacity (pooling mismatched multi-vendor VRAM together to load massive local models that would otherwise overflow the system).

I wanted to drop a quick teaser of the backend selector interface and the active telemetry log matrix to see what architectural bottlenecks you guys foresee when running multi-vendor driver switching matrices over high-speed USB 3.2 interfaces!

The Attached Technical Visual (Top / Bottom Montage):

• Image 1: The Asymmetric Acceleration Profile Selector

• Caption: Heterogeneous (mixed-vendor) pool combining mismatched silicon. Allows the operator to toggle between Faster Performance (high-speed lanes) or Maximum Capacity (pooled VRAM for massive models).

• Image 2: The Live nvtop Telemetry Matrix

• Caption: Live nvtop hardware telemetry mapping a highly heterogeneous cluster. Tracking a concurrent mix of triple NVIDIA RTX 2060s, an AMD RX 6700 XT, an Intel Arc B580, and an Intel UHD 630 iGPU running side-by-side without driver collisions.
  • DevNodeMan@lemmy.worldOPEnglish
    1·
    8 days ago

    How feasible would it be to implement sbc npus such as rockchip with this? Some of them are already in mainline

    It is feasible, in that on ARM, your compute options are going to be the vendor-native paths (like the NPU/VFP), GPU Vulkan lanes (Mali, Imagination, or Adreno), or generic GPU/CPU OpenCL.

    Real-world mileage is going to vary massively based on the specific vendor’s silicon routing and how they handle unified memory. As, you probably know, these tiny boards share one single memory bus across the CPU, GPU, and NPU, bandwidth starvation becomes your main bottleneck the second you try sharding models.

    That said, the CPU fallback path will absolutely work out of the box to guarantee a baseline.