Looking for Windows users with 2–8 GB GPUs to test a low-VRAM local LLM runtime

Community Calls
1

Hi everyone,

I’ve been developing a Windows local-LLM application called StreamAI, specifically focused on running useful language models on systems with limited GPU memory.

My development machine is deliberately modest:

AMD Radeon RX 570 — 4 GB VRAM
32 GB system RAM

On that system, the current beta has qualified two modes:

Fast mode
Qwen2.5 1.5B Instruct
About 1.1 tokens/sec

Quality mode
Qwen1.5 7B Chat
About 0.45 tokens/sec

The goal of the project is not to compete with high-end GPUs. I’m trying to determine how useful local LLM inference can remain on older or limited-VRAM Windows hardware.

StreamAI uses a memory-bounded CPU/GPU streaming approach so that model execution does not depend on keeping the entire working model resident in GPU memory.

I have reached the point where testing only on my own RX 570 is no longer useful. I am looking for a small number of Windows testers with different hardware.

I am especially interested in:

  • AMD GPUs with 2–8 GB VRAM
  • older Radeon cards
  • NVIDIA GPUs with 2–8 GB VRAM
  • Intel Arc or Intel integrated graphics
  • laptops with limited or shared GPU memory
  • machines that normally struggle with local LLM runtimes

I would initially like to test on roughly 6–10 different machines rather than distribute the beta widely.

The beta includes an automated hardware and inference qualification process. After testing, it creates a small results ZIP that the tester can inspect and manually send back to me.

There is no automatic telemetry or automatic uploading of results.

The beta is currently a compiled Windows application and uses a short-lived machine-bound tester license while I keep the test group controlled.

If you are interested, please reply with:

GPU / VRAM / System RAM / Windows version

For example:

GTX 1650 / 4 GB / 16 GB RAM / Windows 11

I am interested in failures just as much as successes. My goal is to eventually publish the hardware results, performance measurements, and practical limits so the information is useful to other people working with constrained hardware.

The main question I am trying to answer is:

How low can the hardware requirements go before local LLM inference stops being genuinely usable?

If there is interest, I’ll share the results from the different machines here as the testing progresses

2

Happy to help test. My specs:

RTX 5060 Laptop GPU / 8 GB VRAM / 32 GB RAM / Windows 11

(Also have an Intel UHD iGPU on this laptop if you want a data point closer to your integrated-graphics target range — let me know if your tool can target that instead of the discrete GPU.)

CPU: i7-14700HX, for context since your streaming approach leans on CPU too.

A few questions before I run it:

  • Is the build code-signed, or should I expect a SmartScreen warning?
  • Can you share a SHA256 checksum for the download so I can verify it?
  • Does the app make any network calls at all (including for the license check), or is it fully offline as described?
  • if you have pushed the code on github send me the link i will tell you what to can i do
3

Hi ottale. I could run that test on a laptop on Monday:
RTX 3050 laptop / 4GB / 16GB RAM / Windows 11 Pro.

I have an older PC that I could try to get working this weekend:
RTX 1050 / 2GB / 16GB RAM / windows 11 pro. (Ready)

Let me know if these are helpful, and I’ll start researching how to revive the old PC this afternoon. No rush, though. If I don’t have time, you’ll only be able to use the first one for your test on Monday, and the other one sometime during the week.

Good luck with your project.

5

Hi TopAss, my apologies for the late response.

To answer your questions:

Code signing / SmartScreen

The current controlled beta is not yet Authenticode code-signed, so you should expect Windows SmartScreen or an “Unknown publisher” warning. I’m being upfront about that because I don’t want anyone assuming the installer has a publisher signature that it does not have.

SHA-256

Yes. I will provide a SHA-256 checksum for the exact tester-kit ZIP you receive, and the package also contains checksums for its contents so you can verify the installer before running it.

Network access

The beta does not send telemetry, qualification results, or hardware reports automatically.

The machine-bound beta license is verified locally using a signed license file. It does not contact an online licensing server each time the application starts.

The application can make network requests to download the required model files, primarily from Hugging Face, when those model files are not already present locally. Once the required model/cache files are available locally, inference itself is local.

The qualification results are packaged into a ZIP on your machine, and you decide whether to send that ZIP back to me.

Source / GitHub

The core StreamAI runtime is not currently public on GitHub. This is a protected controlled beta while I validate the architecture across different hardware. I may publish supporting documentation, benchmarks, and selected tooling separately, but I don’t want to imply that the runtime source is open source when it currently is not.

I have the controlled beta tester kit ready. For security, I’d prefer to share it through a restricted Google Drive link rather than post the executable publicly. If you’re comfortable doing so, please send me the email address you’d like me to grant access to. I’ll use it only to provide access to the beta file.and its SHA-256 checksum.

6

Hi Logos-Adri, my apologies for the late reply.

Both systems would be very helpful.

The RTX 3050 laptop with 4 GB VRAM is especially useful because my current qualified development machine is an AMD RX 570 with 4 GB as I had mentioned, so this would give me an important NVIDIA comparison.

I would also definitely like to test the older 2 GB system. In fact, machines near the lower hardware limit are particularly useful. If it doesn’t work, that result is still valuable because part of the testing is determining where the practical hardware boundary is. Just one clarification: you listed “RTX 1050.” Did you mean a GTX 1050? There is no rush to revive the older PC. The RTX 3050 test on Monday would already be very useful.

I’ll send you the controlled Windows tester kit along with the SHA-256 checksum and instructions. The software does not automatically upload telemetry or results. After testing it creates a results ZIP that you can inspect before sending it back to me.

The beta uses a temporary machine-bound test license. After installation, the application will generate a machine-license request. Send that request back to me and I’ll generate the temporary tester license for that machine.

I have the controlled beta tester kit ready. For security, I’d prefer to share it through a restricted Google Drive link rather than post the executable publicly. If you’re comfortable doing so, please send me the email address you’d like me to grant access to. I’ll use it only to provide access to the beta file if your ok with that.

Thank you for helping with this, it’s muched appreciated.

7

Hi ottalese,

You are completely right! My apologies for the typo, it is indeed a GTX 1050 with 2 GB VRAM.

I will send you a private message right now with the details. Thanks!

8

(post deleted by author)

9

Publishing the failures will make this unusually useful. I would keep a small matrix for GPU, driver, system RAM, model, quantization, first-token latency, sustained tokens per second, peak working set, and whether the run completed. A checksum plus the exact configuration used for each result will also make community reports much easier to trust.

10

I agree alikhaters. That is very close to what I want the external qualification phase to produce.

I’m treating failures as useful results rather than discarding them, particularly because one of the goals is to establish the practical hardware boundary for running these models.

The current qualification workflow already produces machine-specific JSON results, and I’m standardizing the external results around a matrix containing:

  • GPU / VRAM
  • GPU driver
  • system RAM
  • model
  • quantization/cache configuration
  • first-token/startup latency
  • sustained tokens/sec
  • memory usage
  • completion/failure status
  • failure stage and error information
  • StreamAI/build version
  • configuration/profile used

I’m also providing SHA-256 checksums for the distributed tester kit and installer so testers can verify exactly which build produced their results.

Your suggestion about publishing the failures is especially important. For example, my first external low-VRAM test didn’t reach inference, but it exposed inconsistencies in the machine-ID and clean-install model/cache paths that weren’t visible on my development machine. Those findings resulted directly in fixes for the next beta.

Once I have enough results across AMD, NVIDIA and Intel hardware, I’d like to publish an anonymized compatibility matrix showing both successful and unsuccessful configurations rather than only the best-performing systems.

I like the suggestion of recording the exact configuration alongside every result as well. That should make the numbers much more reproducible and useful to others.

Thanks for the feedback.

11

StreamAI Preliminary Compatibility Matrix — First External 2 GB VRAM Inference Success

Quick update on the StreamAI controlled beta: we now have our first successful external inference test on a 2 GB discrete GPU.

A beta tester, Logos-Adri, tested StreamAI 0.3.9.12 on an NVIDIA GeForce GTX 1050 with 2 GB VRAM. The engine initialized successfully and completed local Qwen2.5 1.5B inference.

Preliminary compatibility results

GPU VRAM System RAM Profile Status Observed Performance
AMD Radeon RX 570 4 GB 32 GB Fast Qualified ~1.11 tok/s
NVIDIA GeForce GTX 1050 2 GB Pending tester confirmation Ultra-Low-VRAM Fast Experimental PASS ~0.99 tok/s
Intel Iris Xe 2 GB reported ~16 GB Earlier test profile Unqualified / needs retest —
Intel UHD 770 2 GB reported ~16 GB Earlier test profile Unqualified / needs retest —

Important: “Unqualified” does not mean incompatible. It means that configuration does not yet have enough evidence to be classified as qualified.

Similarly, I’m deliberately calling the GTX 1050 result Experimental PASS, not Qualified, until the structured qualification suite is completed.

Why the GTX 1050 result is interesting

This is a direct before/after test on the same 2 GB machine.

StreamAI 0.3.9.11

  • Clean-machine provisioning: PASS
  • Automatic model download: PASS
  • Model/cache preparation: PASS
  • Engine initialization: FAIL — insufficient GPU memory
  • Inference: not reached

The failure occurred while attempting an approximately 445 MiB GPU allocation.

Rather than classifying the GTX 1050 as incompatible, I used that failure to create a separate experimental low-VRAM execution profile in StreamAI 0.3.9.12.

StreamAI 0.3.9.12

  • GTX 1050 detected: PASS
  • VRAM detected: 2.0 GB
  • DirectML: Ready
  • Low-VRAM profile selected: PASS
  • Engine initialization: PASS
  • Local inference: PASS

Two external generations produced approximately:

0.9852 tok/s — TTFT 1.9166 s

0.9926 tok/s — TTFT 1.6131 s

The tester successfully generated complete responses locally, including the standard test prompt:

Explain how a black hole forms.

4 GB regression test

Before sending 0.3.9.12 externally, I also tested it on the existing Radeon RX 570 4 GB qualification machine.

StreamAI retained its existing qualified Fast configuration and completed inference at:

1.1087 tok/s — TTFT 1.0677 s

So the new experimental 2 GB path did not require replacing or weakening the existing qualified 4 GB path.

What happens next

The GTX 1050 tester is now running StreamAI’s structured qualification suite.

I’m looking for more than a successful screenshot before calling the profile qualified. The qualification results will give us standardized evidence for repeated generation, context handling, determinism/lifecycle behavior, and the machine/runtime configuration.

If those tests pass, the matrix will be updated from:

GTX 1050 2 GB — Experimental PASS

to:

GTX 1050 2 GB — Qualified

If something fails, I’ll publish that result too.

Publishing failures

Several people here suggested publishing unsuccessful configurations as well as successful ones. I agree.

Going forward, the compatibility matrix will distinguish:

Qualified — completed the defined qualification suite
Experimental PASS — actual inference completed; qualification pending
Unqualified / needs retest — insufficient evidence or qualification wasn’t completed
Failed — a specific tested configuration failed, with the failure stage recorded

This distinction matters because a failed configuration doesn’t necessarily mean the hardware itself is incapable of running StreamAI. The GTX 1050 is a good example: the earlier configuration failed, while a subsequent memory-bounded profile successfully reached inference on the same hardware.

Future matrix entries will include as much verified information as practical: GPU, VRAM, system RAM, StreamAI version/profile, context, TTFT, sustained tokens/sec and failure stage.

There is no automatic telemetry collection for these beta tests. Qualification results remain local and testers choose whether to return them.

More testers welcome

I’m still looking for Windows users interested in testing StreamAI on low-VRAM hardware, especially:

  • NVIDIA 2–4 GB GPUs
  • AMD 2–8 GB GPUs
  • Intel Arc / Iris Xe / UHD graphics
  • laptop GPUs
  • integrated graphics
  • older or unusual DirectML-capable hardware

Windows 10/11 and a DirectML-capable GPU are the primary requirements. 16 GB+ system RAM is preferred, although lower-memory systems are also useful for establishing the actual operating boundaries.

The objective is not simply to collect successful runs. I want to establish, with reproducible real-hardware testing, how far local LLM inference can be pushed when GPU memory is the primary constraint.

A big thank-you to Logos-Adri for testing both builds. The original GTX 1050 failure provided the information that led directly to the experimental 2 GB profile and this successful inference result.

I’ll update the matrix again when the structured GTX 1050 qualification results are available.