Hardware Lab: what AI can your computer run?
Work out whether a model fits before you download it, see what each size of graphics card can handle, and check real speeds measured on our own computers.
Pick a model size to see whether it fits.
Model pages on download sites list the size as a number followed by B, such as 8B. These are estimates for language models. Image and video tools are covered on each tool's page.
What each graphics card can run
- No graphics card39 of 59 local tools run
- 8 GB graphics card52 of 59 local tools run
- 12 GB graphics card55 of 59 local tools run
- 16 GB graphics card56 of 59 local tools run
- 24 GB graphics card59 of 59 local tools run
Tested on our own computers
These are real runs on the computers we use, not figures from a spec sheet. Each was a single run, so treat them as examples rather than guarantees.
| Computer | Model | Run with | Task | Speed |
|---|---|---|---|---|
| GMKtec EVO-X2 with an RTX 3090 | DeepSeek V4.1 Flash 2.00-bit set, 257 GB on disk | Strata | Probability problems 5 of 5 answers correct | 23.8 tokens/s |
| GMKtec EVO-X2 with an RTX 3090 | DeepSeek V4.1 Flash 2.00-bit set, 257 GB on disk | Strata | Power plant scheduling puzzle 6 of 6 checks correct | 27.3 tokens/s |
| GMKtec EVO-X2 with an RTX 3090 | DeepSeek V4.1 Flash 2.00-bit set, 257 GB on disk | Strata | Finding bugs in database code Found seven bugs | 26.0 tokens/s |
| GMKtec EVO-X2 with an RTX 3090 | DeepSeek V4.1 Flash 2.25-bit set, 274 GB on disk, tuned for accuracy | Strata | Plain-language explanation (two runs) About 620 tokens each | 20.2 and 24.7 tokens/s |
| GMKtec EVO-X2 with an RTX 3090 | DeepSeek V4.1 Flash 2.25-bit set, 274 GB on disk, tuned for accuracy | Strata | Writing a Python script 2,586 tokens | 21.8 tokens/s |
| GMKtec EVO-X2 with an RTX 3090 | DeepSeek V4.1 Flash 2.25-bit set, 274 GB on disk, tuned for accuracy | Strata | Maths word problem 496 tokens | 22.6 tokens/s |
| HP Z840 with an RTX 4070 | Qwen 3.8 Flash Pack size not recorded | Strata | Long reasoning answer | 35.7 tokens/s |
| HP Z840 with an RTX 4070 | Qwen 3.8 Flash Pack size not recorded | Strata | Physics problem | 45.5 tokens/s |
GMKtec EVO-X2 with an RTX 3090
- AMD Ryzen AI Max+ 395, 16 cores
- 128 GB memory
- NVIDIA RTX 3090 with 24 GB, attached as an external card over USB4 (runs as a PCIe 4.0 x4 link)
- 4 TB WD_BLACK SN850X NVMe drive
- Pop!_OS 24.04
HP Z840 with an RTX 4070
- Two Intel Xeon E5-2650 v3, 20 cores in total
- 128 GB memory
- NVIDIA RTX 4070 with 12 GB
- 1 TB NVMe drive
- Released in 2014, so the processors and memory are a decade old
What this shows. A graphics card only helps as far as the model fits on it. The DeepSeek model above takes 257 GB on disk, far more than any home graphics card holds, so most of it runs from main memory and the SSD, and it still reaches a comfortable reading speed. The smaller Qwen model fits its card better and runs faster on a much older computer.
Which DeepSeek set we use. We have run both the 2.00-bit and 2.25-bit sets. The 2.25 set is noticeably more accurate in our use, and after a lot of trial and error with the launch settings it averages 22 tokens a second, a little behind the 2.00 set, so it is the one we keep. Strata's own measurements agree: the 2.25 set loses about half as much quality against the original model as the 2.00 set.
The settings that worked best for us. These launch settings were the fastest of the many combinations we tried on the GMKtec with the RTX 3090, on Strata 0.2.13. The 2.25 results above were also run for the fewest mistakes rather than top speed: thinking set to High, temperature 0.10, top-p 0.80 and top-k 10. Lighter settings answer everyday chats faster. Your best settings may differ on other hardware.
strata launch \
--model ist-das-2.25bpw \
--family deepseek41 \
--context 128k \
--vision off \
--speed full \
--set cpu-threads=32 \
--set dspark=on \
--set dspark-conf=0.65 \
--tune skip \
--access local \
--port 8080 \
--no-tuiCommon questions
How much memory does an AI model need?
A rough rule: take the model's size in billions of parameters, multiply by 0.67, and add 1.5. That is the memory in GB needed at the common Q4 quality level. An 8 billion model needs about 7 GB and a 32 billion model about 23 GB.
What is the difference between RAM and VRAM?
RAM is your computer's main memory. VRAM is the memory on the graphics card. AI models run fastest when they fit entirely in VRAM. If a model is too big, the rest spills into RAM and it runs several times slower.
What does tokens per second mean?
A token is a piece of a word, roughly three quarters of a word on average. At 25 tokens per second a model writes about as fast as most people read. Below 5 it feels slow, and above 40 it feels instant.
Can I run a model that is bigger than my memory?
Sometimes. Engines such as Strata keep what fits on the graphics card, put more in main memory, and read the rest from a fast SSD. We ran a model that takes 257 GB on disk on a computer with a 24 GB graphics card and 128 GB of memory at 24 to 27 tokens per second.