Intel's Arc Pro B70 graphics card is positioning itself as an affordable alternative for running large language models locally, with four cards costing under $4,000 and delivering 128 gigabytes of combined memory. Recent testing from Puget Systems reveals how this budget-focused approach compares to premium alternatives when running production-quality AI inference workloads like chatbots and image generation.

How Does Intel's Pricing Strategy Compare to NVIDIA?

The Arc Pro B70 costs $949 per card with 32 gigabytes of video memory. Four of these cards total approximately $3,800 and provide 128 gigabytes of aggregate memory. By contrast, NVIDIA's GeForce RTX 5090 costs $1,999 per card and also has 32 gigabytes of memory. Two RTX 5090 cards would deliver 64 gigabytes of memory for roughly $4,000. This means Intel's four-card setup offers twice the total memory at roughly the same price point.

The tradeoff involves raw speed. NVIDIA's RTX 5090 has memory bandwidth of 1,792 gigabytes per second, nearly three times faster than the Arc Pro B70's 608 gigabytes per second. NVIDIA's software ecosystem also supports advanced compression techniques called AWQ and GPTQ quantization, which Intel's XPU backend does not yet support. These techniques allow models to run faster and use less memory.

What AI Models Can Actually Run on This Hardware?

A single Arc Pro B70 card comfortably runs smaller language models with 8 billion parameters. This includes models like Llama 3.1 8B and DeepSeek R1 Distill 8B. For larger models in the 27 billion to 35 billion parameter range, which represent the current sweet spot for serious local inference work, users need multiple cards. The four-card configuration tested makes this tier accessible.

The testing revealed specific performance characteristics across different model sizes:

• 3 Billion Parameter Models: Qwen2.5 3B achieved 72.9 tokens per second with a 48-millisecond response time, delivering output nearly instantaneously at roughly 3,300 words per minute.
• 8 Billion Parameter Models: DeepSeek R1 Distill 8B reached 66.9 tokens per second with exceptionally low latency of 15 milliseconds between tokens, making it suitable for interactive chat applications.
• Reasoning Models: Qwen3 8B, which generates internal reasoning tokens before visible output, delivered 34.7 tokens per second with 69-millisecond response time, fully suitable for interactive use once measured with the correct extended window.

Models requiring bfloat16 precision, such as the Gemma family, cannot currently run on this hardware through vLLM, though a future software update could address this limitation.

How Does Performance Scale Across Multiple Users?

The testing examined how performance changes when multiple users access the system simultaneously. For the Qwen2.5 3B model, throughput scaled from 73 tokens per second with a single user to 280 tokens per second with four concurrent users and 526 tokens per second with eight concurrent users. This represents a 7.2 times increase in throughput while latency barely increased from 4.6 seconds to 5.2 seconds. The 3B model parallelizes almost perfectly on a single card.

DeepSeek R1 8B showed similar scaling characteristics, reaching 486 tokens per second at eight concurrent users, a 7.3 times increase from the single-user baseline of 66.9 tokens per second. The 15-millisecond inter-token latency remained exceptionally smooth even under load.

What Are the Real-World Power and Cost Implications?

The testing setup included power monitoring to calculate real-world operating costs. The four-card configuration draws 230 watts per GPU under load, with an idle floor of approximately 190 watts across the three unused cards during single-GPU tests. This allows for cost-per-token calculations against cloud-based API pricing.

The practical implication is significant for organizations running inference workloads continuously. A four-card setup under $4,000 can handle the 27 billion to 35 billion parameter models that represent the current production standard, making local inference economically viable for many use cases that would otherwise require expensive cloud API calls.

What Technical Barriers Remain?

Setting up multi-GPU inference on Intel's Battlemage architecture requires specific configuration steps that go beyond typical plug-and-play installation. The process involves driver conflict resolution, fork-safety workarounds, and PCIe topology configuration. Intel's LLM Scaler container addresses some of these challenges by adding INT4 and FP8 online quantization as well as GPTQ support on the XPU backend, capabilities not yet available in upstream vLLM.

The current testing used full FP16 precision weights, meaning models consume their maximum memory footprint. Quantized inference, which compresses models to use less memory and run faster, remains an area for future benchmarking. This represents a significant opportunity for performance improvement once software support matures.

How to Set Up Multi-GPU Inference on Intel Arc Pro B70

• Driver Installation: Use Intel's PPA kobuk-team/intel-graphics driver version 26.09.x or later to ensure proper GPU support and multi-GPU communication capabilities.
• Container Configuration: Deploy Intel's purpose-built llm-scaler-vllm container version 0.14.0-b8.2.1 with oneCCL for inter-GPU communication to handle tensor parallelism across multiple cards.
• PCIe Topology Verification: Confirm your workstation's PCIe topology supports the number of GPUs you plan to install, as this affects inter-GPU communication bandwidth and stability.
• Model Selection: Start with 8 billion parameter models on a single card, then scale to 27 billion to 35 billion parameter models using the full four-card configuration for production workloads.
• Quantization Planning: Prepare for future quantization support by selecting models compatible with INT4 and FP8 compression, which will improve performance once fully integrated into the software stack.

The Arc Pro B70 represents Intel's deliberate positioning of the card as an AI-first GPU rather than a general-purpose professional graphics processor. For organizations seeking to run production-quality language models locally without the per-token costs of cloud APIs, the four-card configuration offers a compelling cost-to-memory ratio, provided users can navigate the current software setup complexity.