Model library / ibm-granite

granite-4.2-8b

Granite-4.2-8B is an 8.8 billion-parameter decoder-only language model from IBM, designed for reasoning-intensive tasks.

8.8B

Parameters in total

131,072

Context length in tokens, as published

16 GiB

Weights at the released precision

apache-2.0

Licence declared on the repository

Overview

What this model is.

Granite-4.2-8B is a dense transformer model released by IBM under the Apache 2.0 licence. IBM describes it as a mid-size reasoning model for enterprise and research use, offering built-in chain-of-thought capabilities and flexible thinking modes to handle math, coding, and multi-step problems.

The model contains 8.8 billion parameters, uses bfloat16 precision, and supports a 131,072-token context window. It employs grouped query attention with 32 heads, SwiGLU-based feed-forward layers, and RMSNorm. The lack of expert layers means it runs as a single dense network, suitable for deployment on hardware that can handle its size and memory requirements.

Specification

The published shape.

Repository	ibm-granite/granite-4.2-8b
Publisher	ibm-granite
Published	7 August 2026
Architecture	GraniteForCausalLM
Model type	granite
Parameters	8,791,592,960
Layers	40
Hidden size	4,096
Attention heads	32, 8 key/value heads
Context length	131,072 tokens
Vocabulary	100,352 tokens
Weight format	bfloat16
Licence	apache-2.0
Base model	ibm-granite/granite-4.1-8b-base

Memory

How much VRAM the weights need.

Parameter count multiplied by the bytes each format uses. The key-value cache comes on top.

Precision	Weights	Notes
bf16	16.4 GiB	Full precision as released
fp8	8.2 GiB	8-bit, near-lossless on most models
int4	4.1 GiB	4-bit, smallest footprint

The cache costs 160 KiB per token at 16-bit, so the full 131,072-token context of one request needs about 20.0 GiB. Concurrency multiplies that number, not the weights.

Hardware

Which card runs it.

At the released precision, with 90% of the card given to the server and 2.5 GiB kept for the runtime. The context column is what is left for the cache on that setup.

GPU	Cards needed	Free for cache	Context it holds
RTX 4090 24GB	1	2.7 GiB	about 17,854 tokens
RTX 5090 32GB	1	9.9 GiB	about 65,040 tokens
L40S 48GB	1	24.3 GiB	the full 131,072 tokens
A100 80GB	1	53.1 GiB	the full 131,072 tokens
H100 80GB	1	53.1 GiB	the full 131,072 tokens
RTX PRO 6000 96GB	1	67.5 GiB	the full 131,072 tokens
H200 141GB	1	108.0 GiB	the full 131,072 tokens

Serving

Running it yourself.

A vLLM launch line for the shape above. Check the model card for a runtime the publisher recommends.

vllm serve ibm-granite/granite-4.2-8b \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.90

Questions

The things people ask about this model.

How much GPU memory does granite-4.2-8b need?

The weights take about 16 GiB at the released precision. That fits on one RTX 4090 24GB, which leaves roughly 3 GiB for the key-value cache.

What licence does granite-4.2-8b use?

The repository declares apache-2.0. Read the licence text before commercial use: the name alone does not tell you what is allowed.

How long a context does it support?

The configuration allows 131,072 tokens. Whether the whole window is usable depends on the memory left for the cache, which the hardware table works out per card.

Can I use it through an API instead of hosting it?

This page is a reference, and Aiqre does not serve this model today. We deploy open-weight models on dedicated EU hardware on request, so if you want an OpenAI-compatible endpoint for it, email [email protected] with the model name and roughly what volume you expect.

Also in the library

Want this model on a dedicated EU GPU?

Tell us the model and roughly what volume you expect. We reply with a price and an OpenAI-compatible endpoint, with your prompts never stored.

[email protected] Browse the library