Qwen4 architecture surfaces through Qwen3.8 and Transformers

Qwen's next architecture is already visible in public tooling, even though the Qwen4 family has not received a conventional launch announcement.

NVIDIA published an August 26 technical report on Qwen3.8-Flash-Next. It describes the model as a 176B-parameter multimodal mixture-of-experts system and documents experiments on GB300 NVL72 hardware for agentic coding. NVIDIA also links a NeMo AutoModel page for the model.

The unusual part is the routing scale. NVIDIA says the model has a 125B-parameter main model plus 51B N-gram embeddings, while activating 6B parameters per token. It has a native 262,144-token context window that can be extended to 1 million tokens with YaRN. These figures explain why the release matters for serving, but they do not independently prove a benchmark advantage.

A separate official signal appeared in Hugging Face Transformers v5.16.0. The stable release adds Qwen4-Exp. Its notes describe a hybrid text and multimodal architecture that builds on Qwen3.5.

The release notes name three key components: GatedResidual, Qwen Sparse Attention and Per-Layer Embedding. They also say Qwen4-Exp combines Gated DeltaNet with sparse attention for long-context inference. This is implementation support in a widely used library, not only a codename in a leak post.

The public evidence points to architecture and software support, not a full Qwen4 product launch. Qwen has not announced a broad Qwen4 lineup, pricing or a general availability date in the cited materials. Performance claims around Qwen3.8-Flash-Next come from vendor-published benchmarks and should be read with that limitation.