Date190126General agent

Jan-v3-4B

A compact 4B model distilled from a larger teacher, with strong instruction following, built as a base you can fine-tune.

2 min read

Overview

Parameters
4B (3.6B non-embedding)
Base model
Qwen3-4B-Instruct-2507
Context length
262,144 tokens (native)
Architecture
36 layers, 32 Q heads / 8 KV heads (GQA)
License
Apache 2.0

18.0

Aider polyglot · % pass rate (strict)

Jan-v3-4B-base-instruct is a compact 4B parameter model created via post-training distillation from a larger teacher model. It is designed as an ownable, general-purpose base that is straightforward to fine-tune, with strong instruction following out of the box.

Try it in Jan

Capabilities

  • Instruction following: Strong compliance with user instructions out of the box
  • Fine-tuning base: Designed as a starting point for downstream fine-tuning
  • Coding assistance: Effective lightweight coding help (a dedicated code variant, Jan-Code-4B, is also available)
  • General purpose: Distillation preserves broad capabilities across standard benchmarks

Performance

Aider Benchmark (Polyglot Code Editing)

Jan-v3-4B leads its weight class on the Aider polyglot benchmark with a strict pass rate of 18, nearly 50% higher than the next best model (Qwen3-4B-Thinking at 12.1):

BenchmarkJan-v3-4B-base-instructQwen3-4B-Thinking-2507Ministral-3-8B-Instruct-2512Olmo-3-7B-ThinkPhi-4-Mini-InstructLFM2.5-1.2B-ThinkingLFM2-8B-A1BNemotron-Mini-4B-Instruct
Aider polyglot (Code editing · % pass rate (strict))18.012.16.83.31.91.81.40.9
Jan-v3 leads with high efficiency.

General Performance Comparison

Jan-v3-4B-base-instruct outperforms Qwen3-4B-Instruct-2507 (its base) across coding, math, and instruction-following benchmarks while remaining competitive on reasoning:

GroupBenchmarkQwen3-4B-Instruct-2507Jan-v3-4B-base-instruct
General reasoningAIME 20250.470.47
MMLU0.72600.7261
MMLU Pro0.68260.6734
IF-Bench (Prompt)0.30950.3265
IF-Bench (Instruct)0.32540.3463
EQBench0.84810.8469
Creative0.80490.7960
SimpleQA63.767.3
Math & STEMGSM8K0.88700.9219
GPQA0.45540.4551
GPQA-Diamond0.500.5101
CodingAider-polyglot12.918.0
Jan-v3 is better at coding, math, and instruction following.

Requirements

  • Memory:
    • Minimum: 8GB RAM (with Q4 quantization)
    • Recommended: 16GB RAM (with Q8 quantization)
  • Hardware: CPU or GPU
  • API Support: OpenAI-compatible at localhost:1337

Using Jan-v3-4B

Quick Start

  1. Download Jan Desktop
  2. Click Open in Jan above, or open the Hub and search for Jan-v3-4B-base-instruct
  3. Choose a variant that fits your hardware, then click Download
  4. Once downloaded, select Jan-v3-4B-base-instruct from the model selector in any chat
  5. Start chatting — no additional configuration needed

Demo

Try it live at chat.jan.ai (opens in a new tab)

Deployment Options

Using vLLM:


vllm serve janhq/Jan-v3-4B-base-instruct \
--host 0.0.0.0 \
--port 1234 \
--enable-auto-tool-choice \
--tool-call-parser hermes

Using llama.cpp:


llama-server --model Jan-v3-4B-base-instruct-Q8_0.gguf \
--host 0.0.0.0 \
--port 1234 \
--jinja \
--no-context-shift


temperature: 0.7
top_p: 0.8
top_k: 20

What Jan-v3-4B Does Well

  • Long context: Native 262k token context window for handling large documents
  • Instruction following: Reliable and consistent response to user instructions
  • Fine-tuning: Designed to be a strong, ownable starting point for specialization
  • General tasks: Broad capabilities preserved through distillation from a larger model

Limitations

  • Model size: 4B parameters limit complex multi-step reasoning compared to larger models
  • Specialized domains: For coding specifically, consider Jan-Code-4B

Downloads

GGUF quantizations

  • Q4_K_M: Good balance of size and quality
  • Q5_K_M: Better quality, slightly larger
  • Q8_0: Highest quality quantization (recommended)

Talk to us