gemma-3-12b-it-qat-6bit

mlx-community/gemma-3-12b-it-qat-6bit

Vision + text · transformers

6-bit Open on Hugging Face

Parameters

—

Weights (est.)

—

Mac memory (est.)

—

Size is estimated from the parameter count and quantisation. Memory allows a fifth extra for context plus 2 GB for the system; long contexts need more.

Run it

With mlx-vlm on an Apple silicon Mac:

pip install mlx-vlmpython -m mlx_vlm.generate --model mlx-community/gemma-3-12b-it-qat-6bit --image photo.jpg --prompt "Describe this image"

Details

Base model: OpenGVLab/InternVL3-1B-Instruct (finetune)
Vendor: OpenGVLab
Task: Vision + text
Quantisation: 6-bit
Licence: qwen
Downloads: 24 in the last 30 days · 732 all time · 0 likes
Published: 2025-04-15 · updated 2025-04-21 (1y ago)
Languages: multilingual
Tags: gemma3, internvl, custom_code

Other MLX builds of OpenGVLab/InternVL3-1B-Instruct

Build	Quantisation	Weights	Mac memory	Downloads
gemma-3-4b-it-qat-4bit	4-bit	≈ 3.0 GB	8 GB+	42K
InternVL3-8B-3bit	3-bit	≈ 4.0 GB	8 GB+	36
InternVL3-9B-4bit	4-bit	≈ 4.9 GB	8 GB+	56
gemma-3-12b-it-qat-bf16	bf16	≈ 26.4 GB	32 GB+	30
gemma-3-27b-it-qat-4bit	4-bit	—	—	20K
gemma-3-12b-it-qat-4bit	4-bit	—	—	19K
Kimi-VL-A3B-Thinking-8bit	8-bit	—	—	260
Kimi-VL-A3B-Thinking-6bit	6-bit	—	—	147
InternVL3-8B-6bit	6-bit	—	—	102
InternVL3-8B-8bit	8-bit	—	—	85
InternVL3-14B-4bit	4-bit	—	—	81
gemma-3-27b-it-qat-8bit	8-bit	—	—	79
InternVL3-38B-4bit	4-bit	—	—	62
gemma-3-12b-it-qat-3bit	3-bit	—	—	56
gemma-3-27b-it-qat-3bit	3-bit	—	—	50
InternVL3-8B-bf16	bf16	—	—	43
InternVL3-38B-8bit	8-bit	—	—	36
InternVL3-78B-4bit	4-bit	—	—	34
gemma-3-12b-it-qat-8bit	8-bit	—	—	32
gemma-3-27b-it-qat-6bit	6-bit	—	—	32
InternVL3-38B-6bit	6-bit	—	—	29
InternVL3-14B-8bit	8-bit	—	—	27
InternVL3-38B-bf16	bf16	—	—	25
gemma-3-12b-it-qat-6bit (this one)	6-bit	—	—	24
InternVL3-9B-8bit	8-bit	—	—	20
InternVL3-14B-bf16	bf16	—	—	20
InternVL3-14B-6bit	6-bit	—	—	18
InternVL3-9B-6bit	6-bit	—	—	13
InternVL3-9B-bf16	bf16	—	—	13