Mistral-Small-3.1-24B-Instruct-TXT-2503-3bit-AWQ

mlx-community/Mistral-Small-3.1-24B-Instruct-TXT-2503-3bit-AWQ

Text generation

3-bit Open on Hugging Face

Parameters

3.0B

Weights (est.)

≈ 2.5 GB

Mac memory (est.)

8 GB+

Size is estimated from the parameter count and quantisation. Memory allows a fifth extra for context plus 2 GB for the system; long contexts need more.

Run it

With mlx-lm on an Apple silicon Mac:

pip install mlx-lmmlx_lm.generate --model mlx-community/Mistral-Small-3.1-24B-Instruct-TXT-2503-3bit-AWQ --prompt "Hello"

Details

Base model: anthracite-core/Mistral-Small-3.1-24B-Instruct-2503-HF (finetune)
Vendor: anthracite-core
Task: Text generation
Quantisation: 3-bit
Licence: mit
Downloads: 24 in the last 30 days · 234 all time · 0 likes
Published: 2025-05-20 · updated 2025-05-20 (1y ago)
Tags: mistral