Model Gallery

175 models from 1 repositories

Filter by type:

Filter by tags:

qwen3.8-flash-next-coder-iq1_m
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in ISTA-DASLab's Coder release (IQ1_M): half of the experts (256 of 512 per layer) are kept, chosen on code, agent and image data, so it is smaller and faster but weaker outside code and in languages other than English (Chinese and other CJK text included) (about 59 GB to download). It reads images through the vision projector published for the original model, and the Strata backend parses its reasoning and tool calls. Needs about 32 GB of RAM (23 GB of experts in RAM) with a suitable GPU, and fast SSD storage: the 29 GB n-gram table stays on disk. Needs a GPU the strata backend supports; NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the experimental strata backend. On the first load the strata backend builds the Strata pack from the GGUF files (a few minutes, a few GB of extra disk); a finished pack is never rebuilt. The experts are read from the GGUF files in place (mapped); add --resident-experts to engine_args to copy them into RAM when there is room. Licensed under the Qwen Community License 1.0 (inherited from Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work Assistant business needs a licence from Qwen. Read the LICENSE file in the repository first. The GGUF files are the Coder GSQ-RCO quantization by ISTA-DASLab; Strata is by Niko1221. The MTP draft layer is not configured here.

Repository: localaiLicense: qwen-community-1.0

qwen3.8-flash-next-gyro-s
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in agentionai's Gyro-S quantization (trellis-coded experts), the smallest option, for machines with little memory and lower in quality (about 59 GB to download). It reads images through the vision projector published for the original model, and the Strata backend parses its reasoning and tool calls. Needs about 28 GB of VRAM and RAM together for the weights. Half of the file is the n-gram table, which stays on disk and is read for every token, so keep it on a fast local SSD. Needs a GPU the strata backend supports; NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the experimental strata backend. On the first load the strata backend builds the Strata pack from the GGUF files (a few minutes, a few GB of extra disk); a finished pack is never rebuilt. The experts are read from the GGUF files in place (mapped); add --resident-experts to engine_args to copy them into RAM when there is room. Licensed under the Qwen Community License 1.0 (inherited from Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work Assistant business needs a licence from Qwen. Read the LICENSE file in the repository first. The GGUF file is the Gyro quantization by agentionai; the vision projector is ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP draft layer is not configured here.

Repository: localaiLicense: qwen-community-1.0

qwen3.6-35b-a3b-ud-q4_k_xl-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active parameters, in Unsloth's 4-bit (Unsloth Dynamic), the smallest GGUF with the MTP layer (about 24 GB to download). It reads images through Unsloth's vision projector, and the Strata backend parses its reasoning and tool calls. Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not run this model) and about 32 GB of RAM: the engine keeps every expert it can in VRAM and computes the rest on the CPU. Runs only on the experimental strata backend. On the first load the strata backend exports the tokenizer and chat template from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP draft layer inside the file is used for speculative decoding. Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files are the Unsloth quantization; Strata is by Niko1221.

Repository: localaiLicense: apache-2.0

bge-small-en-v1.5-f16
BGE-small-en-v1.5 is a 33M English embedding model for semantic search and retrieval. This F16 GGUF build produces 384-dimensional embeddings with a 512-token context. For retrieval, prefix queries with "Represent this sentence for searching relevant passages: "; leave passages unprefixed.

Repository: localaiLicense: mit

bge-small-en-v1.5-q8-0
BGE-small-en-v1.5 is a 33M English embedding model for semantic search and retrieval. This Q8_0 GGUF build produces 384-dimensional embeddings with a 512-token context. For retrieval, prefix queries with "Represent this sentence for searching relevant passages: "; leave passages unprefixed.

Repository: localaiLicense: mit

bge-small-en-v1.5-q4-k-m
BGE-small-en-v1.5 is a 33M English embedding model for semantic search and retrieval. This Q4_K_M GGUF build produces 384-dimensional embeddings with a 512-token context. For retrieval, prefix queries with "Represent this sentence for searching relevant passages: "; leave passages unprefixed.

Repository: localaiLicense: mit

qwen3.8-flash-next-q4
Qwen3.8-Flash-Next is Qwen's 125B-parameter, 6B-active experimental vision-language mixture-of-experts model. It targets agentic coding, reasoning, tool use, and long-context workloads with a native 262K-token context window. This default entry uses Unsloth's UD-Q4_K_XL GGUF and BF16 vision projector. Linked variants offer Q8_0 and AtomicChat's smaller IQ4_XS and Q4_K_M builds with a separate n-gram table shard.

Repository: localaiLicense: other

qwen3.8-27b-dflash2
Qwen3.8-27B with DFlash 2 speculative decoding on the llama.cpp backend. DFlash 2 is a small drafter model that guesses a whole block of upcoming tokens in one pass. It keeps several candidates for each position and picks one coherent path through them. The full Qwen3.8-27B model then checks the guessed tokens in a single step and keeps the ones it agrees with, so generation is faster and the output is the same as without the drafter: greedy output is identical, and sampling keeps the target's distribution. The drafter is not a standalone model; it only runs paired with the target. This entry downloads the Q4_K_M target (about 19 GB) and the Q4_K_M drafter (about 1.1 GB). Any standard Qwen3.8-27B GGUF quantization can serve as the target. The block size is set to 7 draft tokens. Flash attention is required and is enabled. A GPU is recommended. This entry is text only.

Repository: localaiLicense: apache-2.0

huihui-qwen3.8-27b-abliterated
Huihui Qwen3.8 27B is an abliterated vision-language model published by huihui-ai. This BF16 GGUF build includes the shared BF16 vision projector and enables MTP speculative decoding through llama.cpp. Q4_K, Q5_K, Q6_K, and Q8_0 variants are available as smaller downloads.

Repository: localaiLicense: apache-2.0

ornith-1.5-35b-a3b-apex-compact
Ornith-1.5-35B-A3B in the smaller APEX Compact GGUF format, with the shared BF16 vision projector for multimodal prompts.

Repository: localaiLicense: mit

thomson-1.0-small-q4
Thomson-1.0-Small is a 35B-parameter mixture-of-experts model with about 3B active parameters. It focuses on legal, tax, journalism, research, reasoning, tool use, and document processing. It supports text and image input with a native context window of 262K tokens. This default entry uses the Q4_K_M GGUF and BF16 vision projector. A higher-quality Q8_0 model is available as a variant.

Repository: localaiLicense: polyform-strict-1.0.0

thomson-1.0-small-q8
Thomson-1.0-Small in the higher-quality Q8_0 GGUF format, with the shared BF16 vision projector for multimodal prompts.

Repository: localaiLicense: polyform-strict-1.0.0

qwen3.8-27b-gsq-rco-iq2-xs
Qwen3.8-27B with ISTA DASLab's smallest GSQ-RCO mixed quantization. The 2.50-bit IQ2_XS model uses 8.4 GB for the weights and retains the shared BF16 vision projector.

Repository: localaiLicense: apache-2.0

qwen3.8-27b-gsq-rco-iq3-xxs
Qwen3.8-27B with ISTA DASLab's 3.00-bit GSQ-RCO IQ3_XXS mixed quantization. The 10.1 GB model offers a smaller alternative to IQ3_S and retains the shared BF16 vision projector.

Repository: localaiLicense: apache-2.0

qwen3.8-2b-q4
Qwen3.8-2B is Empero AI's smallest Qwen3.8 reasoning distillation. It uses the Qwen3.5-2B architecture and targets mathematics, instruction following, tool use, and edge deployment with a native 262K-token context window. This default entry uses Q4_K_M weights; a higher-quality Q8_0 build is available as a variant.

Repository: localaiLicense: apache-2.0

nemotron-3.5-lightning-30b-a3b-q4
NVIDIA Nemotron 3.5 Lightning is a text-only hybrid Mamba-2, attention, and mixture-of-experts model with 30B total parameters and 3B active parameters. It targets reasoning, coding, tool use, multilingual chat, and long-context agent workflows, with a context window of up to one million tokens. This entry uses the official Q4_K_M GGUF. Automatic variant selection can choose the smaller NVFP4 build or the higher-quality Q8_0 build when it fits.

Repository: localaiLicense: openmdw-1.1

nemotron-3.5-lightning-30b-a3b-nvfp4
NVIDIA Nemotron 3.5 Lightning 30B-A3B in the official NVFP4 GGUF format. This is the smallest linked build and retains the model's reasoning, coding, tool-use, multilingual, and long-context capabilities.

Repository: localaiLicense: openmdw-1.1

muse-glimmer-30b
Muse Glimmer is Meta Superintelligence Labs' Apache-2.0 dense 30B model for autonomous agentic work, coding, tool use, long-horizon reasoning, and multimodal understanding. It supports more than 100 languages, interleaved text and image input through its 1.8B-parameter perception encoder, and a 131K-token context window. This entry uses the publisher's higher-quality dynamic K-quant GGUF and official quantized vision projector. Automatic variant selection can use the smaller 17 GB quantization or a DFlash-accelerated build when it fits.

Repository: localaiLicense: apache-2.0

muse-glimmer-30b-17gb
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized perception encoder. It preserves the model's agentic, coding, tool-use, multilingual, and image-understanding capabilities for hosts with less memory than the dynamic quantization requires.

Repository: localaiLicense: apache-2.0

muse-glimmer-30b-17gb-dflash
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized perception encoder and DFlash drafter. This is the lowest-memory published build that retains image understanding and block-speculative decoding. Flash attention is enabled for the DFlash path.

Repository: localaiLicense: apache-2.0

deepseek-v4-flash-0731
# DeepSeek-V4-Flash-0731 Technical Report👁️ ## Introduction **DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached. DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available. Notes: 1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`. 2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems. ## Chat Template ...

Repository: localaiLicense: mit

Page 1