Filter by type:
Filter by tags:
inkling
# Inkling
BF16 |
NVFP4 |
Playground |
Tinker Cookbook |
Acceptable Use
## 1. General Information
Inkling is a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages. The model is designed to be used by developers building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems, and is suitable for general-purpose conversational use, instruction-following, and other natural language and multimodal tasks. It is released with open weights to support research, fine-tuning and integration into third-party products by downstream developers.
**Languages:** English, with general multilingual capabilities across other languages.
## 2. Getting Started
Try Inkling on the Tinker Playground or access via API using the Tinker Cookbook.
Inkling supports local deployment using the following open-source libraries:
* SGLang (recipe, PR)
* vLLM (recipe, PR)
* TokenSpeed (recipe, PR)
* Unsloth (recipe, PR)
* Huggingface (recipe, PR)
...
Repository: localaiLicense: apache-2.0
qwythos-9b-v2
Empero AI
# Qwythos-9B-v2 — the new and improved Qwythos
The next iteration of Qwythos: **all the reasoning of Qwythos-9B, with the looping behavior fixed.** v2 keeps the deep chain-of-thought, the uncensored research posture, and the 1M-token context of its predecessor, and cleans up the rough edges that showed up in real use.
- 🔁 **Looping behavior eliminated** — repetition/degeneration under greedy or low-temperature decoding dropped from **6.7% → 0%**. You can serve it *without* leaning on `repetition_penalty` as a band-aid.
- 🧠 **Reasoning fully preserved** — MMLU, GSM8K, GPQA, ARC and HumanEval are all held at (or above) the v1 level. This is a *hygiene* upgrade, not a capability regression.
- 🧩 **MTP head restored** — the native multi-token-prediction module (dropped in the previous export) is back, so config and weights agree and speculative-decoding setups work.
- 🪪 **Cleaner identity** — the model no longer prefaces unrelated answers with its identity; it introduces itself only when you actually ask.
- 🔓 **Still intentionally uncensored** for research, cybersecurity, red-teaming, biology, chemistry, pharmacology and clinical work.
- 📜 **St
...
Repository: localaiLicense: apache-2.0
qwen3.6-27b-fable-fusion-711-uncensored-heretic-nm-dau-neo-max-mtp
Important: This is the first fine tune to exceed 700 "arc-c" (The OpenAI, Claude and Gemini "zone of intelligence") in both 8 bit and 4 bit. This repo contains both "regular" and "MTP" Neo MAX Imatrix quants.
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF
The strongest, smartest open source multi-stage model fine tune for consumer hardware ever and BUILT on consumer hardware via Unsloth.
The first model of this size/type to breach "700" ARC-C in both 8 bit and 4 bit; hench the "711" in the name.
This model (both 4 bit and 8 bit) exceeds the base Qwen 3.6 27B in 6 out of 7 benchmarks, and matches it on the 7th AND exceeds all 7 benchmarks for Qwen3.6-35B-A3B.
The 700 "intelligence club" is reserved for OpenAI, Claude and Gemini closed source models.
This is the one they fear.
This is a multi-stage fine tune, multi-fine tune, and multi-stage merge.
A Colab between myself (multiple fine tunes, including multi-stage), Nightmedia (merge/benching), TeichAI (Polaris Dataset), armand0e (Light fable 5 traces) and trohrbaugh (heretic'ing the model).
...
Repository: localaiLicense: apache-2.0
minicpm5-1b-claude-opus-fable5-v2-thinking
# MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
GGUF quantizations for local deployment: **MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-GGUF**
中文说明
**MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking** is a compact 1B **Thinking** language model built on openbmb/MiniCPM5-1B. Compared with V1, this V2 release is further fine-tuned on **Fable 5** data with a stronger focus on **tool calling / function calling**, while also improving **coding** and **instruction-following**. It keeps MiniCPM5's native Thinking chat template and XML tool-call format.
Previous version: **MiniCPM5-1B-Claude-Opus-Fable5-Thinking** (V1)
For llama.cpp / Ollama / LM Studio deployment, see the **GGUF repository**.
## Overview
## Capabilities
- **Tool calling (enhanced in V2)** — more reliable XML / function-calling style tool use on top of MiniCPM5's native format
- **Coding** — code generation, debugging, and software-engineering-style tasks
- **Instruction following** — more reliable adherence to user prompts and structured constraints
- **Thinking mode** — chain-of-thought reasoning via the MiniCPM5 chat template
- **Long context** — up to **128K tokens** (131,072 tokens per `config.json`)
...
Repository: localaiLicense: apache-2.0
hy3
中文 ｜ English
[](#license)
[](https://huggingface.co/tencent/Hy3)
[](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)
[](https://cnb.cool/ai-models/tencent/Hy3)
[](https://ai.gitcode.com/tencent_hunyuan/Hy3)
🖥️ Official Website |
💬 GitHub
## Table of Contents
- Model Introduction
- Stronger Agent Capabilities
- More Reliable Product Experiences
- Benchmark Appendix
- News
- Model Links
- Quickstart
- Deployment
- vLLM
- SGLang
- Finetuning
- RL Post-training
- Quantization
- License
- Contact Us
## Model Introduction
**Hy3** is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Following the Hy3 Preview launch in late April, we gathered feedback from 50+ products and scaled up post-training with higher quality data. Today, we introduce Hy3, which outperforms similar-size models and rivals flagship open-source models with 2-5x parameters. It also shows significant gains in utility across various products and productivity tasks.
## Stronger Agent Capabilities
...
Repository: localaiLicense: apache-2.0
bonsai-8b-1bit
Bonsai 8B (PrismML) is an end-to-end 1-bit language model built on the Qwen3-8B dense
architecture (GQA, SwiGLU, RoPE, RMSNorm, 36 layers, 65,536 context). Every weight is a
single sign bit (`-scale` / `+scale`) with one FP16 scale per group of 128 weights, for
an effective 1.125 bits/weight and a ~1.15 GB footprint (14.2x smaller than FP16) while
matching full-precision 8B instruct models at ~70.5 average across 6 benchmark
categories.
The Q1_0 quantization is only decodable by the PrismML llama.cpp fork, so this entry
runs on LocalAI's `bonsai` backend (that fork), not the stock `llama-cpp` backend.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-8b
Ternary Bonsai 8B (PrismML) is a 1.58-bit ternary language model on the Qwen3-8B dense
architecture. Each weight takes a value from {-1, 0, +1} with one shared FP16 scale per
group of 128 weights (GGUF Q2_0, ~2.18 GB deployed, 7.5x smaller than FP16). The extra
zero state recovers more of the full-precision model than the 1-bit build: it ranks 2nd
among compared 6-9B models at 75.5 average despite being ~1/8th their size.
Q2_0 is the recommended, ternary-lossless variant. The Q2_0 kernels are only in the
PrismML llama.cpp fork, so this runs on LocalAI's `bonsai` backend. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-8b-q2-g64
Ternary Bonsai 8B (PrismML), GGUF Q2_0 with group-64 packing (each FP16 scale shared
across 64 weights instead of 128). Slightly larger (~2.31 GB) but matches llama.cpp's
native 64-value Q2_0 block layout. Runs on LocalAI's `bonsai` backend. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-8b-pq2
Ternary Bonsai 8B (PrismML), GGUF PQ2_0 (packed Q2_0) ternary variant (~2.18 GB).
Same {-1, 0, +1} weight alphabet as Q2_0. Runs on LocalAI's `bonsai` backend.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
bonsai-27b-1bit
Bonsai 27B (PrismML) is a full 27B-class reasoning model in end-to-end 1-bit weights,
derived from the Qwen3.6-27B hybrid-attention backbone (~75% linear attention, 262K
context). At a true 1.125 bits/weight it deploys in ~3.9 GB (~14.2x smaller than FP16)
while retaining 89.5% of FP16 intelligence across 15 thinking-mode benchmarks (math
91.66, coding 81.88). Ships an optional 4-bit vision tower (mmproj) for image input,
included here.
The Q1_0_g128 weights and hybrid-attention kernels are only in the PrismML llama.cpp
fork, so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-27b
Ternary Bonsai 27B (PrismML) is the quality-oriented operating point of the Bonsai 27B
family: full 27B-class reasoning in ternary {-1, 0, +1} weights on the Qwen3.6-27B
hybrid-attention backbone (262K context). At a true 1.71 bits/weight it deploys in
~7.2 GB (GGUF Q2_0_g128) and retains 95% of FP16 intelligence (80.49 average across 15
thinking-mode benchmarks) - a higher score than a conventional IQ2_XXS build at less
than two-thirds its footprint. Ships an optional 4-bit vision tower (mmproj), included.
The Q2_0 weights and hybrid-attention kernels are only in the PrismML llama.cpp fork,
so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-27b-pq2
Ternary Bonsai 27B (PrismML), GGUF PQ2_0 (packed Q2_0) ternary variant (~7.17 GB) with
the 4-bit vision tower (mmproj) included. Runs on LocalAI's `bonsai` backend.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-27b-q2-g64
Ternary Bonsai 27B (PrismML), GGUF Q2_0 with group-64 packing (~7.59 GB), matching
llama.cpp's native 64-value Q2_0 block layout, with the 4-bit vision tower (mmproj)
included. Runs on LocalAI's `bonsai` backend. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
minicpm5-1b-claude-opus-fable5-thinking
# MiniCPM5-1B-Claude-Opus-Fable5-Thinking
GGUF quantizations for local deployment: **MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF**
中文说明
**MiniCPM5-1B-Claude-Opus-Fable5-Thinking** is a compact 1B **Thinking** language model built on openbmb/MiniCPM5-1B. It is further fine-tuned on **Fable 5** data to improve **coding** and **instruction-following** while keeping MiniCPM5's native Thinking chat template and tool-call format.
For llama.cpp / Ollama / LM Studio deployment, see the **GGUF repository**.
## Overview
## Capabilities
- **Coding** — code generation, debugging, and software-engineering-style tasks
- **Instruction following** — more reliable adherence to user prompts and structured constraints
- **Thinking mode** — chain-of-thought reasoning via the MiniCPM5 chat template
- **Tool calling** — inherits MiniCPM5's XML tool-call format
- **Long context** — up to **128K tokens** (131,072 tokens per `config.json`)
## Quick start
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking"
...
Repository: localaiLicense: apache-2.0
deepseek-v4-flash
# DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Technical Report👁️
## Introduction
We present a preview version of **DeepSeek-V4** series, including two strong Mixture-of-Experts (MoE) language models — **DeepSeek-V4-Pro** with 1.6T parameters (49B activated) and **DeepSeek-V4-Flash** with 284B parameters (13B activated) — both supporting a context length of **one million tokens**.
DeepSeek-V4 series incorporate several key upgrades in architecture and optimization:
1. **Hybrid Attention Architecture:** We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only **27% of single-token inference FLOPs** and **10% of KV cache** compared with DeepSeek-V3.2.
2. **Manifold-Constrained Hyper-Connections (mHC):** We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.
3. **Muon Optimizer:** We employ the Muon optimizer for faster convergence and greater training stability.
...
Repository: localaiLicense: mit
qwopus3.6-35b-a3b-coder-mtp
# 🌟 Qwopus3.6-35B-A3B-v1
## 💡 Base Model Overview
**Qwen3.6-35B-A3B** is an advanced hybrid sparse MoE (Mixture-of-Experts) model developed by Alibaba Cloud. It features 35B total parameters with only 3B active parameters per token, ensuring high inference efficiency. Architecturally, it combines Gated DeltaNet linear attention with standard gated attention layers, routing tokens across **256 experts**. It natively supports a massive **262k context window** and is specifically designed for high-performance agentic coding, deep reasoning, and multimodal tasks.
## 🚀 Model Refinement & Logic Tuning （Qwopus3.6-35B-A3B-v1）
🪐**Qwopus3.6-35B-A3B-v1** is a reasoning-enhanced MoE (Mixture of Experts) model fine-tuned on top of **Qwen3.6-35B-A3B**.
### 🛠 Training Strategy
The fine-tuning process for this model is structured into **three distinct stages of distributed SFT (Supervised Fine-Tuning)**, progressively scaling reasoning complexity and data diversity. This systematic approach ensures the model inherits the base MoE capabilities while sharpening its logic-handling depth.
...
Repository: localaiLicense: apache-2.0
ornith-1.0-9b-mtp
[](https://deep-reinforce.com/ornith.html)
# Ornith-1.0-9B
Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.
Highlights:
- **State-of-the-Art Coding Agents**: Available in 9B-Dense, 31B-Dense, 35B-MoE, and 397B-MoE (post-trained on top of Gemma 4 and Qwen 3.5), achieving state-of-the-art performance among open-source models of comparable size on coding benchmarks such as Terminal-Bench 2.1, SWE-Bench, NL2Repo and OpenClaw.
- **Self-Improving Training Framework**: Ornith-1.0 employs RL to learn to generate not only solution rollouts, but also the scallfold that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model discovers better search trajectories and generates higher-quality solutions.
- **Licence**: MIT licensed, globally accessible, and free from regional limitations.
## Ornith 1.0 9B
This model card documents **Ornith-1.0-9B**, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment.
### Benchmarks
Ornith-1.0-9B
Qwen3.5-9B
Qwen3.5-35B
Gemma4-12B
Gemma4-31B
Agentic Coding
...
Repository: localaiLicense: mit
qwen-agentworld-35b-a3b
# Qwen-AgentWorld-35B-A3B
📑 Technical Report |
📖 Blog |
🤗 Hugging Face |
🤖 ModelScope |
💻 GitHub |
🖥️ Demo
> [!Note]
> This repository contains the model weights and configuration files for **Qwen-AgentWorld-35B-A3B**, a native language world model trained for agentic environment simulation.
>
> These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, etc.
**Qwen-AgentWorld** is the first language world model to cover seven agent interaction domains within a single model. It simulates agentic environments via long chain-of-thought reasoning, predicting the next environment state given an agent's action and interaction history. Trained through a three-stage pipeline — CPT injects environment knowledge, SFT activates next-state-prediction reasoning, RL sharpens simulation fidelity — Qwen-AgentWorld is a **native world model**: environment modeling is the training objective from the CPT stage onward, not a post-hoc add-on.
## Highlights
...
Repository: localaiLicense: apache-2.0
ornith-1.0-9b
[](https://deep-reinforce.com/ornith.html)
# Ornith-1.0-9B-GGUF
Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.
Highlights:
- **State-of-the-Art Coding Agents**: Available in 9B-Dense, 31B-Dense, 35B-MoE, and 397B-MoE (post-trained on top of Gemma 4 and Qwen 3.5), achieving state-of-the-art performance among open-source models of comparable size on coding benchmarks such as Terminal-Bench 2.1, SWE-Bench, NL2Repo and OpenClaw.
- **Self-Improving Training Framework**: Ornith-1.0 employs RL to learn to generate not only solution rollouts, but also the scallfold that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model discovers better search trajectories and generates higher-quality solutions.
- **Licence**: MIT licensed, globally accessible, and free from regional limitations.
## Ornith 1.0 9B
This model card documents **Ornith-1.0-9B**, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment.
### Benchmarks
Ornith-1.0-9B
Qwen3.5-9B
Qwen3.5-35B
Gemma4-12B
Gemma4-31B
Agentic Coding
...
Repository: localaiLicense: mit
ornith-1.0-35b
[](https://deep-reinforce.com/ornith.html)
# Ornith-1.0-35B-GGUF
Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.
Highlights:
- **State-of-the-Art Coding Agents**: Available in 9B-Dense, 31B-Dense, 35B-MoE, and 397B-MoE (post-trained on top of Gemma 4 and Qwen 3.5), achieving state-of-the-art performance among open-source models of comparable size on coding benchmarks such as Terminal-Bench 2.1, SWE-Bench, NL2Repo and OpenClaw.
- **Self-Improving Training Framework**: Ornith-1.0 employs RL to learn to generate not only solution rollouts, but also the scallfold that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model discovers better search trajectories and generates higher-quality solutions.
- **Licence**: MIT licensed, globally accessible, and free from regional limitations.
## Ornith 1.0 35B
This model card documents **Ornith-1.0-35B**, the lightweight member of the Ornith family, designed for efficient single-GPU deployment.
### Benchmarks
Ornith-1.0-35B
Qwen3.5-35B
Qwen3.6-35B
Gemma4-31B
Qwen3.5-397B
Agentic Coding
...
Repository: localaiLicense: mit
gemmable-4-12b-mtp
## Gemmable 4 12B
Gemmable 4 12B is a GGUF export of Gemma 4 12B fine-tuned on Fable-5 style
reasoning and assistant traces.
## Highlights
- Base model: `google/gemma-4-12B`
- Format: GGUF
- Training style: Fable-5 style reasoning and assistant traces
- Distribution: fp16 GGUF plus matching assistant GGUFs for each quant
- Intended use: local inference, coding, reasoning, and assistant workflows
## How to use
### llama.cpp
Standard load:
```bash
llama-server -m "gemmable-4-12b-fp16.gguf"
```
Speculative / draft-MTP load:
```bash
llama-server -m "gemmable-4-12b-Q4_K_M.gguf" \
--spec-draft-model "gemmable-4-12b-Q4_K_M-mtp.gguf" \
--spec-type draft-mtp \
--spec-draft-n-max 4
```
Use the matching fp16 or quantized main file with its `-mtp` companion.
### LM Studio
1. Search this repo, download target + mtp file.
2. Load target.
3. Load settings → Speculative Decoding → select mtp file file.
(Requires a llama.cpp runtime with Gemma 4 MTP support from ggml-org/llama.cpp#23398. LocalAI's pinned llama.cpp backend already carries it, so this entry runs draft-mtp out of the box.)
## GGUF / local inference notes
...
Repository: localai
Page 1