Inferbench

@JoniMartin27

About Inferbench

InferBench's MCP server lets coding agents run, serve and benchmark local LLMs (text + image, llama.cpp + Stable Diffusion) on your own hardware on demand. Measures real tokens/sec, picks the optimal quant for your GPU, and exposes a 124-model catalog. Local-first, no cloud requi

#llm #local-llm #benchmark #llama-cpp #inference #image-generation

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "inferbench": {
      "command": "C:\\Users\\<user>\\AppData\\Local\\Programs\\InferBench\\resources\\sidecar\\inferbench-backend.exe",
      "args": [
        "--mcp"
      ]
    }
  }
}

Tools

No tools detected

We auto-extract tools from the README. The maintainer can list them under a ## Tools heading to populate this section.

Overview

What is Inferbench?

Inferbench is an MCP server that lets coding agents run, serve, and benchmark local LLMs (text + image, via llama.cpp and Stable Diffusion) on your own hardware. It measures real tokens/second, picks the optimal quantization for your GPU, and exposes a 124-model catalog.

How to use Inferbench?

—

Key features of Inferbench

Run, serve, and benchmark local LLMs on demand
Measures real tokens per second throughput
Selects optimal quantization for your GPU automatically
Supports text and image models (llama.cpp + Stable Diffusion)
Exposes a catalog of 124 models
Operates entirely local-first, no cloud required

Use cases of Inferbench

Benchmarking local LLM performance across different quantizations and hardware
Running coding agent tasks against locally served text and image models
Identifying the best-performing model quantization for a given GPU

FAQ from Inferbench

What models does Inferbench support?

Inferbench supports text models via llama.cpp and image models via Stable Diffusion, with a catalog of 124 models.

Where does data stay and is the cloud required?

Inferbench is local-first and requires no cloud.

What performance metrics does Inferbench measure?

It measures real tokens per second throughput.

Frequently asked questions

What models does Inferbench support?

Inferbench supports text models via llama.cpp and image models via Stable Diffusion, with a catalog of 124 models.

Where does data stay and is the cloud required?

Inferbench is local-first and requires no cloud.

What performance metrics does Inferbench measure?

It measures real tokens per second throughput.

Comments

Basic information

More Other MCP servers

Unity MCP ✨

justinpbarnett

Unity MCP acts as a bridge between AI assistants and your Unity Editor. Give your LLM tools to manage assets, control scenes, edit scripts, and automate tasks within Unity.

12.2K