Mesh LLM

Mesh LLM pools GPUs and memory across multiple machines—homelabs, businesses, or the internet—and exposes them through a single OpenAI-compatible endpoint. It automatically splits large models into smaller pieces, routing layers to appropriately-sized nodes so memory-constrained GPUs can run smaller slices. Users can run many models with routing by model selection, or split one large model across nearby GPUs.

The system works as a drop-in replacement for OpenAI clients, supporting tools like VS Code and any application using the OpenAI API format. It offers both router mode (selecting models by name) and distributed inference mode (splitting models across nodes). Mesh includes a pluggable architecture for agents, web console, and support for multiple platforms including Rust, Node.js, Swift, and Kotlin.