Usar LetronnaLocal models
Choosing a model
Which local model fits your memory, why it must call tools, and the context window.
Esta página aún no está traducida, así que se muestra en inglés. Puedes traducirla en GitHub. GitHub ↗
What fits
A model has to sit whole in memory to run at a useful speed: the GPU's memory on a PC with a graphics card, or the shared memory on an Apple silicon Mac. The download size is a good guide to what it takes, plus some room for the conversation.
| Memory free for the model | Size to look for |
|---|---|
| 8 GB | about 4 billion parameters (4B) |
| 16 GB | 7B to 9B |
| 32 GB | up to about 30B |
| 64 GB or more | 70B and the larger mixture-of-experts models |
These assume the usual 4-bit download. A model that does not fit either fails to load or spills onto the processor and slows to a crawl.
Tools
The agent works through tool calls, so pick a model that was trained for them; its page on the server's library usually says "tools". Qwen and Llama models with tool support do well. Smaller models lose track over long tasks sooner: give them one job at a time, and switch to a hosted model for the hard parts.
The context window
The context window is how much of the conversation the model reads at once. Local servers often serve a far smaller window than the model was trained for, and may cut a long prompt short without saying so. So unless a model's window is known, Letronna keeps a server with no key to a small budget and clears old tool results early.
If you have raised the window on the server, tell Letronna how large it is: see A model on your own server answers badly.
/compact summarises earlier turns when a conversation still grows too long.