The Architecture of Modular Intelligence: Project Granite Switch
Most teams still solve “add one AI capability” by retraining, fine-tuning, or forking the whole model. IBM Research has introduced a cleaner answer: treat adapters like software packages, not model surgery.
Project Granite Switch composes a Granite base model with specialized adapter functions into a single deployable checkpoint. A capability can then be activated when needed and replaced or upgraded independently without retraining the base model.
That is more than an inference optimization. It is a software architecture idea applied directly to model weights.
The core abstraction: adapter functions
An adapter function has a defined job and a known input/output contract. Instead of asking a general-purpose model to generate more open-ended text, an adapter can:
- score a document for relevance;
- rewrite a query;
- detect a hallucination;
- check whether an answer satisfies a requirement;
- make a safety or policy decision.
This is much closer to calling a function from a library than prompting an all-purpose model and hoping it follows the requested format.
IBM’s Granite Libraries currently group these functions into RAG, core, and guardian capabilities. Mellea provides the orchestration layer: it handles the activation tags and constrained output so applications can call the adapters through typed Python functions.
The technical trick: activated LoRA
Standard LoRA adapters create a practical problem in multi-step workflows. Switching from one adapter to another can invalidate the existing key-value cache, forcing the model to recompute the prompt context before continuing.
Activated LoRA, or aLoRA, changes when the adapter weights apply. A control token activates an adapter for the relevant token positions while the adapters share a normalized base KV cache. Earlier context can therefore be reused across steps instead of being processed again each time.
Granite Switch adds the switching mechanism, embeds compatible adapters with the base model, and prepares the control tokens and chat template required for inference. The resulting checkpoint can be served with vLLM or used with Hugging Face.
The same instinct, one layer up the stack
I encountered a smaller version of the same architectural problem while building HomeCortex , my self-hosted voice assistant for Home Assistant.
A voice assistant is not one model. It is a pipeline:
- Speech-to-text (STT) converts the request into text.
- The LLM interprets the request and produces a response or action.
- Text-to-speech (TTS) turns the response back into audio.
Hard-coding any one provider into that flow makes the entire assistant harder to evolve. HomeCortex therefore exposes each stage as an independent backend slot, selected through configuration:
- STT: Whisper MLX or faster-whisper;
- LLM: Ollama or llama.cpp;
- TTS: Piper for fully local, low-latency speech; XTTS v2 for higher local voice quality; or ElevenLabs for a cloud option.
The modularity exists at a different level, but the design instinct is the same. Granite Switch swaps specialized capabilities inside a model checkpoint. HomeCortex swaps specialized engines inside an application pipeline.
One pattern, two layers
| Project Granite Switch | HomeCortex | |
|---|---|---|
| Modularity layer | Model checkpoint | Application pipeline |
| Stable foundation | Granite 4.1 base model | Voice assistant orchestration |
| Swappable units | LoRA and aLoRA adapter functions | STT, LLM, and TTS backends |
| Selection mechanism | Control tokens and switching layer | Configuration |
| Operational benefit | Reuse one deployment for multiple targeted capabilities | Change one engine without rewriting the pipeline |
This separation creates three useful properties for platform teams:
- A smaller operational footprint. One checkpoint or one pipeline an expose several capabilities without running a separate end-to-end system for each one.
- Independent evolution. A component can be tested, replaced, or upgraded like a dependency version.
- Pay only when invoked. When a capability is not selected, the rest of the system continues without carrying its full execution cost.
There are limits to the analogy. An adapter modifies the behavior of a shared neural model, while a backend module is a conventional service boundary with its own runtime and latency profile. Their implementation and failure modes are different.
But the architectural lesson survives that distinction:
Composition beats monoliths at every layer of the stack.