Nvidia researchers have demonstrated a way to transfer memory between large language models mid-session without restarting the conversation, cutting a major bottleneck in enterprise AI workflows. The technique, called cross-model KV cache transfer, maps the prefilled Key-Value cache from a source model directly into a target model using simple linear math, avoiding the costly prefill phase that normally forces a full recomputation when switching models.
Experiments across six model families showed the approach runs 2.7 to 25 times faster than recomputing conversations while retaining up to 98% of the target model’s standalone accuracy. For example, transferring a 32,768-token cache from a 14-billion parameter Qwen3 model to a 32-billion parameter version took just 278 milliseconds, compared to nearly 7 seconds for a standard re-prefill. The method works best within model families like Qwen3 or Llama 3.1, where shared architectures allow straightforward linear mapping.
The team also found that linear regression alone struggles with some model pairs, requiring a lightweight neural mapper to recover accuracy. Still, the technique offers a practical path to reduce compute costs in long-running agentic systems where models are frequently swapped to balance performance and efficiency.
