What Is an Infinite-Parameter LLM, and What Does It Change for Data Engineering?

What is an infinite-parameter LLM?

An infinite-parameter LLM is a language model that generates its own working weights at run time from whatever data you feed it, rather than reading that data through a prompt on every request. The definition that matters for data engineers comes from a September 2026 paper by researchers at Boltzbit Limited and the University of Cambridge (Jinli Hu, Ross M. Clarke, Yichuan Zhang, José Miguel Hernández-Lobato).¹² Their design uses a compact hypernetwork that reads live data and compresses it into a low-rank modulation of a shared base network, instead of storing that knowledge in a fixed parameter bank.¹² The model keeps a running belief about what it has learned and updates that belief as new data arrives mid-session, rather than freezing after one read of the context.¹² The weights on disk never grow. But the number of distinct weight configurations the model can compile from live input is effectively unbounded. That's the whole name, in one sentence.

Why prompts are a leaky bucket for live data

Right now, if you want a model to answer questions about your company's data, you have two blunt instruments: cram relevant text into the prompt (RAG), or bake it into the weights ahead of time (fine-tuning). RAG re-reads retrieved context on every request and throws it away the moment the response finishes. Nothing persists. The system re-fetches, re-ranks, and re-injects the same facts over and over, and every added document eats into the context window your team is already paying for. That's the same budget problem covered in our breakdown of agentic context management: the bill isn't really about the model, it's about what you keep shoving into its memory.

Fine-tuning solves the persistence problem but creates a freshness problem: once training ends, the model can't absorb anything that happens afterward without another costly training run.³ Building the RAG side isn't free either. IBM's own comparison of the two approaches notes that RAG systems require data engineers to build and maintain pipelines connecting the organization's data lakehouses to the LLM, plus the chunking, embedding, and vector infrastructure that goes with it.³ Infinite-parameter research is aimed squarely at this gap: keep the freshness of RAG, skip the repeated context-window tax, and skip the retraining cycle of fine-tuning.

From Mixture-of-Experts to weight-generating hypernetworks

The lineage matters here. This isn't a wild leap, it's an extension of ideas already shipping in production models.

• Mixture-of-Experts (MoE) replaces a single dense feed-forward layer with a bank of "expert" subnetworks plus a router that activates only a handful of them per token, letting total parameter count scale without scaling compute per token at the same rate.⁴⁵ Mixtral 8x7B is the textbook example: it holds 47 billion total parameters, but because only 2 of 8 experts fire per layer, roughly 12 billion parameters are actually doing work on any given token.⁴⁵
• LoRA (Low-Rank Adaptation) took a related idea to fine-tuning: freeze the pretrained weights and train small rank-decomposition matrices on top. Applied to GPT-3's 175 billion parameters, LoRA cuts trainable parameters by up to 10,000x and GPU memory needs by 3x versus full fine-tuning, while matching or beating full fine-tuning quality.⁶
• The infinite-parameter twist takes LoRA's low-rank adapters and stops treating them as something you train once, offline. Instead a hypernetwork generates the adapter on the fly from live data, and keeps regenerating it as a session evolves, using that online Bayesian belief update.¹²

Put plainly: MoE taught models to activate only the parameters they need. LoRA taught models to adapt cheaply with small weight patches. The Boltzbit/Cambridge paper teaches the model to write its own patches from whatever data shows up right now.

Why do two different papers share the name "infinite-parameter LLM"?

Here's where a data engineer researching this term gets tangled: "infinite-parameter LLM" names two unrelated research efforts.

The one this article focuses on, Hu, Clarke, Zhang, and Hernández-Lobato's 2026 paper, generates weights from live, run-time data.¹² The other is a 2024 paper by an independent author, Fei Ding, describing an architecture called IP-LLM: a 24.5-billion-parameter model split into 22 knowledge categories, where inference loads only a shared 7.2B base, a 0.75B router, and the 0.75B parameters for the relevant category, for 8.7 billion parameters total: a 65% cut in inference memory versus loading the full dense model.⁷ Ding's IP-LLM is about lifelong learning without catastrophic forgetting through selective loading of stored parameter groups, not about generating weights from data you feed it at query time. Same name, different problem, different mechanism. Search "infinite parameter LLM" and you'll likely land on both. Know which one you're reading.

How does an infinite-parameter LLM compare to fine-tuning, RAG, and long context windows?

Line up the four approaches a data engineer might actually reach for:

• Fine-tuning. Persists knowledge in weights permanently, but can't absorb anything post-training without a new run. Catastrophic forgetting gets worse, not better, as you scale models from 1B to 7B parameters in continual fine-tuning setups.⁸ Expensive to keep current.
• RAG. Stays fresh by design, but re-reads retrieved context on every request, burns context-window budget, and needs ongoing pipeline, chunking, and vector-database maintenance.³
• Long context windows. Sidesteps retrieval infrastructure by just stuffing more in, but cost and latency scale with every token you keep resident, and it still discards everything at session end.
• Live-weight generation (infinite-parameter LLM). Carries run-time knowledge in the weights rather than the prompt. The authors argue this is amortized in compute, frees the context window, persists across conversational turns, and can generalize better than in-context learning or retrieval. They propose an evaluation protocol built to test that claim head-to-head against RAG and in-context learning.¹²

The honest caveat: this is a proposed architecture and a proposed evaluation protocol, not a benchmarked system with public numbers against production RAG stacks.

The feature store parallel data engineers already know

If you've built a feature store, this should feel familiar even though the machinery is new. A feature store exists to compute and serve transformed data consistently across training and production, specifically to prevent training-serving skew: the same feature, computed the same way, everywhere it's used.⁹ Infinite-parameter architectures push that same discipline one level deeper. Instead of curating live signals into prompt-level context that a model reads and forgets, the hypernetwork turns those signals into weight-level features, computed fresh each session and carried forward as the model adapts. It's the feature-engineering instinct, applied to the weights instead of the input pipeline.

What this means for your data engineering roadmap

None of this is production-ready today. Treating it as a drop-in replacement for your RAG stack would be premature. But it changes what to watch for:

• Context-window pressure may become optional, not fixed. If live-weight generation matures, the endless arms race of bigger context windows and more aggressive chunking stops being the only lever you have for freshness.
• Session-scoped weight state is a new infrastructure question. If a model's effective weights change turn by turn, someone has to own versioning, auditing, and rollback of that state, a close cousin of the persistent-memory problem covered in our explainer on AI persistent memory.
• Evaluation gets harder before it gets easier. You'll need protocols that compare live-weight generation against in-context learning and retrieval directly, not benchmark leaderboards that were never built for this comparison, the same caution we've raised about choosing models by benchmark score alone.
• The infrastructure choice still comes back to ownership. Whether you're running RAG glue code today or evaluating live-weight approaches tomorrow, the underlying question for internal tools teams is the same one platforms like Remy are built around: do you own the pipeline that turns your company's data into model behavior, or are you renting someone else's black box.

Where this lands in the shadow AI conversation

This matters beyond research labs because employees are already building their own AI tooling on top of live company data, with or without IT's blessing. A Cybernews survey of more than 1,000 US employees found 59% use unapproved AI tools at work, a number that jumps to 93% among executives and senior managers.¹⁰ Three-quarters of those shadow AI users admit to feeding sensitive employee, customer, or internal document data into those unapproved tools.¹⁰ IBM-sponsored research found 80% of American office workers use AI in their roles, but only 22% rely exclusively on employer-provided tools. Breaches linked to shadow AI cost, on average, $670,000 more than breaches involving only sanctioned tools.¹¹¹⁰

Architectures that let models absorb live company data more durably, instead of discarding it after one prompt, raise the stakes on that governance gap. If a model can carry what it learned from a session forward, IT needs to know what data went in, who approved it, and how to audit or wipe it. A leaky, stateless prompt never forced anyone to answer those questions clearly.

The bottom line

An infinite-parameter LLM, in the sense that matters for data engineers, is a model that generates its own weights from live data at run time using a hypernetwork, instead of reading that data through a prompt and discarding it.¹² It builds on Mixture-of-Experts' sparse activation and LoRA's low-rank adapters, but generates the adapter on the fly instead of training it offline.⁴⁵⁶ It's early-stage research, not a shipped alternative to your RAG pipeline, and it shares its name with an unrelated 2024 architecture aimed at lifelong learning through selective parameter loading.⁷ Watch this space. Don't rip out your vector database yet.