JarvisBitz Tech
Insights

Guides from the people who build the systems.

Answer first, no hype, and updated as the field moves rather than left to rot with a publish date on it.

Latest
JarvisBitz
Retrieval
Living guide9 min

Is RAG still needed now that context windows are huge?

Yes, for most business systems. Long context solved the cases where all your data fits and you can afford to resend it every call. Retrieval still wins on cost, freshness, permissions, and being able to show where an answer came from.

Updated 8 September 2026Read
More guides
JarvisBitz
Agents
Living guide8 min

A2A explained: when AI agents need to talk to each other

Most businesses do not need it yet. A2A standardises how agents built by different teams or vendors delegate work to each other, which only starts to matter once you have more than one agent and the other one is not yours to change.

Updated 8 September 2026Read
JarvisBitz
Agents
Living guide7 min

Prompt injection: why filters fail and what actually works

Not with filters. A model cannot tell where an instruction came from, so any defence that tries to spot a malicious one will eventually be talked past. What holds is constraining what the agent is able to do after it reads untrusted content, which is an architecture decision rather than a prompt.

Updated 8 September 2026Read
JarvisBitz
Buying AI
Living guide7 min

AI vendor security questions: what a good answer sounds like

The standard questionnaire will not tell you what you need. SOC 2 covers how a vendor runs their own company, not what the agent they build for you is able to do when it is talked into the wrong action. The questions that separate a serious builder are about permissions, credentials, and what capability they gave up to get there.

Updated 8 September 2026Read
JarvisBitz
Agents
Living guide7 min

Where human approval actually belongs in an AI agent

In front of actions that cannot be undone, or that reach further than a blast radius you have deliberately chosen. Gating everything is close to gating nothing, because a control that fires constantly stops being read. The design work is deciding which actions qualify, then shrinking permissions so the list stays short.

Updated 8 September 2026Read
JarvisBitz
Buying AI
Living guide9 min

Build vs buy an AI agent: the arithmetic that decides it

You cannot answer it until you know how many billable actions one conversation takes, because every pricing model now meters actions rather than conversations. That makes your bill a consequence of how the agent is designed, and it means the same use case can land either side of the line depending on choices you have not made yet.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

LLM data residency: what to configure and what to verify

Residency is a configuration you set, not a property you inherit, and the defaults differ per vendor and change without notice. Establish four things in writing for each provider, then enforce the rest at your own gateway, because that is the only layer you control.

Updated 8 September 2026Read
JarvisBitz
Voice
Living guide8 min

Your voice agent is fast and still feels wrong. Here is why

Almost always turn-taking rather than raw speed. A fast agent that interrupts you mid sentence or leaves a beat of dead air feels worse than a slightly slower one that takes its turn correctly. Chasing the latency number first is how teams spend weeks and change nothing a caller notices.

Updated 8 September 2026Read
JarvisBitz
Retrieval
Living guide8 min

Why your RAG gives wrong answers that sound right

Because retrieval and faithfulness are different failure modes and most teams only measure the first. A better retriever can make wrong answers more convincing, not less, so faithfulness has to be scored separately and low scoring answers routed to abstention.

Updated 8 September 2026Read
JarvisBitz
Agents
Living guide8 min

How to test an AI agent before it touches production

Run it against real inputs with its writes going nowhere, compare what it proposed against what actually happened, then give it a small share of live traffic with spend caps and a fast stop. Stages without written promotion criteria are decoration.

Updated 8 September 2026Read
JarvisBitz
Retrieval
Living guide7 min

Should you fine-tune a model on your company documents?

Almost certainly not. Fine-tuning shapes behaviour and format, it does not reliably install facts. Most requests to train a model on internal documents are a retrieval or prompting problem wearing a more expensive hat.

Updated 8 September 2026Read
JarvisBitz
Vision
Living guide8 min

Document extraction accuracy, and the threshold that ships it

A single accuracy number for a document is close to meaningless, because accuracy varies enormously by field type. Headers are easy, line items and long alphanumeric fields are not, and the thing that makes extraction production safe is a per field confidence threshold routing the uncertain ones to a person.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

LLM spend: attribution before optimisation

You cannot cut what you cannot see. Most teams optimise models and prompts before they can attribute a single dollar to a team, feature or customer, which means they are guessing. Meter at the gateway first, then cache, route and cap in that order.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

What to log in an LLM system so you can debug it later

A 200 response tells you almost nothing about an AI system, because the failure you care about is a confident wrong answer that returned successfully. Log the inputs, the retrieved context, every tool call and the guardrail outcomes, or you will be debugging from screenshots.

Updated 8 September 2026Read
JarvisBitz
Vision
Living guide7 min

Vision API or a custom model? A decision rule that holds

Use a general vision API for semantic work at modest volume, and train a specialist when the task is geometric, high throughput, real time or on device. The split is not capability in the abstract, it is whether you are asking what is in the picture or exactly where and how much.

Updated 8 September 2026Read
JarvisBitz
Retrieval
Living guide8 min

RAG permissions: why your assistant leaks and how to stop it

Because a vector index has no idea who is asking, and the filter is the easy half. The hard half is keeping permissions synchronised with a source of truth that changes during the working day, which is where the leak window opens and why this never shows up in testing.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

Your model is being retired. How to move without regressions

The API call will port in an afternoon. What breaks is behaviour: the same prompts producing subtly different output, and you cannot tell without something to compare against. The migration is a measurement problem, and the teams that find it easy built the measurement before the email arrived.

Updated 8 September 2026Read
JarvisBitz
Buying AI
Living guide8 min

What you should own when the AI build ends

The code is the least valuable thing you funded. Name the prompts, the eval set, the embeddings and index configuration, the fine-tuned weights if any, and the infrastructure definition, in usable formats. The eval set matters most, because it is the artifact a successor cannot cheaply rebuild.

Updated 8 September 2026Read
JarvisBitz
Agents
Living guide8 min

Most agents should be workflows. Here is the test

If you can draw the steps in advance, write them as code and call the model inside them. Reach for an agent only when the next step genuinely depends on something you cannot know until runtime. Most systems described as agents fail that test, and they cost more and debug worse for it.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

Which LLM should we standardise on? Do not standardise

The question assumes standardising is the goal, and it is not. Benchmarks will not tell you which model is best at your task, models get retired on roughly annual cadence, and the cost of switching is what actually matters. Evaluate on your own data and keep at least two viable behind one gateway.

Updated 8 September 2026Read
JarvisBitz
Retrieval
Living guide8 min

Do you need a knowledge graph? Usually schema and filters

Usually not. Most problems that look like they need a graph are solved by a schema and metadata filters on the retrieval you already have. Graphs earn their cost on genuine multi-hop questions, where the answer requires connecting facts across documents rather than finding the right passage.

Updated 8 September 2026Read
JarvisBitz
Buying AI
Living guide8 min

Can an AI project be fixed price? Only after discovery

Not the build, and yes the discovery. Fixed price requires knowing what correct looks like before work starts, and on an AI project that knowledge is the output of the first phase rather than an input to it. Fix the price of discovery, and let it produce the evidence a build number can stand on.

Updated 8 September 2026Read
JarvisBitz
Buying AI
Living guide8 min

Why your AI pilot never reached production

Usually one of three things, none of which is model quality: nobody agreed a baseline so improvement could not be shown, nobody owned it after the demo, or it never had write access to the system it was supposed to change. All three are decided before a pilot starts.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

Should you strip personal data before it reaches the model?

Selectively, per field, not as a blanket policy. Redacting everything is easy to sell and breaks any task where the personal detail is the point. Decide by whether the model needs to understand the value or merely carry it, and measure both what leaks and what you destroyed.

Updated 8 September 2026Read
JarvisBitz
Delivery
Living guide8 min

What maintaining an AI system actually involves after launch

Mostly work that has no equivalent in ordinary software. The system can get worse without anything breaking, because the model underneath it changed, so maintenance here means continuously measuring behaviour rather than watching for errors.

Updated 8 September 2026Read
JarvisBitz
Agents
Living guide8 min

Stopping your assistant saying something you have to answer for

Not with a classifier bolted on the end. The controls that hold are structural: narrow what the assistant is allowed to discuss, require answers to be grounded in a source you control, and design refusal as a real behaviour rather than a failure state. A filter on the output is the last line, not the plan.

Updated 8 September 2026Read
JarvisBitz
Retrieval
Living guide8 min

Changing your embedding model is a migration, not a setting

You have to re-embed. A different model produces vectors in a different space, so old and new are not comparable and cannot share an index. It looks like a configuration change and behaves like a database migration, which is why it goes wrong so predictably.

Updated 8 September 2026Read
JarvisBitz
Agents
Living guide8 min

Should your AI agent remember users between sessions?

Usually not. Memory is sold as an upgrade and mostly buys you a privacy surface, a staleness problem, and a system that confidently repeats something a user corrected months ago. It is worth it when continuity is genuinely the product, and that is a smaller set of cases than the market implies.

Updated 8 September 2026Read
JarvisBitz
Vision
Living guide8 min

Should cameras run inference on the device or in the cloud?

Bandwidth and what happens when the link drops decide it, more often than model quality does. Work out what leaving the site costs before comparing accuracy, because at real frame rates the network is usually the binding constraint and it settles the argument.

Updated 8 September 2026Read
JarvisBitz
Voice
Living guide8 min

Can a voice agent handle Hinglish and regional accents?

Not as well as the demo suggests, and the gap is measurable. Recognition accuracy degrades substantially on code-switched speech compared to a single language, which matters because everything downstream reads the transcript. Scope it by measuring on your own calls before anyone promises a number.

Updated 8 September 2026Read
JarvisBitz
Buying AI
Living guide6 min

How much does custom AI development cost?

Nobody can quote you honestly without seeing your data. What they can do is tell you which five things move the number, and how to spend a little before you spend a lot.

Updated 7 September 2026Read
JarvisBitz
Agents
Living guide7 min

AI agent vs chatbot: which does your business actually need?

A chatbot answers. An agent acts. If resolving the request means changing something in one of your systems, you need an agent. If the answer already exists in a document, you do not, and buying one costs more and carries more risk.

Updated 7 September 2026Read
JarvisBitz
Delivery
Living guide7 min

MCP integration explained for business systems

MCP is a standard plug between AI systems and your tools, so you build a connector once instead of rebuilding it for every model and every assistant. It does not solve permissions, reliability, or the fact that a tool server is a privileged piece of infrastructure.

Updated 7 September 2026Read

How we write these

Guides are living pages: we keep them current and show when they last changed, because a page stamped with an old date is worth less than one that is still true. We do not publish numbers we cannot stand behind, and we say plainly when something depends on your situation rather than inventing a figure.

Want this applied to your business?

The free AI audit measures your live setup and shows where AI would actually pay off.