The video could not be loaded. Press Play to try again, or read the transcript below.
- With the player focused:
- Space play or pause
- C captions
- JL back or forward 10 seconds
- F full screen
RAG in ServiceNow
AI Search embeds retrieval, ranking and semantic indexing directly into platform records.
Transcript
When it comes to AI, retrieval-augmented generation, or RAG, is one of the most fundamental building blocks of any intelligent application.
Most people think of RAG as a bolt-on: something that fetches data from a document store, shoves it into a large language model, and hopes for the best. But inside ServiceNow, RAG is not an afterthought. It’s embedded right into the fabric of the platform, through AI Search.
The words around it can be daunting: indexing, retrieval, chunking, scoring, re-ranking and embeddings. But once you understand how ServiceNow structures them, you can start to tune RAG for your own use cases.
At the core of ServiceNow’s RAG is semantic search, built on a set of linked configuration tables.
At the top is the search application: the experience itself, such as Now Assist, Virtual Agent or the portal. The search profile decides which data sources it uses, and how it ranks and filters the results. A search source is a single table, such as knowledge articles or incidents. An indexed source marks it as enabled for semantic search. The semantic index config controls the vector embeddings, the heart of semantic search. And semantic fields are the fields whose text is embedded into vectors.
Together, they convert ordinary records into vectorised meaning. Indexing runs from the bottom up, from each semantic field to the search application.
When a user or an AI agent asks a question, like “How do I reset my VPN password?”, ServiceNow turns the query into an embedding, and compares it with the stored vectors.
The RAG Search API handles this behind the scenes. It goes through the search profile to the semantic index, and returns the most relevant results, based on similarity scores and metadata filters. Those results go to the LLM, which writes the response.
The result is not just a keyword match. It’s an answer grounded in your own platform data, which is the essence of RAG.
Where things get really interesting is when you start customising this.
RAG in ServiceNow works as a layered pipeline, and each layer is a place to add intelligence or control. The agent orchestration layer decides when to call RAG, which profiles to use, and how to combine the sources. The retrieval layer handles chunking, embedding and scoring, so you can change chunk sizes or embedding models for different content. The ranking layer refines the results, for example boosting recent content, or weighting technical documentation. The prompt composition layer builds the message to the LLM. And the post-processing layer verifies, formats and cites the answer, before the user sees it.
This turns RAG into a programmable system, not a fixed capability.
For example, a scripted REST API could query specific search profiles through the RAG Search API, re-rank the results with your own logic, and inject them into a Now LLM prompt.
That’s where ServiceNow’s native RAG becomes part of a broader agentic workflow, where agents reason, retrieve and act, all within the boundaries of the same platform. You move from asking questions to operating workflows.
RAG in ServiceNow isn’t a mystery box any more. It’s a structured system that you can tune, extend and experiment with.
The systems that succeed won’t be the ones with the flashiest models. They’ll be the ones that retrieve the right context, at the right time, for the right task. Because in the end, the intelligence of an agent depends on what it knows, and that depends on what it can find.
