Select the knowledge
Choose content, add supplementary information and prepare the document index.

SDX · RAG AI assistant
A chat widget integrated into the CMS: simple exchanges handled locally, document retrieval when needed and answers generated on the visitor’s GPU.
All case studies
Enlarge screenshot01 / RAG
Website pages explain an offer, pricing and how a business works. Visitors do not always know where to look. The goal was to let them ask a question and find an answer connected to the information already published.
I built this application on SDX Development’s own website. This internal project is live in production and brings together document retrieval, local models, knowledge administration and a conversation interface.
UPDATE · 3 OCTOBER 2026
Greetings, thanks and exchanges about the visitor’s name now follow a local path without loading a model or retrieving documents. An introduction can supply a name; asking for it again recalls that declaration rather than treating the question as a new name.
RAG runs when knowledge is useful. Follow-up questions about a price or delivery time retain the project explicitly described by the visitor. Offer sections containing the requested amount or duration now rank more prominently, reducing confusion with a different service.
A request such as “summarise this page” uses the current public page, its title and its address. Reading goes through the server and its URL protections; an explicit visitor link takes priority. Page context is treated as content, not as instructions for the model.
A name can be kept in an optional local visitor profile, separate from conversation history. After navigation or two minutes of inactivity while the chat is closed, an invitation appears beside the widget and can use that name. It does not automatically open the conversation window.
The initial 3 October tests checked introductions, name recall, page summaries, pricing and timelines in Windows Chrome. A subsequent test used the actually connected Galaxy S25 Ultra and desktop: Measurements provides total timings, tokens/s and observed defects with the saved models.
The small mobile model remains a project limitation. It can still refuse a price answer despite a correct source. Additional checks block some price omissions, amounts belonging to another offer and internal comments, but the semantic reviewer can still accept errors. Qwen3-0.6B was compared without replacing the production model: with the tested prompts, its reviewer accepted four incorrect drafts out of six.
02 / RAG
Simple exchanges start without retrieval. When a question needs website or service information, RAG connects the answer to sources, with the browser and website server sharing the work.
The CMS prepares an index from selected content and information added by the administrator.
The browser computes a representation of the question. The server searches the index for relevant passages and returns their sources.
A model running in the browser writes an answer using the retrieved passages. Additional retrieval can enrich the context.
A local check reviews the draft before display. If the bounded attempts produce no acceptable answer, the assistant explains the limit and may suggest contact.
Generation runs on the visitor’s device with GPU access. The service is not fully offline: it downloads models and queries the server for sources. Unanswered questions may be retained to improve the knowledge base.
The server stores the knowledge and retrieves passages. The browser runs query rewriting, embedding, writing and review models. Optional local conversation memory is a third resource: it helps follow the request without replacing documentary sources.
The current request, the explicitly described project, recent exchanges and any public reference. Simple name exchanges and greetings can be handled locally before this RAG path.
The model turns the request into a self-contained documentary query. The original question is retained alongside its rewrite.
Queries become numerical vectors calculated with WebGPU. These vectors are sent to the site API.
The index selects passages in the right language, applies the similarity threshold and combines rankings. It returns texts, titles and source links.
If allowed by the visitor, a related earlier question is retrieved from IndexedDB. It supplements recent exchanges without treating its old answer as evidence.
The response model uses CMS instructions, the question, conversation context and retrieved passages. The draft is not yet displayed.
The same model checks the draft against the sources. If rejected: at most one additional search and a second writing attempt.
An accepted draft is displayed with sources. When evidence is insufficient, a human handoff is offered. An eligible exchange may be saved locally.
Before questions begin, the administrator prepares content and computes its embeddings in the browser. Document vectors are stored on the server; visitors compute their question vectors without reindexing website pages.
EXECUTION
WebGPU gives browsers access to GPU computation. Here it handles inference: applying model weights to produce an embedding or generate text. Local execution avoids calling a hosted generation API for each answer. It still requires compatible browser support, drivers and GPU resources, on an HTTPS page.
The presence of navigator.gpu is insufficient. The application checks the hardware adapter and rejects a fallback software adapter. Physical tests included the Samsung Galaxy S25 Ultra’s Adreno 830 in Samsung Internet and an NVIDIA Ampere GPU on a computer. Features and buffer limits vary: browser branding and total phone RAM do not prove that an export will run.
Gemma 4 E2B uses LiteRT-LM, with @litert-lm/core pinned to 0.17.1, the GPU_ARTISAN backend explicitly requested, a 4,096-token context window and thinking disabled. The LiteRT-LM Web API remains an early preview, qualified here on the tested devices. E5 embeddings use a separate path: ONNX Runtime Web with WebGPU. Several ONNX exports, WebLLM/MLC and compact runtimes were evaluated before this choice.
Experimental compact runtimes also required work on WGSL shaders, quantized weights and cached model states. The Nanbeige 4.2 path must respect its architecture’s two passes: dropping a pass to save time would not be an equivalent optimization. A lightweight format or smaller model is insufficient; its operations must run correctly on that GPU and the RAG pipeline must remain reliable.
Integration required two separate Web Workers. The LiteRT SDK uses importScripts and starts in a classic worker; the embedding worker is a JavaScript module. This keeps the response model loaded during E5 retrieval and the interface responsive. Cancellation covers both workers; GPU loss releases the runtimes and reports an actionable error.
KNOWLEDGE
The CMS selects published pages, approved knowledge entries and eligible form information. Prepared text is split using the actual embedding tokenizer: 400-token passages with a 40-token overlap, a bounded title and a check of the 512-token limit after adding the prefix. Oversized passages are rejected instead of silently truncating information.
The selected Multilingual E5 Base profile produces 768-dimensional vectors. Inputs distinguish query: for questions and passage: for documents. Outputs use masked mean pooling and L2 normalization. Indexing and retrieval must use identical operations: a correctly sized vector can still be numerically wrong.
On mobile, the MatMulInteger path in some quantized exports was incompatible with the runtime used. A reproducible preparation tool retains INT8 weights while adapting matrix operations to a portable FP32 path. E5 artefacts undergo SHA-256 verification and comparison with four numerical reference fixtures, requiring a cosine of at least 0.999. References are calculated outside the browser; visitor inference remains on WebGPU.
MariaDB stores vectors in a VECTOR(1024) column. Remaining dimensions are padded with zeroes, preserving cosine calculations. Each index is linked to its embedding profile and each passage to its content hash. Outdated batches and unnormalized vectors are rejected.
For each question, the server ranks passages by cosine distance, filters language and still-published sources, applies the configured threshold and limits passages from any one document. The illustrated configuration keeps four passages with a 0.75 threshold. E5 Small, 384 dimensions and about 118 MB versus Base’s 279 MB, is available for further testing; it needs its own index and cannot be derived by shortening Base vectors.
Since 3 October, price and delivery questions favour published offer sections matching the declared project and containing the actual amount or duration in the context sent to the model. Site, language and publication filters still apply. Ranking creates no prices and does not remove the need to cite sources.
CONTEXT
A follow-up such as “what about maintenance?” is hard to search without its earlier topic. A first model pass creates a self-contained query from the request, recent exchanges and any reference. Output cleaning prevents model commentary from becoming a search query. If rewriting fails without a fatal runtime error, search can fall back to the original question.
The search retains the question and its rewrite when they provide distinct queries. The server combines their rankings through reciprocal rank fusion: passage ranks are combined rather than letting a rewrite entirely replace the visitor’s request. This reduces the risk of losing an important detail during rewriting.
Persistent memory is optional and stored in the visitor’s browser through IndexedDB: up to 60 exchanges for 7 days. Entries contain a question, answer, vector, language, embedding profile and timestamp. Expired entries are removed when accessed; users can disable or clear memory. If storage is unavailable, discussion continues without persistence.
Memory includes local semantic retrieval. The current question vector is compared with earlier questions in the same language and embedding profile; the best result above the profile threshold can enrich context. Recent exchanges take priority. An old generated answer is not reintroduced as a factual source: only the relevant earlier question supplies distant background context.
The visitor profile is separate from this memory: it holds only an explicitly declared name, including given and family names when available. Persistence is optional, stays in the browser and expires within 90 days without renewal. A forget request deletes the name. Name recall across language changes and navigation does not depend on an old saved conversation turn.
QUALITY
The writing prompt combines the CMS role and instructions, latest request, useful conversation history and retrieved passages. Sources are presented as data separate from system instructions. Drafts remain private: the interface displays progress, followed by an accepted answer. Waiting therefore includes multiple inference passes, not just visible writing.
A second pass of the same model assesses whether the draft addresses the question and whether essential facts are supported. LiteRT structured exchanges use a local emit_result tool with no external action. The application validates JSON objects and required field types. Valid JSON alone does not establish that the assessment is correct.
One real bug involved unequal source context: the writer saw up to 1,250 characters per passage, while the reviewer saw only 650. A price range farther down could be missing from review. Both passes now receive the same sources with a 1,250-character limit, avoiding a request to verify evidence withheld from the reviewer.
A rejected draft can trigger corrective retrieval and a second version. The process is bounded: at most two drafts and one additional search. Repeated old answers and internal commentary are filtered too. Without an acceptable answer, the assistant states the limitation and offers contact.
Native generation counters are retained; empty outputs and outputs reaching their token cap are rejected. No completion signal absent from the SDK is fabricated. Review reduces some defects but uses the same model as writing and can be wrong itself. False drafts and missing-information cases are essential tests; visitors can still inspect sources.
Additional local checks reject a price without an amount found in the requested offer, an amount taken from another offer, and certain internal-comment markers. These checks cover specific cases; they do not prove every commercial condition or make the model’s judgement infallible.
03 / CMS
The CMS brings together the settings that keep the assistant useful: knowledge, behaviour, models and appearance.
Choose content, add supplementary information and prepare the document index.
Separate mobile and desktop response models, selected on the client. The embedding model is configurable.
Answer tests and validation in the administration interface before making the assistant available.
Name, greeting, instructions and available features, with a route towards contacting the business.
Colours, avatar, position and preview. Adjustable aura: motion, direction, speed and maximum expansion.
Uncovered questions help identify information to add before testing the workflow again.
Production CMS screenshots in Chrome on 3 October 2026, with no settings changed. The capture excludes the account bar. The knowledge form is empty; the appearance preview shows a demonstration conversation.
Enlarge screenshot
Enlarge screenshot
Enlarge screenshot04 / MOBILE
Shadow DOM isolates the widget from site styles. On Android, readability also requires tracking the keyboard and the actual visible viewport.
The panel uses the available height and hides launcher-shell during discussion. Secondary controls are collapsed to leave room for messages.
VisualViewport supplies visible height and offset when the Android keyboard opens. The panel follows them; input grows from 44 to at most 96 px and send remains accessible.
Screen Wake Lock is requested during discussion, reacquired when returning to the foreground and released on closing. The browser or system can refuse it; no permanent phone setting is changed.
Enlarge screenshoton desktop, for the two price requests after the first computation
05 / Measure on mobile and desktop
On 3 October 2026, the same price question was asked three times in one French conversation, using the saved models and already cached files. On the desktop with an RTX 3070 Ti, Gemma answered in 11 seconds, then 7 seconds. On the connected Galaxy S25 Ultra, LFM displayed an answer in 22 seconds, then 14–17 seconds.
Timing runs from submission until the interface is ready again: live retrieval, writing, checking and display. The first computation includes runtime preparation, with no initial weight download. Three consecutive requests do not establish a guaranteed delay for other questions, devices or browsers.
| Device and model | First price request | Next two requests | Generation including context processing | Observed quality |
|---|---|---|---|---|
| Desktop · RTX 3070 Ti · Chrome 154 · Gemma 4 E2B | 11.0 s | 7.0 s / 6.9 s | 28.2–30.0 tokens/s | €1,500 excluding VAT with citation; correctly retained the 2–3 week brochure website timeline. |
| Galaxy S25 Ultra · Android 16 · Samsung Internet 30 · LFM2.5-350M | 22.2 s | 14.2 s / 17.3 s | 3.6–4.4 tokens/s | Found the €1,500 amount, without VAT qualification or citation; wrong brochure website timeline. |
Shared question, asked in French: “Quel est le prix de départ d’un site vitrine ? Réponds en une phrase.” The next two requests stay in the same conversation without reloading the page. History grows; models and browsers differ. These tests use the public site configuration and live retrieval, with no simulated excerpts or answers.
The table reports the two subsequent writing passes: native output token count divided by generation duration, including prompt processing. Native counts include technical output tokens; retrieval, model loading, checking and animation are outside this rate. Gemma produces 58 tokens from 1,946–2,010 input tokens; LFM produces 19 tokens from 1,809–1,852 input tokens.
The LiteRT SDK also provides a decoding-only rate for Gemma: 41–44 tokens/s on these two answers. The ONNX runtime used here supplies no separate counter for this phase, so its published rate includes prompt processing. No displayed-text token rate is estimated from word count.
The delivery follow-up takes 9.0 seconds on desktop and 19.2 seconds on mobile. Desktop gives the brochure offer’s 2–3 weeks; LFM wrongly gives 8–14 weeks, the e-commerce timeline. E-commerce pricing appears in 6.4 seconds on desktop and 16.8 seconds on mobile: €12,000 excluding VAT with citation on desktop, without VAT qualification or citation on mobile. Introducing and recalling a name takes around 0.7 seconds on each device, with no AI engine call.
Production configuration on 3 October 2026: LFM2.5-350M for mobile and tablet, Gemma 4 E2B for desktop, E5 Base for retrieval. Current measurements appear above. The September table below keeps the older Gemma S25 tests as historical results.
Measurements from 27 September 2026. Timings cover a RAG request through its reviewed answer, with weights already downloaded. They include retrieval and multiple model passes, rather than decoding alone.
| Device / candidate | Observed duration | Pipeline result |
|---|---|---|
| S25 Ultra · Samsung Internet · Gemma 4 E2B | 36.11 s | Price question: complete, reviewed answer, €1,500–€5,000 excluding VAT. |
| Desktop · Chrome / NVIDIA Ampere · Gemma 4 E2B | 12.25 s | Same question: 114 tokens; review also rejects a false-price draft. |
| Same desktop · Nanbeige 4.2 Compact | 120.73 s | Correct nominal answer, 238 tokens; false draft incorrectly accepted in a counter-test. |
| Same desktop · Gemma E2B · final validation | 13.06 s | Real test recorded in the CMS, followed by activation of the model shared by both profiles. |
Answers differ in length, and rewriting can change retrieved passages: these timings are not a universal model ranking. The counter-test supplied a €15,000–€100,000 draft against €1,500–€5,000 sources. A correct simple answer from Nanbeige was insufficient to qualify its reviewer.
On this desktop, Gemma E2B handled a price objection in 15.02 s and maintenance in 12.99 s. A false “24 hours for €100” promise and an unpublished exact turnover led to human handoff in 25.91 s and 23.59 s. Refusing to invent information was part of the test.
Mobile trials showed that smaller downloads do not guarantee a reliable pipeline: candidates lost the GPU, repeated text or failed review. The tested Qwen 3.5 4B and Gemma 3 4B ONNX exports produced no final desktop answer on this path. This describes those exports and runtimes, not every use of the models.
Embedding was measured separately on the S25 with cached weights: Base took 4.275 s for loading, checks and the first question, then 90–103 ms; Small took 3.487 s, then 24–27 ms. The small ranking test used five passages, not the site’s 855 fragments. Most of the long wait came from response-model passes: replacing embeddings alone would not fix it.
DATA
Response inference and persistent memory run on the device. The service remains connected: it downloads models and libraries, sends search vectors to the server and receives source texts. An embedding is not an anonymity guarantee. Uncovered questions can be recorded in the CMS to improve knowledge, for at most 90 days without a new request and up to 5,000 questions per site.
APIs enforce site scope, administrator sessions and operation-appropriate CSRF protections. Incoming vectors are checked for profile, dimensions, finite values and normalization. Search excludes withdrawn pages and unauthorized forms; draft knowledge entries are not exposed in answers.
When web references are enabled, the server can read a public URL supplied by the visitor. Protocols, network addresses and redirects are checked to block private-network access. Downloads have size and time limits. Useful text is extracted without running page scripts and remains external reference data, not a new system instruction.
The reference can also be the public page currently open: the client supplies its title and URL when a question refers to it. An explicit visitor URL takes priority. Reading uses the same server protections as other web references; administration addresses are never used as pages to summarise.
The assistant can also help prefill an authorized form from information explicitly supplied by the visitor. Field schemas, language, target form and validity duration are checked. Existing values are preserved, consent is never automatically checked, and no form is submitted without user action.
RELEASE
The CMS does more than select a model name. It prepares sources, resumes indexing, tracks pending passages and offers real tests. Mobile/tablet and desktop response profiles are independent. Client detection uses browser and device information; narrowing a desktop window does not switch it to a mobile model.
A model, instruction or index change invalidates the relevant publication approval. Changed sources must be prepared and their new passages indexed before the assistant can be validated and reactivated. A real test is recommended but remains optional in the current CMS. Configuration signatures prevent an old approval from surviving a change; purely visual settings retain it.
Verification covered real GPU generation, numerical embedding consistency, price and maintenance questions, false promises and unpublished figures. Mobile fixes were also checked with the keyboard open, panel opening and closing, an accessible send button and no overflow. Releases back up the affected scope and verify public pages.
Remaining work has specific targets: broaden qualification questions, measure cold downloads and quotas, check more browser/GPU pairs and compare new Adreno-compatible exports. A native app could access other accelerators; hosted inference could reduce phone workload. These alternatives would change the architecture and are not presented as already delivered.
Project tests support the choices and timings in this case study. These documents explain the technologies; they do not guarantee performance on a particular device.
Visitors can ask about the content and check its sources. The administrator can enrich the knowledge, configure the assistant and test its answers. The project covers the complete workflow, from editing content to using it in a conversation.
Explore AI & RAG servicesWhat are you working on?
Documentation, catalogues, procedures or services: let’s define sources, users and expected answers to build an assistant around your business.
Enter at least 2 characters to start searching.