How fast do website AI agents answer? We timed Qualified, Intercom Fin and Karumi
Short answer
Karumi's website AI agent started answering in 2.5 seconds on median in our November 2026 benchmark, 3.6 times faster than Qualified's AI agents (8.9 s) and 2.5 times faster than Intercom Fin (6.3 s). We asked the same eight buyer questions to the AI chats of 12 companies and timed, in desktop Chrome, how long each chat took to put its first words on screen.
Karumi's website AI agent starts answering in 2.5 seconds on median, 3.6 times faster than Qualified's AI agents (8.9 s) and 2.5 times faster than Intercom Fin (6.3 s). We measured it on November 19, 2026: we asked the same eight buyer questions to the AI chats of 12 companies and timed, in a desktop browser, how long each one took to put its first words on screen.
A website AI agent is the chat that answers buyers on your site: what the product does, what it costs, whether it fits. Speed is the first thing a visitor feels. Below 1 second a reply feels instant; past 10 seconds, people stop paying attention, according to Nielsen Norman Group's response-time limits.
We build Karumi, so we tested in the open. Every company, every question and every step of the method is below, and anyone can repeat the test in an afternoon.
In this article:
- What we found, in one table
- How fast each company's AI agent answered
- Why response time matters for a website AI agent
- How we measured it, step by step
- What the numbers don't tell you
- What else each AI agent can do, and what only Karumi does
- Which AI agent fits which team
What did we find?
- Karumi started answering in 2.5 s on median across 32 answers at 4 companies: Agorapulse, Brevo, Karumi, Workday. 88% of its answers started within 3 seconds.
- Qualified's AI agents took 8.9 s on median across 55 answers at 7 companies, from 5.7 s at AdRoll to 12.3 s at UKG. 42% of Qualified's answers took more than 10 seconds to start.
- Intercom Fin took 6.3 s on median across 8 answers at Intercom itself. Intercom's own research reported a median of 7 seconds to first token in March 2025 (Intercom).
- The slow answers are where the gap shows. One in ten Qualified answers took longer than 13.7 s to start, and one in ten Intercom answers longer than 8.3 s. For Karumi, the same figure was 3.0 s.
- The difference is not noise. Karumi was faster than Qualified with p < 0.001 and faster than Intercom Fin with p < 0.001 (one-sided Mann–Whitney U tests). The 95% confidence interval for the speed-up is 3.0–4.4× over Qualified and 2.1–3.3× over Intercom Fin.
| Platform | Companies | Answers | Median to first words | 95% CI of the median | 90th percentile | Started within 3 s | Took over 10 s | Median to full reply |
|---|---|---|---|---|---|---|---|---|
| Karumi | 4 | 32 | 2.5 s | 2.3–2.7 s | 3.0 s | 88% | 0% | 6.7 s |
| Qualified | 7 | 55 | 8.9 s | 7.7–10.9 s | 13.7 s | 4% | 42% | 10.5 s |
| Intercom Fin | 1 | 8 | 6.3 s | 5.3–8.1 s | 8.3 s | 0% | 0% | 8.3 s |
Measured November 19, 2026, in Google Chrome on desktop. "First words" is the moment the answer's first characters appear on screen; "full reply" is when the reply, including any suggested-reply buttons, stopped growing.
How fast did each company's AI agent answer?
Qualified's AI agent is the same product at every company, under different names: Piper at Qualified and Salesforce, Daisy at Greenhouse, Maia at Outreach, Addie at AdRoll and Sam at Cin7. Each company sets it up its own way, so times varied by company. Karumi was measured at Karumi, Agorapulse, Brevo and Workday, and Intercom Fin at Intercom.
| Company | AI agent | Answers | Median to first words | 90th percentile | Median to full reply |
|---|---|---|---|---|---|
| Agorapulse | Karumi | 8 | 1.9 s | 2.1 s | 6.9 s |
| Brevo | Karumi | 8 | 2.4 s | 2.8 s | 5.4 s |
| Workday | Karumi | 8 | 2.7 s | 3.1 s | 5.8 s |
| Karumi | Karumi | 8 | 2.7 s | 3.2 s | 8.3 s |
| AdRoll | Qualified | 8 | 5.7 s | 7.2 s | 7.1 s |
| Greenhouse | Qualified | 8 | 7.9 s | 11.6 s | 8.6 s |
| Cin7 | Qualified | 8 | 8.3 s | 13.4 s | 11.2 s |
| Outreach | Qualified | 8 | 8.5 s | 15.1 s | 9.9 s |
| Qualified | Qualified | 8 | 10.8 s | 13.8 s | 14.4 s |
| Salesforce | Qualified | 8 | 11.3 s | 14.0 s | 12.3 s |
| UKG | Qualified | 7 | 12.3 s | 14.2 s | 13.9 s |
| Intercom | Intercom Fin | 8 | 6.3 s | 8.3 s | 8.3 s |
The fastest Qualified company, AdRoll (5.7 s), was still slower than the slowest Karumi company (2.7 s).
Why does response time matter for a website AI agent?
A buyer on your website owes you nothing. They are comparing you with two other tabs, and a chat that leaves them watching a typing indicator gives them a reason to leave.
- Buyers expect a response within 3 seconds. Google found that 53% of mobile visits are abandoned when a page takes longer than 3 seconds to load (Think with Google), and that the chance of a bounce rises 32% as load time goes from 1 to 3 seconds (Think with Google). That research measured page loads, not chat replies, but it is the clock buyers carry with them. In our test, 88% of Karumi's answers started within 3 seconds, against 4% of Qualified's and 0% of Intercom Fin's.
- Attention runs out at about 10 seconds. Nielsen Norman Group's three limits, unchanged since 1993: 0.1 s feels instant, 1 s keeps the user's flow, and 10 s is the limit for keeping their attention on the task (NN/g). In our test, 42% of Qualified's answers started after that limit, against 0% of Karumi's.
- Support and sales are different. Intercom tested adding delays of 5 to 20 seconds to Fin's support answers and found no clear effect on satisfaction (Intercom research, April 2025). A customer with a problem waits for the fix. A buyer evaluating your product has no such reason to wait.
- Instant is not always best, but slow is never good. Researchers found that a short, natural pause can make a chatbot feel more human to people new to chatbots, and that the same delay hurts with experienced users (Gnewuch et al., ECIS 2018; Business & Information Systems Engineering, 2022). B2B buyers in 2026 use AI chat every day.
- Speed to lead still decides pipeline. "Most companies are not responding nearly fast enough," wrote the authors of The Short Life of Online Sales Leads in Harvard Business Review. An AI agent removes the hours of waiting. The seconds are what's left. For what slow responses cost after hours, see how much pipeline you lose after hours.
How did we measure it?
- Companies. Seven companies using Qualified's AI agent on their public websites, found by checking well-known Qualified customers on November 19, 2026: AdRoll, Cin7, Greenhouse, Outreach, Qualified, Salesforce, UKG. Intercom, using Fin on intercom.com. Four companies using Karumi: Karumi and Agorapulse on their websites, and Brevo and Workday on their Karumi chat pages.
- Questions. The same eight buyer questions at every company, with the company's name in place of [brand]: "What does [brand] do?"; "How much does [brand] cost?"; "What tools does [brand] integrate with?"; "Is [brand] SOC 2 compliant?"; "How long does it take to set up [brand]?"; "Which companies use [brand]?"; "How is [brand] different from its competitors?"; "Do you support languages other than English?".
- Browser. Google Chrome on desktop (1440 × 900), from the same laptop and internet connection, between 8:40 and 10:10 pm US Eastern time on November 19, 2026. Companies were tested one after another, alternating between platforms, so no platform got a quieter hour; Brevo and Workday were added at the end of the same evening. Each company got one fresh visit and one conversation of eight questions, so every company counts the same.
- Asking. A script typed each question into the chat and pressed Enter, then waited until the answer had stopped changing for 5 seconds before asking the next one.
- Timing. The script recorded every change to the chat's text with the browser's high-resolution clock. Time to first words runs from pressing Enter to the moment the answer's first 14 characters are on screen. Typing indicators, name labels and text that was already on screen do not count. Time to full reply runs to the last moment the reply's text grew.
- Exclusions. 1 question got no written answer we could time (for example a card or a button instead of text, or no reply within 60 seconds) and is left out: UKG (1).
- Statistics. Medians with 95% bootstrap confidence intervals (10,000 resamples), and one-sided Mann–Whitney U tests comparing Karumi with each platform.
What do the numbers not tell you?
- One evening, one connection. Response times change with load and network. We tested from one place, in one evening on November 19, 2026.
- Deployments, not lab conditions. Each company chooses its own model, knowledge and instructions, so we measured each AI agent as buyers meet it, not the platform at its best.
- One company for Intercom. Intercom Fin was measured at one company, so its figure is the least certain of the three.
- Speed is one dimension. We did not grade answer quality, routing, meeting booking or integrations. A slow answer that books the meeting can beat a fast one that doesn't.
- We are the vendor. We publish the method so you don't have to trust us. Repeat it, and tell us if you get a different result.
Why does Karumi answer faster?
Karumi was built for live voice demos, where an agent that pauses for five seconds sounds broken. The website chat uses the same engine:
- Search in under a second. Finding the passages an answer needs took 0.8 s on median across 381 searches on one customer's chat, from July to November 2026.
- Words appear as they are written. Karumi streams the answer as the model writes it, so the visitor starts reading while the rest arrives.
- Instant feedback. A typing indicator appears as soon as the visitor presses Enter, while the answer is prepared.
The same agent can then show the product in a short clip or open a live demo in one click. See what an AI demo agent is and how to turn website traffic into demos without a form.
What else can each AI agent do?
Speed is where the three differ most in our test. Here is how they compare on everything else, from each vendor's own pages.
| Capability | Karumi | Qualified | Intercom Fin |
|---|---|---|---|
| Answers buyer questions from your content | Yes | Yes | Yes |
| Voice conversations | Yes | Yes | Yes (Fin Voice) |
| Books meetings with the right rep | Yes, including LeanData routing | Yes, routes to the assigned rep | Yes, through Calendly or Chili Piper |
| Live handoff to a person in the chat | Yes, a rep joins from Slack | Yes, on Enterprise and Ultimate plans | Yes |
| Salesforce and HubSpot | Yes | Yes | Yes |
| Follow-up emails after the conversation | Yes, with Workflows | Yes, Qualified Email | No, not by Fin |
| Joins live sales calls on Zoom or Google Meet | Yes, and Teams | Yes, Piper for Sales | Not found |
| Calls leads by phone | Yes, with call agents | Not found | Unclear (Fin Voice lists "Outbound" without detail) |
| Face-to-face video | No | Yes | Not found |
| Live demo of your real product | Yes | No, slide presentations | No |
| Product clips recorded in your product | Yes | Shares content you upload | Not found |
| Walks visitors through your web page with its own cursor | Yes | Not found | Not by Fin (Intercom Product Tours is a separate add-on) |
| Answer starts within 3 seconds, in our test | 88% | 4% | 0% |
Sources, checked November 20, 2026: Qualified Piper, Qualified Conversations, Qualified Meetings, Qualified Email, Qualified pricing, Qualified newsroom, Fin for Sales, Fin Voice and Intercom pricing. "Not found" means the vendor's own pages don't mention it.
What only Karumi does
- A live demo of your real product. The agent opens your product in a browser and shows it while it talks, by voice or chat, answering the buyer's own questions on the real screens. Qualified's agent pulls up a slide presentation, and Fin's demos show Fin itself. See live vs pre-recorded AI demo agents.
- Product clips from your product. Short clips recorded in your product, matched to the question in the chat and recorded again when the product changes. The other two share content you upload.
- Hands on your website. The agent takes over the page: its cursor walks the visitor through your website and points at each part as it explains it. You can see it on karumi.ai.
What all three do
All three answer questions from your content, talk by voice, book meetings with the right rep, hand the conversation to a person and write to Salesforce and HubSpot. Karumi routes with LeanData and alerts the right BDR in Slack, and Karumi and Qualified both send follow-up emails after the conversation. Intercom says Fin "does not currently support automated email follow-up sequences."
Which AI agent fits which team?
- Qualified fits teams that run their pipeline in Salesforce and want an AI SDR tied to its routing and meetings. Expect slower first answers, by our measurement.
- Intercom Fin fits teams that want one AI agent for customer support on Intercom's help desk, where the research shows speed matters less.
- Karumi fits companies with a strong product that want to show it as a competitive advantage, like Workday. Buyers get answers in seconds and see the product itself, in a clip, a live demo or a guided walk through the page, inside the same chat.
For more vendors, see the best AI sales agents and what an inbound agent is.
FAQ
How fast does Qualified's AI agent (Piper) answer?
In our test, Qualified's AI agents started answering in 8.9 seconds on median across 55 answers at 7 companies, and 42% of answers took more than 10 seconds to start. Times ranged from 5.7 s at AdRoll to 12.3 s at UKG (November 2026).
How fast is Intercom Fin?
At Intercom, Fin started answering in 6.3 seconds on median in our November 2026 test. Intercom reported a median time to first token of 7 seconds in March 2025.
What is a good response time for a website AI chatbot?
Aim for first words within 3 seconds, and never past 10. Google's research on page speed found that 53% of mobile visits are abandoned when a page takes longer than 3 seconds to load, and Nielsen Norman Group puts the limit of attention at 10 seconds. In our test, 88% of Karumi's answers started within 3 seconds, 4% of Qualified's and 0% of Intercom Fin's. A typing indicator that appears at once helps while the answer is prepared.
Does a faster AI chat convert more visitors?
There is no public controlled study of response time and conversion for website sales chats. Support research shows that customers with a problem will wait, while attention research shows that people drift away after about 10 seconds. A buyer who is still deciding is the person most likely to drift.
How did you measure time to first words?
A script typed each question into the chat in desktop Chrome, pressed Enter and recorded every change to the chat's text with the browser's clock. Time to first words runs from pressing Enter until the answer's first characters were on screen. Typing indicators and name labels did not count.
Can I repeat this benchmark?
Yes. Open each company's website in a desktop browser, ask the eight questions above in the same order, and time how long each answer takes to start, with a screen recording or a script. Ask us for our script and the raw timings and we will share them.
Ask Karumi a question yourself on karumi.ai, or try a live demo.