QlearnAI

5 min

Getting started: what an AI chatbot really is

See what a chatbot is, where it fits, and how it builds a reply one small piece of text at a time.

By the end of this lesson you can

  • Say in one sentence what an AI chatbot is
  • Show where the language model sits between you and the reply
  • Describe how a chatbot writes a reply, one token at a time

An AI chatbot is an app that answers your messages in everyday language. Underneath it sits a language model: a program trained to predict what text comes next.

The problem it solves: you can ask in your own words. "Explain this lease clause like I'm new to renting" gets you one plain answer, instead of ten search results to read and stitch together yourself.

Where it fitsYou type a messageChat appLanguage modelExtras: web search, saved notesReply, one token at a timeSends the model your chat sofar, plus anything its extrasfoundPredicts the reply. This isthe part the course opens up

The mental model for this whole course: a very well-read autocomplete. Your phone keyboard guesses your next word. A chatbot does the same job at a far bigger scale, after training on a huge amount of text. Every lesson comes back to this picture.

Phone keyboard: suggests the next word. Chatbot: suggests the next word, but it read a huge pile of text first Picture: Split scene: on the left a small, sleepy cartoon phone with a tiny keyboard; on the right the same phone character wearing graduation clothe

When not to use it: when you need a fact you can't easily check, like a medicine dose, a legal deadline or an exact quote. Use it for explanations, drafts and ideas. Check facts against an original source, or ask a professional.

You ask a chatbot to explain photosynthesis like you're five. It writes a perfect little paragraph in two seconds. Somewhere a biology teacher feels a chill. So what is it actually doing in there? Thinking? Searching? Copying? Some apps even show a "thinking" step. That's the model predicting notes to itself before the reply. So: none of the above, mostly.

Take a guess

When a chatbot writes a reply, what is it doing at each step?

No pressure. Guess now, and the lesson will show you if you were right.

Zoe

ok so it googles the answer and rewords it, right

Dev

nope. the model writes the reply one small chunk at a time. each chunk is a token

Zoe

a token is a word?

Dev

often a whole word. longer words get split into pieces

Zoe

so it's literally autocomplete

Dev

a ridiculously good one. it scores every possible next token, picks one, adds it, then does it again

Zoe

then why did it give me two different answers to the same question

Dev

the pick has a bit of randomness on purpose. different early picks, different reply 🎲

The chatbot choosing the next word of your essay: rolls weighted dice, again, for every single token Picture: A cartoon robot at a desk rolling a pair of oversized dice with some faces painted much bigger than others, a half-written essay beside it,

Here's the mechanism. Your text is split into tokens. In English, a token averages about three-quarters of a word. At each step the model gives every possible next token a probability. One gets picked, usually a likely one but not always the top one. A setting called temperature controls how adventurous that pick is. Low: safer and more repetitive. High: more varied, and more likely to wander off.

Predict, then reveal

Plot twist time. Everyone assumes the chatbot reads your message letter by letter, like you do. Does it?

Chatbot: writes you a sonnet on request. Same chatbot: sometimes miscounts the r's in strawberry Picture: A cartoon robot squinting at a giant strawberry cut into two big chunks, scratching its head, with a confused but confident face.

Why can you get two different answers to the exact same question?

Pick one answer.

What is a token?

Pick one answer.

FlashcardsIndex cardsFlashcards ⚡

Card 1 of 5

Worth keeping

  1. A chatbot is an app on top of a language model: a very well-read autocomplete.
  2. Replies are built one token at a time, each picked from likely options. Nothing is looked up by the model itself.
  3. Each token is picked with some randomness, and a setting called temperature controls how much. That's why the same question can get different answers.
  4. Use it for explanations, drafts and ideas. Check facts against an original source.

Okay, it predicts the next token. But predicting "the capital of France is…" means it has to know Paris. Where does that come from, if there's no database to look it up in?