Foundations, Models & Prompting

The phrase “artificial intelligence” was coined in 1956 at a summer workshop at Dartmouth College. For the next fifty years, progress came in waves. Early researchers hand-wrote rules — expert systems that tried to capture human knowledge as long lists of “if this, then that.” They worked in narrow domains and then hit a wall, and funding dried up in periods now called the AI winters.

The modern era runs on a different idea: instead of writing the rules, you let a system learn patterns from data. That is machine learning, and its most powerful form — deep learning with neural networks — took off in the 2010s as computers got fast enough and data got plentiful enough. The pivotal moment was a 2017 research paper that introduced the transformer, the architecture behind today’s chatbots. When that was scaled up on enormous amounts of text, the generative-AI boom of 2022 followed: ChatGPT, Claude, Gemini, and the tools you are here to master.

Why now? Three things arrived together — enough data, enough computing power, and an architecture that scales. None alone was enough; the combination changed everything.

A large language model (LLM) does one deceptively simple thing: it predicts the next chunk of text. Given everything so far, it estimates the most likely continuation, adds it, and repeats. “Chunks” are called tokens — roughly word-pieces. That is the whole trick. Everything that feels like reasoning, writing, or conversation emerges from next-token prediction done extraordinarily well.

There are two phases. Training happens once, ahead of time: the model reads a huge slice of the internet and books and adjusts billions of internal numbers (its weights) until its predictions are good. Inference is what happens when you use it: the weights are frozen, and the model simply runs its prediction over your prompt. This is why a model has a knowledge cutoff — it only knows what was in its training data, up to a certain date.

Two consequences matter enormously for using AI well. First, an LLM is not a database and not a search engine. It does not “look things up”; it generates plausible text. Second, when it lacks the real answer, it will often produce a confident, fluent-sounding wrong one — this is a hallucination. The model is not lying; it is doing exactly what it was built to do — predicting likely text — and likely text is not always true text. Your job as a power user is to know when that gap matters and to verify.

A plain chatbot gives you one reply and stops. An agent runs a loop: perceive → think → act → observe → repeat. It reads the situation, decides on a step, takes an action — searching the web, running code, calling a tool, editing a file — then looks at the result and decides the next step. It keeps going until the goal is met.

That loop is the whole difference between “a model that answers questions” and “a system that gets things done.” A chatbot can tell you how to book a trip; an agent can check dates, compare options, and fill the form. The loop is also where autonomy — and risk — comes from, which is why later in the week we spend real time on what you should and should not let an agent do on your behalf.

Three dials control every model, and understanding them turns confusing behaviour into something you can predict.

Tokens are the pieces of text the model reads and writes — a token is about three-quarters of a word. You are billed by the token, and limits are counted in tokens.

The context window is how many tokens the model can “hold in mind” at once — your prompt plus its reply plus the conversation so far. When a chat gets long and the model seems to forget what you said earlier, you have overflowed the window: the oldest text falls out of view. The fix is not to repeat yourself endlessly but to re-state the essentials, or start fresh with a tight summary.

Temperature controls randomness. Low temperature makes the model pick the single most likely next token every time — good for consistency, code, and facts. High temperature lets it wander into less likely choices — good for brainstorming and creative writing. This is why the same prompt can give different answers twice: unless temperature is zero, there is deliberate randomness in the pick.

There is no single “AI.” There is a landscape, and power users choose deliberately within it.

Open vs. closed. Closed models (like GPT and Claude) run on the maker’s servers; you use them through an app or an API and cannot see the weights. Open-weight models (like Meta’s Llama, Mistral, DeepSeek, and Qwen) can be downloaded and run yourself. Closed models are often the most capable and the most convenient; open models give you control, privacy, and no per-use fee. Neither is “better” — they trade off differently, and by Friday you will have run an open model on your own laptop.

The major families. GPT (OpenAI), Claude (Anthropic), Gemini (Google), Llama (Meta), plus Mistral, DeepSeek, and Qwen. They differ in tone, strengths, price, and openness — which is exactly what your peer mini-teach on Wednesday will map out.

Modalities. Models specialise by what they handle: text, images, video, audio and voice, and speech-to-text. A multimodal model does several at once — it can see an image, read text, and reply in words. Multimodal is the direction of travel, and it is the whole theme of Day 2.

Reasoning models. Some models are tuned to “think” before answering — working through a problem step by step, visibly or invisibly, before giving a final answer. They are slower and cost more, and they are worth it for hard, multi-step problems and not much else. Matching the model to the task — small and fast when that is enough, large and thoughtful when it is not — is a core power-user skill.

A prompt is not a magic spell; it is a brief. The habit that separates power users from casual users is giving the model a complete brief, and there is a simple framework for it:

Role · Task · Context · Constraints · Examples · Format. Tell the model who to be (“You are a careful copy editor”), what to do, the background it needs, the limits (length, tone, what to avoid), an example or two of what good looks like, and the exact output shape you want (a table, a checklist, JSON, three bullet points). The more of these you supply, the less the model has to guess — and guessing is where weak answers come from.

A system prompt is a special, persistent instruction that sits above the whole conversation and sets the model’s persona and rules — it applies to every message, not just one. Writing your own is how you turn a general chatbot into a tool that behaves consistently: “Always answer in plain English, flag anything you are unsure about, and never invent a citation.”

Finally, treat AI as a conversation, not a vending machine. Your first prompt is a draft. Read the reply, correct it, add the constraint you forgot, ask for a different format — iterating two or three times reliably beats trying to write one perfect prompt. In the lab, you will do exactly that.

Concepts stick when you use them. Work through today’s lab and peer exercise now, and the homework and essay are due Wednesday.