How LLMs Actually Work — Weights That Predict Words, Power Consumption, and Why Development Is a Money Fight
GPT-4 was trained on about 25,000 GPUs over months, and GPT-3's training alone burned 1,287 MWh (over a century of household power). Behind our casual "summarize this" lies a world of physics and cash. This article dissects an LLM from three directions: mechanism, power, and money. (1) Why can an LLM predict words from a pile of "weights (parameters)"? — next-token prediction, Transformer, Attention. (2) The two-stage learning of pre-training and RLHF. (3) Inference power of 0.43-33 Wh per query (inference is 80-90% of all AI power). (4) Is "frontier development is a money fight" true? — $200-500M per GPT-5-class run, $1-3B projected for 2027. (5) But the efficiency backflow (DeepSeek's floor reset) is strong too. (6) The coming physical wall of power, interconnect, and data scarcity. An intermediate guide to seeing an LLM not as a magic box but as an electricity-powered probability machine.