Reinforcement learning is how a robot learns by trying things and getting a score. Do something good, get points. Do something bad, lose points. Do this a million times and the robot starts choosing the moves that get the most points. Pieter Abbeel's lab once taught a robot arm to fold a towel this way, which sounds simple until you realize a towel is one of the hardest objects in robotics — it changes shape every time you touch it.

If your robot got the same score every time, no matter what it did, would it ever get better?

Use Scratch or a paper game. Make a robot that moves in four directions. Give it +10 points for reaching a goal square and -1 for every step it takes. Run it 10 times and track the score.

import random

# States: 0 = start, 3 = goal
# Actions: 'right' moves +1, 'left' moves -1
Q = {0: {'right': 0, 'left': 0}, 1: {'right': 0, 'left': 0}, 2: {'right': 0, 'left': 0}}

def step(state, action):
    nxt = state + (1 if action == 'right' else -1)
    nxt = max(0, min(3, nxt))
    reward = 10 if nxt == 3 else -1
    return nxt, reward

for episode in range(50):
    state = 0
    while state != 3:
        action = random.choice(['right', 'left'])
        nxt, reward = step(state, action)
        Q[state][action] += 0.1 * (reward - Q[state][action])
        state = nxt

print(Q)  # Notice 'right' wins at every state

Be the robot. Sit in a chair while someone hides a small object in the room. You can only ask 'warmer' or 'colder' as you move around. Count your moves to find it.

Change the scoring so the robot is rewarded for being fast, not just for finishing. Watch how its strategy changes.

What did your robot learn that you didn't expect?