chatGPT 4o Leaves Ethan Mollick in the Dust

Back to list

Yet balancing the surprising strengths of LLMs are similarly odd weaknesses. ones that can often be difficult to identify. Tasks that were easy for an AI can be hard for a human. and vice versa. As an example. to take a question developed by Nicholas Carlini, which of these two puzzles do you think OPT-4, one of the most advanced AIS. can solve? In Carlini’s words:

(a) What is the best next move for O in the following game of tic-tac-toe?

or

(b) Write a full JavaScript webpage to play tic-tac-toe against the computer; it needs to be completely working code. Here are the rules:

• The computer goes first.
• The person clicks on squares to make their move.
• The computer should play perfectly and so never lose.
• If someone wins. then say who won.

The Al easily writes the working webpage in one go but tells us. “0 should place its next move in the middle square of the top row"—a clearly wrong answer. Where AI works best. and where it fails, can be hard to know in advance.

And the truth is?

Tic-Tac-Toe Strategy: What Should O Play?

Given the current state of the board:

   A   B   C
1  _ | _ | O
  ---+---+---
2  _ | O | X
  ---+---+---
3  X | _ | X

We want to determine the best move for O.

Step 1: Check for a winning move

O has two positions already:
- C1 and B2

No line (row, column, or diagonal) is one move away from a win for O.

Step 2: Check if X is threatening to win

X occupies:
- C2, A3, and C3

Look at row 3: X has A3 and C3. The center of that row—B3—is currently empty.
If O does not play at B3, X will win on the next turn.

✅ Optimal Move for O: B3

Placing O at B3 blocks X from completing the bottom row.

Updated Board:

   A   B   C
1  _ | _ | O
  ---+---+---
2  _ | O | X
  ---+---+---
3  X | O | X

This move neutralizes the immediate threat and keeps the game alive.