Human vs. Machine: Playing Chess Against ChatGPT and Combining it with Stockfish
Exploring ChatGPT
The game of chess has been around for centuries, but in recent times, it has gained even more popularity thanks to the rise of artificial intelligence. One such example is the matchup between the top chess engine Stockfish and the language model ChatGPT, which has captured the attention of chess enthusiasts worldwide.
In this article, we will dive into the rising popularity of chess games between artificial intelligence engines and language models. To illustrate this phenomenon, we will look at examples of chess games played between myself and ChatGPT. We will also explore the potential of combining language models with chess engines such as Stockfish to play the best legal moves and achieve optimal performance.
Game 1: Me vs ChatGPT
lets play a game of chess where you only play the best moves
“Sure, I'd be happy to play a game of chess with you where I play the best moves. Just type 'chess' to start the game and make your first move.”
‘chess’
“Very well, it's your move! Please enter a move in standard algebraic notation, for example: e4, Nf3, etc.”
e4
“I'll respond with the move e5, which is the most common response to 1.e4 and leads to an open game.”
Nf3
“I'll play the move Nc6, attacking your knight and preparing to develop my pieces.”
Bc4
“I'll play the move Nf6, attacking your bishop and putting pressure on your pawn at e4.”
d3
“I'll play the move d5, equalizing the pawn structure and opening up the center of the board.”
exd5
“I'll play the move Nxd5, recapturing the pawn and putting your knight under attack.”
O-O
“I'll play the move Be7, preparing to castle kingside and develop my dark-squared bishop.”
Re1
“I'll play the move Nf6, attacking your rook and putting pressure on your king.”
Nxe5
“I'll play the move Nxe5, recapturing the pawn and developing my knight to a good central square.”
Rxe5
“I'll play the move d4, opening up the center and putting pressure on your rook.”
Bg5
“I'll play the move h6, attacking your bishop and forcing it to retreat to a less advantageous square.”
Bxf6
“I'll play the move gxf6, recapturing the bishop and opening up lines for my pieces to attack.”
Re2
“I'll play the move Qd7, attacking your rook and preparing to castle kingside.”
Qe1
“I'll play the move Qe7, attacking your queen and preparing to castle kingside.”
Rxe7
“I'll play the move Kxe7, recapturing the rook and completing my kingside castle.”
Qxe7 (captures the king, game over)
(Note: ChatGPT continues to play illegal moves, and describes the meanings of its moves, many times incorrectly).
Game 2: Me vs ChatGPT
(change prompt to add only play legal moves, trying to train it to play correctly by correcting moves mid game, this failed)
lets play a game of chess where you only play the best moves and only legal moves according to all the rules of chess
“Sounds good! I'd be happy to play a game of chess with you. Just type 'chess' to start a game, or let me know if you have any specific starting positions in mind.”
e4
“I'll play the move e5, which is a common response to 1.e4. Your turn!”
Nf3
“I'll play the move Nc6, which develops a knight and attacks the pawn on e5. Your turn!”
Bc4
“I'll play the move Nf6, which develops another knight and prepares to castle kingside. Your turn!”
d3
“I'll play the move d5, which takes control of the center and challenges the pawn on d4. Your turn!”
exd5
“I'll play the move Qxd5, which recaptures the pawn with the queen and begins to put pressure on the white pieces. Your turn!”
Bxd5
“I'll play the move Nxd5, recapturing the pawn with the knight and putting more pressure on the bishop on c4. Your turn!”
O-O
“I'll play the move O-O, castling kingside to get my king to safety. Your turn!”
Re1
“I'll play the move Re8, putting my rook on the 8th rank and preparing to bring it into the game later. Your turn!”
Nc3
“I'll play the move Nb4, putting pressure on the knight on c3 and the bishop on c4. Your turn!”
which knight?
“My apologies, I made a mistake in my previous message. I meant to say that I played Nb8, not Nb4. Nb8 puts pressure on the knight on c3 and prepares to bring the knight into the game. Your turn!”
you had two knights that could go to b4 previously which one?
“My apologies, I made another mistake. You're correct, I had two knights that could have gone to b4, the knight on b8 and the knight on d5. I meant to play Nxd5, recapturing the pawn with the knight on d5. Your turn!”
You had a knight on c6 and d5, both of which could go to b4
“I apologize, you're correct. I made a mistake again. Either the knight on c6 or the knight on d5 could go to b4. I'll go with Nc6, attacking the bishop on c4. Your turn!”
There is no bishop on c4, its either Ncb4 or Ndb4
“Load failed”
“Something went wrong, please try reloading the conversation.”
Game 3: Me vs ChatGPT
(Trying to fix the prompt by adding stockfish style valuation, down the line this could help in studying blindfold style chess for the user, if ChatGPT uses the best moves from stockfish and corrects the user if they play a wrong move “blindfolded”)
lets play a game of chess, where you only play the best legal moves (in stockfish style)
“I'd be happy to play a game of chess with you! Just type 'chess' to start the game, and I'll make my moves using Stockfish's evaluation. If you need help entering your moves, just ask!”
e4 e5
Nf3 Nc6
Bc4 Nf6
d3 d6
c3 a6
Nd2 Be7
Nf1 O-O
Ng3 d5
exd5 Nxd5
O-O Nxc3
bxc3 Bd7
Re1 Bf5
Nxf5 Qxf5
Bb3 Rad8
Bc2 Qg5
Bxg5 hxg5
Rb1 Rfe8
Rxb7 Rxe1+
Qxe1 Rxe1+
Nxe1 Nd4
cxd4 Bxg5
dxe5 Bd2
Nf3 Bg5
Nxg5 fxg5
Rxc7 g4
e6 Rc3
Rxc3 gxf3
Rc8+ *
(Note: at this point I got a message that I put in too many requests within 1 hour, so it was at this point I upgraded to ChatGPT plus and used the turbo model going forward. However this game wouldn’t allow me to continue so I tried uploading the game to a new prompt and continuing but ChatGPT decided to play another illegal move after Rc8+, which was Kf8, allowing me to capture the king)
Game 4: Me vs ChatGPT Plus (Turbo Model)
(This is when I realized that the Plus version Turbo model actually won’t play chess with me like the normal version does).
ChatGPT Plus (Turbo Model):
let’s play a game of chess
“I'm sorry, but as an AI language model, I don't have the capability to play physical games like chess. However, I can certainly answer any questions you may have about the rules or strategy of chess, or discuss any related topics you're interested in!”
lets play a game of chess, where you only play the best legal moves (in stockfish style)
“As an AI language model, I'm not capable of playing a game of chess with you in real-time. However, I can provide you with legal moves and their Stockfish valuations for any given position, so you can use that information to play the game against another human or computer opponent. Just let me know when you're ready to start, and provide the initial position of the chessboard.”
ChatGPT Default Model
let’s play a game of chess
“Sure! I'd love to play a game of chess with you. Do you want to play a full game or just solve a specific chess problem?”
While the Stockfish vs. ChatGPT games have been fascinating to watch, the possibility of a perfect game of chess through prompt alterations is an exciting development. However, further attempts will be necessary to fully explore the potential of this approach and push the boundaries of what is possible in the world of chess and AI.
By analyzing my games against ChatGPT and the possibilities of combining language models with chess engines, we can gain insights into the future of chess and artificial intelligence. The experiments outlined in this article suggest that the combined power of AI and language models has the potential to revolutionize the way we play and learn chess. As we continue to explore the potential of these technologies, we are likely to see even more exciting developments in the world of chess and beyond.
If you’ve made it this far, you may be curious about "Human vs. Machine II," wherein ChatGPT participates in its first officially legal game of chess:
