Klopp's side were on course for an early Christmas present at the Emirates Stadium on Friday after goals from Philippe Coutinho and Mohamed Salah put the visitors in control.
But Liverpool suffered a costly meltdown as Arsenal scored three times in five minutes in the second half through Alexis Sanchez, Granit Xhaka and Mesut Ozil.
The Reds still escaped with a point thanks to Roberto Firmino's equaliser, but it was a missed opportunity to cement their place in the Premier League's top four.
It was not the first time Klopp has overseen a Liverpool performance high on attacking quality but low on defensive solidity.
After Liverpool midfielder James Milner said his team need to be more boring, Klopp demurred when asked if he would be willing to use more conservative tactics in future.
"Controlling the game is what he means, but that was not the situation," Klopp said.
"Just about everyone was around the box when they scored but we didn't react well and they scored.
"It's nothing about controlling the game. I thought we did control the game but we opened the door and Arsenal are a real threat.
"They have this quality. That's why you have to avoid this situation," he said.
"We can't talk too much about defence because we should have scored four. Coming here and creating these type of chances is not easy.
"It's difficult to explain but good to learn from because the players need to react differently
"We opened the door in those five minutes. We need to be angry."
Given it was such a dramatic evening, it was no surprise the always emotional Klopp lost his cool on the touchline.
- 'Fantastic game' -
Celebrating Liverpool's third goal with abandon, Klopp angered Arsenal fans when he turned towards them and one threw a bottle in his direction.
"I was happy, maybe aggressive happy. I apologised, but if the people apologised to me for everything they said it would be a long time. Nothing serious happened," he said.
Gunners boss Arsene Wenger was frustrated his side's epic comeback didn't produce a valuable victory in their battle to qualify for the Champions League via a top four finish.
They remain a point behind fourth placed Liverpool but Wenger admitted it was a privilege to be part of the part of the Premier League's game of the season.
"To be completely honest it could have been game over at half-time. We had a nightmare first half and could have been punished more," he said.
"I must give credit to the players because even at 2-0 we didn't show any weakness and kept coming back.
"I'm frustrated we couldn't hold on but it was a fantastic game with quality going forward on both sides."
Taking a swipe at more defensive managers like old rival Jose Mourinho at Manchester United, Wenger added: "I don't know what you mean about parking the bus. It has nothing to do with football.
"We defended well but they have quality everywhere going forward."
Wenger also revealed Arsenal midfielder Jack Wilshere is close to signing a new contract, capping an unexpected return to the line-up following a loan spell at Bournemouth last season that seemed to spell the end of his time in north London.
"It might be sorted in a few weeks. We'll see. Basically we want to keep these players," Wenger said.