The Clarets began the game just one place above the relegation zone, but gave their chances of beating the drop a huge boost when Ashley Barnes was tripped by Alisson Becker seven minutes from time and then beat the Brazilian from the penalty spot.
Klopp's men have now failed to win in five consecutive league games to slip to fourth in the table, six points behind leaders Manchester United.
Liverpool had not tasted a league defeat at Anfield since April 2017.
Since then the Reds have ended a 30-year wait to win a league title, but their chances of repeating that feat are fading rapidly after a disastrous run over the past month.
"We lost the game, which I think is pretty impossible to lose, but we did it," said Klopp.
"That is my fault because it is my job so that the boys have the right feelings, like they have the right amount of confidence."
The champions have been disrupted by a host of defensive injuries, but it has been scoring goals that has cost Liverpool of late.
Despite having 72 efforts on goal, the Reds have now failed to score in four consecutive league games.
"In the final moment our decision making is not right, that's the problem," added Klopp.
"We left the game open and we lost the game. I have to say I can't believe we lost it, but it's the truth and we have to accept that."
Origi miss
Klopp reacted to Liverpool's longest Premier League goal drought for 16 years by dropping top scorer Mohamed Salah and Roberto Firmino to the bench.
But Divock Origi underlined why the champions have been reliant on their habitual front three when he smashed off the underside of the bar with just Nick Pope to beat after an uncharacteristic error from Ben Mee.
The Belgian has scored a number of important goals in big games for Liverpool, most notably netting twice in a 4-0 Champions League semi-final win over Barcelona two seasons ago. But Origi has scored just one league goal since December 2019.
Salah and Firmino were introduced to partner Sadio Mane early in the second half, but Burnley held out despite incessant Liverpool pressure.
Pope saved brilliantly from Salah at his near post before Firmino showed why he has struggled for goals all season by slicing horribly wide from Andy Robertson's inviting cut-back.
Burnley had scored just once themselves in their last four games.
But Sean Dyche's men took advantage of indecision in the Liverpool defence when Alisson upended Barnes, who coolly slotted home Burnley's first penalty of the season.
"Our shape and our diligence was excellent," said Dyche, whose side move up to 16th and seven points clear of the relegation zone.
"I've always believed in the side to find a moment. Barnes gets us a penalty -- we don't get many -- and slotted it away very well."
Mee then ensured the visitors held on for a first win at Anfield since 1974 when he diverted another Firmino effort off the line in the final minute.
After Sunday's 0-0 draw with United, Klopp claimed Liverpool's focus had to be on a top-four finish and Champions League qualification rather than the title given their current form.
The champions could find themselves down in seventh by the time they are next in league action.
After a daunting visit to Manchester United in the FA Cup on Sunday, Liverpool face five of the top seven in their next six league games, starting with a trip to Tottenham next Thursday.
