British newspaper The Times reported that Japan's government has agreed the Games are doomed and that Tokyo is now seeking to host them in 2032, the next available year.
The newspaper cited an unnamed senior member of the ruling Japanese coalition.
The focus now is to "find a face-saving way of announcing the cancellation that leaves open the possibility of Tokyo playing host at a later date," the Times reported.
"No one wants to be the first to say so but the consensus is that it's too difficult," the source told the Times. "Personally, I don't think it's going to happen."
Japan's Deputy Chief Cabinet Secretary Manabu Sakai denied the report at a news conference.
"We would like to fully deny [the report] by saying there is no such fact," Sakai said.
"Of course we have to take into account the situation abroad, and we will decide on whether to actually hold the event at some point, but until then, the Japanese government will do what needs to be done," Sakai said.
The coronavirus pandemic already forced the International Olympic Committee (IOC) and Japan to delay the 2020 Games by one year to this summer.
And the IOC on Friday also labelled the report in the Times as "categorically untrue," adding that it was "fully concentrated on and committed to the successful delivery" of the Games.
The postponed Olympics are set to start on July 23 and the Paralympics on August 24.
However, Tokyo - the Olympic host city - has been under a state of emergency as it has suffered a massive spike in coronavirus infections in recent months.
