Coventry were leading 1-0 at the 87-minute mark on Wednesday (Thursday AEST) and looking much the better side when substitute Djed Spence slid home an equaliser.
Johnson, who came on after 17 minutes as a replacement for injured Wilson Odobert, then fired past Ben Wilson five minutes later to turn the match on its head for a 2-1 scoreline.
The result came as a huge relief to Tottenham's Australian manager Ange Postecoglou, who had made headlines before the match with the assertion that he always won a trophy in his second season at a club.
In the night's other clash, Brighton and Hove Albion beat Premier League rivals Wolverhampton Wanderers 3-2 on the south coast.
Until the late goals, Coventry had looked headed for a deserved upset.
Brandon Thomas-Asante side-footed the hosts ahead in the 63rd minute of the third-round clash, with Tottenham back-up goalkeeper Fraser Forster making his first appearance of the campaign.
Tottenham had gone through the first half without a single shot on goal and left regulars Son Heung-min, James Maddison and Dejan Kulusevski on the bench.
Jack Rudoni tested Forster in the opening minute, forcing a save, and had another good opportunity after 38 minutes when Destiny Udogie made a crucial block.
Tottenham started the second half looking even more vulnerable, with defender Ben Davies saving a certain goal from Haji Wright after a glaring misjudgment by Forster, who had raced out to clear and left his goal completely exposed.
Kulusevski came on in the 74th minute for Timo Werner and provided the through-ball for Spence.
Son got the nod in the 62nd minute with Maddison as Postecoglou recognised the need for more creativity.
"It was a tough game. Credit to Coventry, I thought they were outstanding," the Australian said of last season's FA Cup semi-finalists.
Tottenham coach Ange Postecoglou was proud of how his side weathered Coventry's challenge. (AP PHOTO)
"It was a cup tie and they really took it to us. We had to just hang in there somehow and we did."
The win came after two losses in the league, including a 1-0 home defeat to Arsenal on Sunday.
"I don't think there's an easy or quick fix to where we want to get to and one result doesn't change that," Postecoglou said when asked about the significance of the result.
"Tonight, at least in the end there, we showed what we have been missing in the first four games, that little bit of spirit to find our way to win.
"We've just lacked that little bit of an edge and hopefully tonight gives the boys a bit of belief."
At Brighton, Carlos Baleba opened his account with the Seagulls on 14 minutes, before Simon Adingra shot low for the second goal in the 31st minute.
Goncalo Guedes pulled one back for the visitors a minute before the break with a deflected side-footed shot, before Ferdi Kadioglu made it 3-1 in the 84th minute on the rebound.
Tommy Doyle then lashed home off the underside of the bar in stoppage time for a nervous finish.
The remaining third-round ties will be played next week, with holders Liverpool at home to Premier League rivals West Ham United on September 25.
