The Japanese had won nine consecutive World Cup qualifiers without conceding but Shogo Taniguchi put Lewis Miller's low cross into his own goal in the 58th minute to give the Socceroos an unexpected lead.
Popovic, in his second game in charge since replacing Graham Arnold last month, had set his side up in an ultra-defensive formation and looked to hold on to their advantage.
But substitute Keito Nakamura eventually unlocked the Socceroos' defence with 14 minutes remaining, dancing through the backline before powering the ball across the face of goal for Cameron Burgess to deflect into his own net.
The result moves Japan, looking for an eighth successive World Cup qualification, onto 10 points from four games with the Australians five points behind in second place and Saudi Arabia and Bahrain, who play later on Tuesday, a point further adrift.
"My experience as part of this team is that in the most difficult moments that's when the boys stand up, no excuses," said Australia captain Jackson Irvine.
"To put in a performance like that tonight, I'm very proud of everybody. The calmness and composure we had to show, having to defend in hard moments right to the very last ball, it's a top foundation for us to move forward with."
China picked up their first win in Group C as Branko Ivankovic's side defeated Indonesia 2-1 in Qingdao with first half goals from Behram Abduweli and Zhang Yuning securing the three points.
South Korea, who have appeared at every World Cup since 1986, secured a 3-2 win over Iraq in Seoul as Hong Myung-bo's side registered a second win in a row to move clear at the top of Group B.
Oh Se-hun put the Koreans ahead in the top-of-the-table clash four minutes before half-time but Aymen Hussein's overhead kick five minutes into the second half levelled the score.
Oh Hyeon-gyu restored Korea's lead with 16 minutes remaining and Lee Jae-sung scored his side's third nine minutes later to leave Ibrahim Bayesh's 95th minute goal as nothing more than a consolation for Jesus Casas' team.
Later on Tuesday, Group A leaders Uzbekistan will host the United Arab Emirates while second-placed Iran take on Asian Cup holders Qatar and Kyrgyzstan entertain North Korea.
Jordan will take on Oman in Group B with Palestine facing Kuwait.