It looked like a replay was on the cards, despite the high-flying Championship hosts playing most of the game with just 10 men.
Captain Grant Hanley received an early red card, although both sides had chances to settle what was an absorbing contest.
But it was the Blues who came out on top, with Green finding the net right at the death to spark scenes of jubilation from the 2,493 travelling fans.
Kenny Jackett made two changes from the side that beat AFC Wimbledon in the league on New Year’s Day.
Dion Donohue replaced Brandon Haunstrup to make his first appearance since late October, while Oli Hawkins made way for Brett Pitman up front.
It was a more unfamiliar line-up for the hosts, who made eight changes to rest several key men for their quest to reach the Premier League.
The game took a while to get going, although Tom Naylor would have breathed a sigh of relief when Jordan Rhodes fouled Jack Whatmough while trying to race on to an under-hit back-pass.
But the hosts were dealt a massive blow on 15 minutes, when their skipper was dismissed by referee Darren Bond.
Hanley sent Ronan Curtis clattering to the turf and, with no other defender behind him, the official had no choice but to brandish a red card.
And the Blues should have immediately made the most of their numerical advantage, with Pitman heading Jamal Lowe’s cross over from just a few yards out.
Christoph Zimmerman then went close at the other end, hitting a long-range piledriver that bent narrowly past the upright.
A curling free-kick from Ben Marshall was comfortable for Craig MacGillivray, while Whatmough made a key challenge to deny Rhodes.
But the flow of the game was being disrupted by a series of niggly fouls from both teams, while Donohue had to be cleaned up after receiving a nasty cut to the head.
There was not an abundance of clear-cut opportunities, but the Canaries went close from a Kenny McLean free-kick.
The ball ricocheted into the path of Rhodes, with Matt Clarke using his face to block the striker’s shot and Ben Godfrey’s follow-up going straight at MacGillivray.
Pompey soon had a chance at the other end when Donohue’s delivery was laid off by Curtis, but a fierce effort from Gareth Evans was blocked before it could test Michael McGovern.
Norwich City 0
It was the visitors who made the brighter start to the second half and they created a number of openings just after the restart.
Naylor’s skidding effort was held by McGovern, while Pitman had a shot on the half-volley that looped over.
Donohue then sent in a dangerous cross from the left and Curtis glanced his header just past the post.
But Norwich fought back into the contest and Todd Cantwell drilled a shot narrowly past the post from 20 yards.
It looked like Dennis Srbeny was then going to slot the hosts into the lead, only for Anton Walkes to make a superb challenge just as he was shaping to shoot.
The match was finely poised, with the Canaries starting to see more of the ball and their opponents posing a threat on the break.
Jackett decided the time was right to make a change on 67 minutes, with Green coming on for Evans.
He made a lively start, although after skipping past one challenge in the Canaries area, could not repeat the trick.
Both sides looked capable of settling the tie and it was Pompey who almost broke the deadlock on 77 minutes.
McGovern failed to hold a drilled effort from Donohue, with Zimmermann just about getting there in front of Pitman to clear the danger.
An attempt from Lowe also had to be clawed behind by the diving keeper, as the Blues started to apply pressure.
MacGillivray then flung himself to the right at the other end, in order to keep out a diving header from Cantwell.
A replay might not have been what either side was looking for, but it looked on the cards until right at the death.
Donohue broke quickly and hit a superb cross-field pass for GREEN, who did not take the best first touch, but recovered well to hit a shot in off the post.
It left the hosts stunned and there was not enough time remaining for them to find an equaliser, with Pompey booking their place in Monday evening’s fourth round draw.
Norwich (4-1-4-1): McGovern; Passlack (Hernandez 85), Zimmermann, Hanley (c), Godfrey; Trybull; Cantwell, Srbeny (Pukki 85), McLean, Marshall (Klose 78); Rhodes
Sent off: Hanley
Subs not used: Krul, Pinto, Aarons, Vrancic
Pompey (4-2-3-1): MacGillivray; Walkes, Whatmough, Clarke, Donohue; Naylor, B.Thompson; Lowe, Evans (Green 67), Curtis (Wheeler 90+2); Pitman (c)
Goals: Green 90+4
Booked: Lowe, Curtis
Subs not used: Bass, Burgess, Haunstrup, May, Dennis
Referee: Darren Bond
Attendance: 23,201 (2,493 Pompey fans)