Conor Chaplin struck at the death to secure a first pre-season victory of the summer for Pompey at Gosport.
The teenage forward had already opened the scoring early on and the win seemed assured when triallist Theo Robinson added another after the interval.
But Justin Bennett and Jamie White scored in quick succession to pull the National League South side level.
There was still left for the Blues to snatch a winner, however, and it was the impressive Chaplin who got it.
There were not an abundance of chances in the opening stages as the game took a while to get into gear.
But CHAPLIN broke the deadlock with the first clear-cut opening on 13 minutes, guiding the ball home following good work from Kal Naismith down the right.
Naismith had an opportunity to add his own name to the scoresheet moments later, but Nathan Ashmore got down low to his right to deny the winger.
Robinson was next to go close on 36 minutes, with his goal-bound effort from the edge of the area blocked by a well-placed Sam Pearce.
Jayden Stockley then had the ball taken off his feet as he prepared to shoot following a scramble in the Borough box.
Gosport had not created much, but they did forge a couple of openings late in the first half.
First Tom Dunford’s drive was deflected over the bar and then Ben Harding scuffed a shot narrowly wide.
Gosport Borough 0
Pompey were unchanged at the start of the second half, while the hosts switched both of their centre-backs.
It was still the visitors looking more threatening, however, and Stockley tested Ashmore’s handling with a curling effort from 25 yards.
But the hosts nearly carved out an equaliser on 52 minutes when Bennett turned sharply in the box and flashed a shot inches over the bar.
The Blues were not troubling their opponents too much, although Ashmore made a regulation save to keep out Stockley’s header.
But they doubled their lead on 66 minutes when ROBINSON collected a pass from Brandon Haunstrup and slid the ball home from an acute angle.
Pompey were easing their way towards a first pre-season triumph, only for Gosport to then score twice in the space of just two minutes.
First Bennett latched onto Steve Ramsey’s pass and took advantage of some hesitant defending to poke the ball home.
And White then drew the hosts level when he charged into the box and drilled a shot into the bottom corner.
Stockley flung himself at a Chaplin cross to try to restore Pompey’s lead on 83 minutes, but could not quite make contact.
But CHAPLIN did secure victory just before the end when he collected Stockley’s centre and slotted past Ashmore.
Gosport (4-diamond-2): Ashmore; Cuthebertson (Poate 65), Walker (Woodford 46), Pearce (c) (Brice 46), Williams (Woodward 65); Ryan (Ramsey 60), Harding, Dunford, Robinson (Scott 60); Wort (White 54), Bennett (Stephens 82)
Goals: Bennett 73, White 75
Pompey (4-2-3-1): Poke; Nilsen, Field, Webster, Haunstrup; Close, Hollands (c); Robinson, Chaplin, Naismith (McGurk 65); Stockley
Goals: Chaplin 13, 89, Robinson 66
Subs not used: Jones, Stevens, Atangana
Referee: Dan Cooke