Brett Pitman went from villain to hero, as Pompey battled to a hard-fought victory over Bolton at Fratton Park.
The striker missed a penalty in the second half, although put that behind him to head home the winning goal.
There was growing frustration from the home fans during a disjointed opening 45 minutes, but their side were much better after the break to seal the three points.
Kenny Jackett made one change from the side that were beaten in the south coast derby earlier in the week.
It was an enforced switch, with the injured Oli Hawkins replaced in the centre of defence by Sean Raggett.
There was room on the bench for a fit-again Marcus Harness – following a thigh problem – and the previously suspended Ellis Harrison.
Pompey had an early chance when Brandon Haunstrup’s corner was nodded back by John Marquis for Pitman, whose header was claimed under the crossbar by Remi Matthews.
There was then an injury scare when Raggett and Tom Naylor collided, although the Blues skipper was able to continue following some on-field treatment.
Chances for the hosts were at a premium during the first half and there was growing frustration at their play in the final third.
It was Craig MacGillivray who was given more to do, although a weak Will Buckley effort was comfortably dealt with.
Marquis and Ronan Curtis combined well at the other end, with Ryan Williams not quite able to reach the latter’s pass before Matthews gathered.
Curtis then had a decent opportunity when Pitman dummied Marquis’ pass, but took too long to get a shot away, allowing the Wanderers defence to deal with the danger.
And the visitors soon started to pose more of a threat themselves, with MacGillivray doing well to parry clear Adam Chicksen’s drive.
Thibaud Verlinden then lifted in a cross that was just out of reach for Daryl Murphy in the centre of the Blues box.
They should have been ahead just before the break, with an unmarked Will Buckley unable to send Murphy’s delivery on target.
But there was still time for one more chance for the hosts, which came after Jack Hobbs had been booked following a scything challenge on Williams.
Pitman stepped up to take the free-kick from 25 yards out and his goal-bound effort was repelled by a one-handed Matthews save.
Bolton Wanderers 0
There was a long-awaited return for Harness at the start of the second half, with the winger taking Williams’ place on the right.
Pompey had a dangerous attack soon after the restart, when Marquis shrugged off several challenges to bundle his way through the Bolton back-line.
But Matthews came bravely off his line to halt his path to goal and had to receive treatment following a clash between the two.
There was still a lack of fluency on display, however, and it took a Bolton error to provide the hosts with an excellent chance to break the deadlock.
Marquis received a pass from Curtis and was tripped by Hobbs, with referee Darren Drysdale pointing straight to the spot.
Pitman stepped up, but the usually reliable penalty taker saw his fierce effort cannon back off the base of the post.
A second Blues substitution followed just past the hour mark, with Curtis making way for Gareth Evans.
Matthews then flapped at a Haunstrup corner, but got enough on the ball to get it clear, before Harness fizzed a low shot narrowly past the post.
And Pompey finally made that growing pressure count on 67 minutes, with a goal that had been coming.
James Bolton sent a cross into the box from deep on the right and PITMAN hung in the air before guiding home a perfectly-placed header.
With the visitors having to commit men forward to chase an equaliser, it ensured the game became a lot more open.
Some dangerous deliveries from either wing had to be dealt with, while Hobbs’ intervention prevented Pitman from having a clear shot on goal.
And Ben Close almost wrapped things up late on, making space for himself with a neat touch before drilling inches wide.
Thy then looked to keep possession in Bolton’s half as the seconds ticked down, although a display of five additional moments led to some audible groans.
Ali Crawford’s 30-yard free-kick was not too far off target, while Close fired a good chance over at the other end soon after.
But the Blues held on through the closing stages and the final whistle was met by a mixture of relief and jubilation.
Pompey (4-4-2): MacGillivray; Bolton, Burgess, Raggett, Haunstrup; Williams (Harness 46), Naylor (c), Close, Curtis (Evans 62); Pitman, Marquis
Goals: Pitman 66
Booked: Curtis, Close
Subs not used: Bass, Walkes, McCrorie, Cannon, Harrison
Bolton (4-1-4-1): Matthews; Emmanuel (L.Murphy 67), Hobbs, Zouma, Chicksen; Bridcutt (c); Buckley, Lowe, Crawford, Verlinden (Politic 60); D.Murphy
Booked: Hobbs, Lowe
Subs not used: Alexander, Boon, Senior, Graham, Weir
Referee: Darren Drysdale
Attendance: 18,382 (824 away fans)
Carabao Cup 3
It was a case of what might have been for Pompey as they exited the Carabao Cup at the hands of rivals Southampton.
They dominated the opening proceedings at a packed Fratton Park, with John Marquis striking the woodwork before a couple of goal-line blocks rescued their opponents.
But the impressive Danny Ings proved the difference between the sides, with two goals before the break.
He then provided an assist for Cedric to ensure a fourth round spot, with Nathan Redmond wrapping things up late on.
Kenny Jackett made two changes from the side that were narrowly beaten at Wycombe in the league on Saturday.
Brett Pitman and Ryan Williams both returned to the starting line-up, with Andy Cannon dropping to the bench and Ellis Harrison serving a one-match ban.
Backed by a vociferous crowd – and with the rain bucketing down – it was the hosts who started stronger against their top-flight opponents.
And they almost broke the deadlock on nine minutes, as John Marquis collected Williams’ pass and bent a shot against the post.
Then, after Tom Naylor had been fouled and the visitors struggled to deal with Brandon Haunstrup’s cross, a fierce Pitman effort was deflected narrowly over.
A corner from Haunstrup led to an excellent chance to break the deadlock midway through the first half.
But Pitman’s glancing header was somehow stopped from crossing the line and Christian Burgess was unable to force the ball home from close range.
It had been one-way traffic for much of the half, but Southampton took the lead against the run of play on 21 minutes.
Craig MacGillivray had done well to keep out an initial effort from Ings, but when the striker received the ball back, he curled a fine shot into the bottom corner.
They almost doubled the advantage moments later, as Ings raced through, only to be denied by MacGillivray.
But the danger was not over and Pierre-Emile Hokbjerg’s sent the loose ball towards the net, with a diving Oli Hawkins getting in the way.
Pompey kept battling, though, and a series of good balls into the box caused Southampton problems before Burgess prodded past the post.
The visitors were causing problems on the break and MacGillivray did well to get behind a skiddy shot from James Ward-Prowse.
Ronan Curtis had an effort deflected narrowly off target, while Pitman’s corner was intercepted before it could reach Naylor at the back stick.
But Southampton grabbed a second goal just before the break, as Ings met Michael Obafemi’s low centre and lifted the ball over MacGillivray.
Several former players were welcomed on to the pitch at the break, with warm applause given to Benjani, Adam Webster, Greg Halford and derby heroes Joel Ward and David Norris.
But it was up to the current crop of Blues to get back into the tie and Curtis had a shot blocked, before Williams skied another over the bar.
For all the passion displayed on the pitch and in the stands, it had not been a dirty contest, with 53 minutes on the clock before Hawkins collected the first yellow card for a foul on Ward-Prowse.
But he was quickly joined in referee Kevin Friend’s book by Pitman, who was penalised for a trip on Obafemi.
Pompey were seeing a lot of the ball without really testing Alex McCarthy and a dangerous corner from Haustrup somehow flashed through a crowded box and out the other side.
The hosts were dealt a blow just before the hour mark, as Hawkins – who had again impressed at the back – limped off to be replaced by Paul Downing.
They continued to push forward and find gaps in the visitors’ defence, but things were not quite landing right for the blue-shirted players inside the box.
And Southampton wrapped things up on 77 minutes, as the ball was cut back by Ings for Cedric to slot home.
Jackett made a couple of substitutions to try to lift the Blues again, with Gareth Evans and Ross McCrorie both brought on.
The hosts kept pushing for the goal that their spirited play deserved, but it was Southampton who instead added a late fourth.
They broke quickly up the pitch and substitute Redmond curled a fine finish past the grasp of MacGillivray.
It did not stop the Fratton faithful from raising the volume levels, though, as they continued to get behind their team for the closing minutes and after the final whistle.
Pompey (4-2-3-1): MacGillivray; Bolton, Hawkins (Downing 59), Burgess, Haunstrup; Naylor (c), Close (McCrorie 82); Williams (Evans 78), Pitman, Curtis; Marquis
Booked: Hawkins, Pitman
Subs not used: Bass, Walkes, Cannon, Maloney
Southampton (4-3-3): McCarthy; Cedric, Bednarek, Yoshida, Bertrand; Ward-Prowse, Romeu, Hojbjerg; Obafemi (Redmond 72), Adams, Ings (Long 83)
Goals: Ings 21, 44, Cedric 77, Redmond 86
Subs not used: Gunn, Stephens, Valery, Boufal, Armstrong
Referee: Kevin Friend
Attendance: 18,707 (1,981 away fans)