Pompey recorded their first League One win on the road in 2019, despite a late comeback by Walsall at the Bescot Stadium.
Brett Pitman opened the scoring from the penalty spot and Omar Bogle made sure it was a commanding lead at the break.
The hosts were much better in the second half, but Viv Solomon-Otabor swept home his first Blues goal.
The Saddlers did not give up, however, and Jon Guthrie converted two corners – the second right at the death – to restore some pride.
Kenny Jackett made three changes from the side that were beaten at Charlton at the weekend.
Skipper Pitman replaced Gareth Evans for his first start since the end of January, while Oli Hawkins made way for a fit-again Bogle up front.
And with Ronan Curtis suffering a freak injury on the morning of the game – getting his finger trapped in a door – Solomon-Otabor came in wide on the left.
Pompey made a bright start to the contest, although Bogle was unable to bend an early shot around Saddlers stopper Liam Roberts.
But they were temporarily down to 10 men when Matt Clarke reopened the wound in his head and started bleeding from the nose.
The medical team had to patch him up and with the defender requiring a change of outfit, he was stood without a shirt in cold conditions for a number of minutes.
It was during that time, however, that the visitors broke the deadlock, with Morgan Ferrier punished for a shove on Nathan Thompson inside the box.
Referee Trevor Kettle immediately whistled and PITMAN stepped up to send Roberts the wrong way from the spot.
Walsall were struggling to deal with the Blues’ attacks and just about forced the ball clear when Solomon-Otabor tried to convert a mis-hit effort from Jamal Lowe.
But the hosts soon had a chance to level when Ferrier raced on to a ball down the right and saw his shot saved by Craig MacGillivray.
And the keeper immediately provided the assist for Pompey’s second goal from a long punt up the pitch.
The ball reached BOGLE, whose shot appeared to be going over, only to dip on to the top of the post and bounce over the line.
Lowe thought he had added a third on 30 minutes after converting a quickly-taken Pitman free-kick, only for Kettle to decide Pompey had acted too soon.
Walsall failed to deal with the set-piece when it was played a second time and Tom Naylor’s strike was deflected behind.
Pitman then lifted a fine pass over the defence late in the half, although Lowe’s touch was slightly too heavy and Roberts was able to gather.
The hosts had been second best throughout the first half, but had a chance to reduce their deficit seconds after the restart.
A long ball forward bounced awkwardly for Clarke and Josh Gordon was not quite able to send his header looping over MacGillivray.
Gordon’s teasing delivery from the left then had to be headed behind by Lee Brown, as Walsall started to provide much more of a threat.
They were pushing Pompey back for long periods, but the defence were holding firm to repel the red-shirted tide before MacGillivray could be tested.
Lowe won a corner to offer some respite, but Brown’s cross was headed wide by Bogle to let the Saddlers off the hook.
But the Blues did have a third goal on 68 minutes – and it meant that all three of the players Jackett brought into the team had scored.
Pitman was falling to the turf as he sent a pass through to Lowe, who unselfishly squared for SOLOMON-OTABOR to slot home and open his Pompey account.
Bogle was soon withdrawn by Jackett and had his name chanted by the travelling fans as he trudged off to be replaced by James Vaughan.
It looked like it was game over, but Walsall gave themselves some hope when Guthrie nodded in a corner from George Dobson.
Pompey appeared to be seeing the game out with relative ease, only for the Saddlers to grab another goal in stoppage-time.
Another corner was scrambled into the net by Guthrie, but there was not enough time for them to mount any more attacks and the Blues were able to celebrate a victory.
Walsall (4-1-4-1): Roberts; Devlin, Scarr, Guthrie, Leahy; Dobson (c); Ismail (Oteh 87), Kinsella, Edwards (Blackett-Taylor 70), Ferrier; Gordon
Goals: Guthrie 75, 90+4
Booked: Kinsella, Gordon, Ferrier
Subs not used: Dunn, Fitzwater, Norman, Laird, Osbourne
Pompey (4-2-3-1): MacGillivray; Thompson, Burgess, Clarke, Brown; Naylor, Close; Lowe, Pitman (c) (Walkes 86), Solomon-Otabor; Bogle (Vaughan 74)
Goals: Pitman 13 (pen), Bogle 25, Solomon-Otabor 68
Subs not used: Bass, Haunstrup, Donohue, Evans, Hawkins
Referee: Trevor Kettle
Attendance: 4,097 (598 Pompey fans)