Evans, Naismith and Lowe on target.
Pompey strengthened their grasp on automatic promotion with a 3-1 victory over Yeovil.
Gareth Evans opened the scoring from the spot, as the Blues finally managed to break their penalty curse.
But an unfortunate own-goal drew the Glovers level after the break and threatened to spoil the party.
Top scorer Kal Naismith restored the advantage, however, and Jamal Lowe came off the bench to net his first goal for the club.
Paul Cook made two changes from the side that won at Hartlepool the previous week.
Lowe dropped to the bench to make way for a fit-again Carl Baker, while Gary Roberts replaced the injured Noel Hunt.
That meant Naismith was employed as a makeshift striker – a role he performed so effectively in the second half at Victoria Park.
Pompey made a bright start in the south coast sunshine and Bevis Mugabi had to be alert to beat Roberts to Evans’ cross.
But it was Yeovil who should have forged ahead moments later, with an unmarked Tom Eaves sending Kevin Dawson’s cross over the bar.
It was generally the hosts looking more threatening, however, and a dangerous low Roberts centre had to be cleared behind by Alex Lacey.
Then, from the resulting corner, Matt Clarke rose highest to meet the ball, only for Liam Shephard to clear from the line.
But the Blues did break the deadlock on 15 minutes – and finally bring an end to their penalty hoodoo.
Roberts did well to nip in and steal the ball from the dithering Mugabi, drawing a clumsy challenge from the centre-back.
Referee Graham Horwood immediately pointed to the spot and EVANS coolly steered his shot into the bottom corner.
They kept pushing forward and Artur Krysiak denied Naismith before Roberts’ cross hit the hapless Mugabi and ricocheted against the inside of the post.
Pompey continued to send some dangerous deliveries into the box that caused the Glovers defence problems.
There was also one at the other end, however, and Mugabi should at least have turned Matty Dolan’s corner on target from just a few yards out.
Yeovil Town 0
Pompey were keen to build on their narrow advantage once the second half got underway, with Naismith and Kyle Bennett proving a nuisance down the left.
Roberts then found Naismith in the box and a glancing header from the hosts’ top scorer was superbly palmed clear by Krysiak.
Instead, the Glovers drew level in bizarre circumstances on 51 minutes. Francois Zoko, Clarke and Stevens were all competing for the ball and the latter two conspired to inadvertently lob it into their own net.
That was the signal for the Blues to camp out in their opponents’ half, with the visitors having to deal with several corners.
Evans and Bennett also had shots blocked, as Pompey – backed by a noisy crowd – tried to restore their advantage.
Stevens then tried to catch Krysiak out at his near post with a low drive, as the Glovers struggled to contain their hosts.
Cook made his first change on 67 minutes, as Conor Chaplin replaced Roberts to give the strikeforce a more conventional look.
But it was the stand-in who fired the Blues back into the lead just seconds later, as Stevens teed-up NAISMITH and Krysiak could not get a strong enough hand to the Scotsman’s shot.
Yeovil boss Darren Way quickly brought on Otis Khan and his effort flew narrowly over the bar, rippling the top of the net in the process.
Baker then made way for Lowe and within two minutes of his introduction, the January signing opened his account for the club.
It was a decent individual effort from LOWE, who bundled his way past a challenge before finding the net from the tightest of angles.
That ensured Pompey could see the game out relatively free of nerves and take another stride towards a place in League One.
Pompey (4-2-3-1): Forde; Evans, Burgess, Clarke, Stevens; Rose, M.Doyle (c); Baker (Lowe 76), Roberts (Chaplin 67), Bennett; Naismith (Kabamba 90)
Goals: Evans 15 (pen), Naismith 68, Lowe 78
Subs not used: O’Brien, Talbot, Whatmough, Linganzi
Yeovil (3-4-3): Krysiak; Mugabi (Smith 57), Lacey, Butcher; Shephard, Dawson (c), Dolan, James; Goodship (Khan 70), Zoko, Eaves (Akpa Akpro 50)
Goals: Stevens 51 (og)
Subs not used: Maddison, Sowunmi, Lawless, Jones
Referee: Graham Horwood
Attendance: 17,564 (443 away fans)