That came after Christian Burgess had been sent off for collecting a couple of bookings in the space of a few seconds.
Francois Zoko had earlier pounced to give the struggling Glovers hope of claiming all three points.
But the Blues were determined not to leave empty-handed and Roberts’ intervention ensured that would not be the case.
Paul Cook unsurprisingly stuck with the same side that eased to victory against Bristol Rovers at Fratton Park the previous week.
Kal Naismith was added to the list of substitutes, however, as Conor Chaplin nursed an ankle injury.
The game started brightly and both sides looked dangerous on the break, without either keeper being tested.
But it soon became an extremely disjointed affair, with Pompey struggling to find any fluidity going forward.
Kyle Bennett had a couple of half-chances, first unable to turn Michael Smith’s cross on target and then fizzing a low shot inches past the post.
And the winger went close again on 28 minutes, firing narrowly over after Artur Krysiak had punched clear a corner from Ben Davies.
Both sides were starting to grow frustrated by the lack of creativity and Smith and Connor Roberts received a talking to from referee Mark Haywood following a spot of handbags.
The closest either side came to breaking the deadlock arrived on 37 minutes – and it would have been a comical goal.
Burgess inadvertently diverted Liam Walsh’s poor effort towards its intended target, but the defender recovered well to hack the ball off the line.
And Cook was then forced into a change when Gareth Evans suffered a knock and was replaced by Ben Tollitt.
Yeovil Town 0
Pompey looked sharper once the second half got underway and came close to forging ahead on 51 minutes.
It was a neat move, with Smith picking out Tollitt with a fine cross-field ball and the substitute cutting along the edge of the box before sliding a pass through to Roberts.
The latter hit a low shot that was heading for the bottom corner before Krysiak stuck out a hand to push the ball clear.
Ben Close then came close to opening his Blues account with a low 30-yard shot that was inches off target.
However, the visitors almost fell behind after losing possession in the centre on 59 minutes.
The ball made its way to Zoko in the box, but Ryan Fulton was quickly off his line to make a vital save.
With the hosts starting to take control, Cook made his second switch of the afternoon, with Bennett making way for Marc McNulty.
But Yeovil broke the deadlock on 75 minutes when Walsh’s corner led to a scramble in the Blues box and Zoko eventually forced the ball home.
Pompey tried to find a route back into the game and Burgess saw his header deflected wide after meeting a free-kick from Davies.
But their afternoon got worse when Burgess was booked twice in the space of just a few seconds to leave the visitors with 10 men.
He was first punished for an off-the-ball tussle and then received his marching orders for bringing down Ryan Dickson on the edge of the box.
That looked to have ended the Blues’ hopes of getting anything from the game, but they found themselves level on 85 minutes.
It was a fine finish from inside the box from ROBERTS, who left Krysiak with no chance to send the large travelling support wild with delight.
McNulty then had a shot deflected wide in stoppage-time before a frantic second half was brought to a close.
Yeovil (4-1-4-1): Krysiak; Roberts, Ward (c), Sokolik, Dickson; Tozer; Cornick (Gillett 86), Walsh, Dolan, Campbell (Goodship 69); Zoko (Bird 90+2)
Goals: Zoko 75
Subs not used: Weale, Lacey, Laird
Pompey (4-2-3-1): Fulton; Davies, Burgess, Webster, Stevens; Close, Doyle (c); Evans (Tollitt 41), Roberts, Bennett (McNulty 73); Smith (McGurk 81)
Goals: Roberts 85
Sent off: Burgess (two yellow cards)
Subs not used: Bass, Clarke, Barton, Naismith
Referee: Mark Haywood
Attendance: 6,051 (2,088 Pompey fans)