Another good away result seemed to be on the cards when John Marquis slotted the visitors into the lead during the first half.
But they were undone by some sloppy play, with ex-Hawks striker Jonah Ayunga scoring a brace to turn the contest around by the interval.
The Blues tried to hit back once the action restarted, only for a Luke Leahy penalty to increase the Pirates’ advantage.
Substitute Marcus Harness hit the post late on, but unlike against Plymouth earlier this month, there was to be no late comeback, with a Marquis spot-kick saved at the death.
Kenny Jackett – watching the action remotely – made one change from the side that beat Swindon on home soil the previous week.
On-loan Swansea midfielder George Byers was handed his first start for the club in a 4-3-3 system, as Harness dropped to the bench.
A couple of set-pieces caused problems for the hosts in the opening stages, although Sean Raggett was unable to turn Ronan Curtis deliveries on target.
And Luke McCormick went close to breaking the deadlock at the other end, with Craig MacGillivray getting down well to keep out the midfielder’s effort and then dive on the loose ball.
There was another opportunity for the Pirates when Brandon Hanlon found space to meet Leahy’s delivery, only to guide his header into the keeper’s arms.
Pompey almost scored midway through the first half, although it would have been a controversial goal.
Referee Christopher Sarginson got in Rovers’ way when the visitors launched a quick counter-attack and Byers slipped a neat pass through to Curtis.
He crossed the ball into the box from the right, with Marquis heading the ball into the turf before it bounced up, struck the post and was hacked from the line by Alfie Kilgour.
But the Blues were ahead just a few moments later, with another intelligent Byers pass finding Williams, who cut back for MARQUIS to slide home.
Bristol Rovers did not take long to level, however, as Leahy’s header forward bypassed James Bolton and Ayunga managed to see off Raggett before rifling into the net.
It was back and forth and Marquis might have had his second of the night, but could not quite bring Byers’ pass under control, allowing Joe Day to come off his line and halt the threat.
Instead it was the hosts who ensured an interval lead, as Leahy’s deep cross was flicked on and Ayunga was left in space to bring the ball under control, then fire it past MacGillivray.
Bristol Rovers 2
Pompey made a triple change at the break, with Harness, Rasmus Nicolaisen and Lee Brown – the latter appearing against his former club – replacing Curtis, Bolton and Charlie Daniels.
And Danish defender Nicolaisen immediately made an impact with a couple of crunching, yet fair, challenges.
The impressive Byers then tried his luck from the edge of the box, although this time dragged his effort wide.
Pompey were trying to get forward, but were just lacking that important final ball and when Brown did send a dangerous delivery across the goal, there was nobody there to capitalise.
Harness was able to dig out a shot on the hour mark and his long-range attempt fizzed not too far wide of the post.
Another change followed on 65 minutes, with former Bristol Rovers striker Ellis Harrison on to replace Andy Cannon.
But Pompey were undone by some more sloppy play when Raggett lost the ball to Leahy inside the area and fouled the wing-back as he tried to win it back.
The defender was booked and Leahy stepped up to take the penalty, calmly sending MacGillivray the wrong way.
There were appeals at the other end when a Harness shot was blocked by George Williams and the visitors were adamant it struck his hand, although Sarginson did not agree.
Pompey threw plenty of men forward and forced their opponents to defend deep, with seven minutes of stoppage-time providing some hope.
But it was not their night and when Nicolaisen won possession and beat two players before laying the ball off for Harness, his low strike came back off the post.
Marquis had the chance to reduce the deficit right at the death when he was shoved over inside the area by Kilgour.
But a miserable night was capped when his penalty was saved by Day, low down to the keeper’s right.
Bristol Rovers (3-4-1-2): Day; Baldwin, Kilgour, Harries; Williams, McCormick (Upson 81), Grant, Leahy (c); Nicholson; Ayunga (Daly 85), Hanlan
Goals: Ayunga 34, 44, Leahy 70 (pen)
Booked: McCormick, Williams
Subs not used: van Stappershoef, Hare, Westbrooke, Rodman, Oztumer
Pompey (4-3-3): MacGillivray; Johnson, Bolton (Nicolaisen 46), Raggett, Daniels (Brown 46); Byers, Naylor (c), Cannon (Harrison 65); Williams (White 80), Marquis, Curtis (Harness 46)
Goals: Marquis 26
Booked: Johnson, Raggett
Subs not used: Ward, Hiwula
Referee: Christopher Sarginson