The visitors were in control for long periods of the contest, but suffered a distinctly off day in front of goal.
And Josh Parker made the most of a rare chance for their opponents to net the first goal of the contest just before half-time.
The Blues had to play the closing stages a man down, with Nathan Thompson limping off after all three changes had been made.
And Gillingham then made sure of all three points in stoppage-time, as Callum Reilly fired home a penalty.
Kenny Jackett stuck with the same side that beat promotion rivals Sunderland on home soil at the weekend.
And there did not appear to be any festive fatigue, as the visitors set about dominating the opening 45 minutes.
But for all their control of possession and neat passing build-up, Gills keeper Tomas Holy was not really tested.
A sliced effort from Ronan Curtis was blocked before it could trouble the stopper, who tipped away a Gareth Evans delivery before the Republic of Ireland international could capitalise.
Moments later, a teasing ball across the face of goal from Ben Thompson somehow managed to evade everyone.
Pompey did go close on eight minutes when a fierce goal-bound effort from Evans took a deflection to divert the ball off target.
Oli Hawkins – who was clearly having his shirt tugged by Alex Lacey – then sent an effort agonisingly wide of the post.
It was pretty much one-way traffic, but the Blues were just lacking that final killer ball to really hurt their hosts.
They won a total of seven corners before the break and a couple of them were headed off target by Hawkins.
And the striker met another Evans set-piece – after Holy had pushed Curtis’ effort wide – only for Barry Fuller to clear from the line.
But the Blues were then caught out by a sucker-punch just before the interval, as a cross from Elliott List was emphatically headed home by Parker.
Pompey immediately set about trying to find a leveller once the action restarted and the hosts eventually hacked clear following a scramble in their box.
A low right-sided delivery from Ben Thompson then somehow managed to evade both Jamal Lowe and Hawkins.
But Gillingham soon threatened at the other end, with Dean Parrett’s deep free-kick missing everyone and having to be pushed behind by Craig MacGillivray.
Holy should at least have been tested on 54 minutes when an unmarked Evans collected a pass from Lee Brown, only to fire over the bar.
Pompey’s skipper hit a cleaner effort from long-range soon after, although the ball was comfortably held by the keeper.
It was generally proving pretty profligate from the visitors and Hawkins could not send his header on target after getting above Gabriel Zakuani to meet a Curtis cross.
And the striker was withdrawn just before the hour mark, with Brett Pitman coming on to add some fresh legs to the forward line.
Jackett made his other two changes not long after, as Lowe and Curtis made way for Andre Green and David Wheeler.
It was Pitman who had the first chance to level, but was unable to make contact with Brown’s low cross at the near post.
And the Blues were dealt a blow when Nathan Thompson pulled up on 77 minutes and had to be helped off the pitch.
With all three substitutions having already been made, it meant the visitors would need to see out the rest of the game with just 10 men.
And their opponents almost took advantage when the ball fell for Billy Bingham inside the box, but he could not beat MacGillivray.
Pompey kept plugging away, but Pitman’s effort – taken on the turn after collecting Green’s pass – lacked enough power to trouble Holy.
And the hosts wrapped things up when Whatmough crudely ended List’s path to goal, with Reilly confidently slamming home the spot-kick.
Gillingham (4-1-4-1): Holy; Fuller, Lacey, Zakuani (c), Ogilvie; Byrne; List, Parrett (Bingham 69), Reilly, Hanlan (O’Neill 76); Parker (Mbo 90+2)
Goals: Parker 45, Reilly 90+1 (pen)
Booked: Byrne, Fuller
Subs not used: Hadler, Garmston, Rees, Charles-Cook
Pompey (4-2-3-1): MacGillivray; N.Thompson, Whatmough, Clarke, Brown; Naylor, B.Thompson; Lowe (Green 69), Evans (c), Curtis (Wheeler 69); Hawkins (Pitman 59)
Subs not used: Bass, Walkes, Burgess, Close
Referee: Nicholas Kinseley
Attendance: 6,940 (1,637 Pompey fans)