They forged ahead through Sean Raggett’s header, only for some sloppy defending to gift Vadaine Oliver a leveller before the interval.
Both sides had chances to secure victory, but failure to take them was – taking into account the sides’ respective positions in the table – more costly for the hosts.
Kenny Jackett made one enforced change from the side that won at in-form Oxford earlier in the week.
Callum Johnson missed out with a calf injury, so Haji Mnoga came in for just his second league start for the club.
It meant that Ronan Curtis continued to be used as a striker, partnering Ellis Harrison in a 4-4-2 system.
There was a familiar face in the opposition line-up, with former Fratton loanee Stuart O’Keefe starting for the Gills, while John Akinde was named among the substitutes.
It did not take long for the game to explode into life, with the hosts going close to breaking the deadlock after breaking quickly up the pitch from a corner.
Andy Cannon and Harvey White combined to find Curtis and he slipped a pass across to Marcus Harness, whose effort came back off the post.
Harness then tried to convert a dangerous Curtis delivery from wide on the left, but could only hook the ball over.
It was not all one-way traffic, though, and Craig MacGillivray did well to adjust his body and save when Connor Ogilvie’s header deflected off Curtis.
There were appeals for a penalty at the other end after Raggett and Rasmus Nicolaisen both tried to attack a corner.
Neither could make a clean connection, though, with the ball bouncing up and striking the arm of Tom O’Connor, but referee Tim Robinson decided any contact was accidental.
Pompey were looking the more likely side to open the scoring and Jack Bonham got down to save after Harrison met White’s cross.
The duo reversed roles a few minutes later and this time a well-struck effort from White was again kept out by the Gills keeper.
And the Blues did finally make the breakthrough on 32 minutes, as Harness skinned Ogilvie and sent the ball into the box.
RAGGETT timed his run to perfection, racing into the centre to stoop and send a header past the helpless Bonham.
But Gillingham hit back to draw level before the break – and it was a poor goal for the hosts to concede.
Lee Brown tried to chest the ball back to MacGillivray, but his touch was not strong enough, allowing Oliver to nip in and score.
There was still time for both sides to carve open another great opportunity before the whistle below.
Curtis showed a neat touch to bring the ball down inside the box and drill in a low shot that was again kept out by the impressive Bonham.
And MacGillivray then kept out a thunderous Olly Lee effort moments later, with Raggett on hand to do the mopping up.
Pompey came tearing out of the blocks and Jack Tucker made a key interception when Harrison tried to play a cute pass through to Curtis.
Head physio Bobby Bacic was then kept busy, having to enter the pitch to treat Harrison and Nicolaisen in quick succession following collisions.
Pompey had the ball in the back of the net just before the hour mark, when Brown beat Jordan Graham and squared for Harrison to slot home.
But the hosts were furious that Robinson decided the left-back had fouled his man and awarded a free-kick.
That was the last touch of the ball for Harrison, with Jackett deciding to replace him up front with John Marquis.
The Blues were trying to force their way through and an effort from White was pushed behind by Bonham.
Then, from the resulting corner, the keeper beat away Tom Naylor’s angled header, with the loose ball not landing for Raggett at the back stick.
But the hosts had fallen a bit flat as the action progressed and were looking for an attacking spark to get them back on track.
Instead it was their opponents who might have scored on 78 minutes, with Oliver unable to keep his header down from a Graham cross.
A second switch soon followed for Pompey, with Ryan Williams adding some fresh legs in place of Curtis.
And after Mnoga was booked for a clumsy challenge on Ogilvie, Jackett also decided to swap the teenager for Jack Whatmough.
There were half-chances for Tom O’Connor and Kyle Dempsey, as Gillingham tried to snatch a late winner.
The best fell to Akinde deep into stoppage-time, with the striker racing through as Pompey committed men forward, only for MacGillivray to come off his line and make a key save.
Pompey (4-4-2): MacGillivray; Mnoga (Whatmough 85), Raggett, Nicolaisen, Brown; Harness, Naylor (c), Cannon, White; Curtis (Williams 79), Harrison (Marquis 59)
Goals: Raggett 32
Subs not used: Ward, Daniels, Byers, Hiwula
Gillingham (4-4-2): Bonham; Jackson, Cundy, Tucker, Ogilvie; Graham, Dempsey (c), O’Keefe, O’Connor; Lee (Akinde 73), Oliver
Goals: Oliver 40
Subs not used: Bastien, McKenzie, Morton, Johnson, Woods, Willock
Referee: Tim Robinson