The striker missed a penalty in the second half, although put that behind him to head home the winning goal.
There was growing frustration from the home fans during a disjointed opening 45 minutes, but their side were much better after the break to seal the three points.
Kenny Jackett made one change from the side that were beaten in the south coast derby earlier in the week.
It was an enforced switch, with the injured Oli Hawkins replaced in the centre of defence by Sean Raggett.
There was room on the bench for a fit-again Marcus Harness – following a thigh problem – and the previously suspended Ellis Harrison.
Pompey had an early chance when Brandon Haunstrup’s corner was nodded back by John Marquis for Pitman, whose header was claimed under the crossbar by Remi Matthews.
There was then an injury scare when Raggett and Tom Naylor collided, although the Blues skipper was able to continue following some on-field treatment.
Chances for the hosts were at a premium during the first half and there was growing frustration at their play in the final third.
It was Craig MacGillivray who was given more to do, although a weak Will Buckley effort was comfortably dealt with.
Marquis and Ronan Curtis combined well at the other end, with Ryan Williams not quite able to reach the latter’s pass before Matthews gathered.
Curtis then had a decent opportunity when Pitman dummied Marquis’ pass, but took too long to get a shot away, allowing the Wanderers defence to deal with the danger.
And the visitors soon started to pose more of a threat themselves, with MacGillivray doing well to parry clear Adam Chicksen’s drive.
Thibaud Verlinden then lifted in a cross that was just out of reach for Daryl Murphy in the centre of the Blues box.
They should have been ahead just before the break, with an unmarked Will Buckley unable to send Murphy’s delivery on target.
But there was still time for one more chance for the hosts, which came after Jack Hobbs had been booked following a scything challenge on Williams.
Pitman stepped up to take the free-kick from 25 yards out and his goal-bound effort was repelled by a one-handed Matthews save.
Bolton Wanderers 0
There was a long-awaited return for Harness at the start of the second half, with the winger taking Williams’ place on the right.
Pompey had a dangerous attack soon after the restart, when Marquis shrugged off several challenges to bundle his way through the Bolton back-line.
But Matthews came bravely off his line to halt his path to goal and had to receive treatment following a clash between the two.
There was still a lack of fluency on display, however, and it took a Bolton error to provide the hosts with an excellent chance to break the deadlock.
Marquis received a pass from Curtis and was tripped by Hobbs, with referee Darren Drysdale pointing straight to the spot.
Pitman stepped up, but the usually reliable penalty taker saw his fierce effort cannon back off the base of the post.
A second Blues substitution followed just past the hour mark, with Curtis making way for Gareth Evans.
Matthews then flapped at a Haunstrup corner, but got enough on the ball to get it clear, before Harness fizzed a low shot narrowly past the post.
And Pompey finally made that growing pressure count on 67 minutes, with a goal that had been coming.
James Bolton sent a cross into the box from deep on the right and PITMAN hung in the air before guiding home a perfectly-placed header.
With the visitors having to commit men forward to chase an equaliser, it ensured the game became a lot more open.
Some dangerous deliveries from either wing had to be dealt with, while Hobbs’ intervention prevented Pitman from having a clear shot on goal.
And Ben Close almost wrapped things up late on, making space for himself with a neat touch before drilling inches wide.
Thy then looked to keep possession in Bolton’s half as the seconds ticked down, although a display of five additional moments led to some audible groans.
Ali Crawford’s 30-yard free-kick was not too far off target, while Close fired a good chance over at the other end soon after.
But the Blues held on through the closing stages and the final whistle was met by a mixture of relief and jubilation.
Pompey (4-4-2): MacGillivray; Bolton, Burgess, Raggett, Haunstrup; Williams (Harness 46), Naylor (c), Close, Curtis (Evans 62); Pitman, Marquis
Goals: Pitman 66
Booked: Curtis, Close
Subs not used: Bass, Walkes, McCrorie, Cannon, Harrison
Bolton (4-1-4-1): Matthews; Emmanuel (L.Murphy 67), Hobbs, Zouma, Chicksen; Bridcutt (c); Buckley, Lowe, Crawford, Verlinden (Politic 60); D.Murphy
Booked: Hobbs, Lowe
Subs not used: Alexander, Boon, Senior, Graham, Weir
Referee: Darren Drysdale
Attendance: 18,382 (824 away fans)