They took an early lead through Connor Essam’s own-goal, but the tide turned when Adam Webster was sent off and Jay Simpson scored the resulting penalty.
Ollie Palmer then added two more for the hosts, who had skipper Mathieu Baudry dismissed in between.
Gareth Evans reduced the deficit to spark hopes of a comeback, but the Blues were unable to find a late leveller.
Paul Cook stuck with the same side that won at Northampton before Christmas.
Michael Doyle was fit enough to feature after injuring his ankle, while Gary Roberts returned from more than two months on the sidelines to take his place among the substitutes.
The hosts selected former Fratton loanees Alex Cisak and Cole Kpekawa in their line-up, with John Marquis on the bench.
Pompey made a storming start and Caolan Lavery’s persistence almost paid off in the opening seconds as he tried to chase down Cisak’s clearance.
But the Blues did break the deadlock on four minutes when Evans failed to get a decent connection on Kyle Bennett’s cross and the ball ricocheted off Essam and into the net.
They almost doubled their advantage soon after, but Cisak made a fine double save to first keep out Bennett’s fierce drive and then Lavery’s diving header.
Lavery was unable to make contact with the resulting corner and Webster headed wide, as the visitors continued to dominate.
But all their good work was undone midway through the first half when Webster was sent off.
The defender failed to control Danny Hollands’ heavy back-pass and then bundled over Jobi McAnuff inside the box.
Referee Graham Horwood immediately pointed to the spot and brandished a red card, while League Two top scorer Simpson stepped up to convert the penalty.
With no recognised defender on the bench, Adam Barton was brought on to partner Matt Clarke at the back, with Marc McNulty the man to make way.
The dismissal also impacted on Pompey’s ability to play their normal pressing, passing game and they were under the cosh for the rest of the half.
Brian Murphy prevented them from falling immediately behind when he dived to his left to palm away McAnuff’s strike.
Palmer then spun neatly and cut inside from the right before hitting a tame effort straight into the keeper’s arms.
And the Blues also survived late in the half when a Kpekawa free-kick led to the ball pinging around their area.
But they did fall behind deep into stoppage-time, with Palmer finding it far too easy to convert a hopeful cross from Kpekawa.
Leyton Orient 2
Pompey appeared destined for defeat at the break, but there was a spark of hope when Leyton Orient were also reduced to 10 men early in the second half.
Baudry did not react well to a challenge on him by Doyle, lashing out at the Blues skipper and then receiving his marching orders.
The visiting fans were not celebrating for too long, however, as from Sean Clohessy’s free-kick, Palmer arrived at the far post to head into the bottom corner.
Pompey were not too far away from reducing the deficit on 55 minutes, with Barton’s cross agonisingly out of reach for Hollands.
Evans then met Ben Davies’ corner and the ball was heading for the top corner until Cisak got across to pluck it from the air.
The hosts had an opportunity on 66 minutes when Simpson cut inside from the right, but his weak effort was comfortably claimed by Murphy.
And the Blues then went straight up the other end of the pitch to reduce their deficit.
Bennett was the architect, turning Jack Payne and then playing a magnificent pass through to EVANS, who calmly slotted the ball past Cisak.
It almost got even better moments later, as the visitors were denied three times in the space of a few seconds.
First Jean-Yves M’Voto did well to prevent Lavery from getting a shot away and then Enda Stevens flashed a dangerous pass across goal that just evaded Bennett.
Davies picked up the loose ball on the right and hit a powerful angled drive that Cisak managed to push clear.
Cisak was again involved on 72 minutes, doing enough to deny Lavery from close-range before Evans volleyed narrowly over.
The keeper then just about beat Clarke to Davies’ corner as Pompey kept knocking on the door in search of another goal.
Roberts and Conor Chaplin were thrown on for the closing stages as Cook made one last roll of the dice.
But it was the hosts who almost scored late on when Marquis latched onto a poor back-pass from Stevens.
The Orient substitute could only find the post with his first effort, however, while Stevens recovered to clear his follow-up attempt from the line.
Pompey’s left-back then sent in a low cross that was met by Evans, but Cisak made a fine save as the Londoners saw out seven minutes of stoppage-time to claim all three points.
Leyton Orient (4-4-2): Cisak; Baudry (c), M’Voto, Essam, Kpekawa; Clohessy, Pritchard, Payne, McAnuff (Turgott 88); Palmer (Marquis 82), Simpson (Shaw 72)
Goals: Simpson 23 (pen), Palmer 45+1, 52
Sent off: Baudry
Booked: Essam, Palmer, Pritchard, Shaw
Subs not used: Grainger, Dunne, Kashket, Moncur
Pompey (4-2-3-1): Murphy; Davies, Webster, Clarke, Stevens; Hollands (Roberts 87), Doyle (c); Evans, McNulty (Barton 24), Bennett (Chaplin 87); Lavery
Goals: Essam 4 (og), Evans 66
Sent off: Webster
Subs not used: Bass, Close, Tollitt, McGurk
Referee: Graham Horwood
Attendance: 5,848 (1,316 Pompey fans)