Prime Minister Boris Johnson on Wednesday told a febrile UK parliament that he would leave office with his "head held high" despite the downfall of his scandal-hit leadership.

Labour leader Keir Starmer took aim at his outgoing foe during the weekly Prime Minister's Questions session, calling him "totally deluded to the bitter end", raising cheers from the opposition benches.

In a sign of things to come, Starmer also set his sights on those Tory MPs battling to become prime minister, including on the complicated tax affairs of wealthy frontrunner Rishi Sunak.

The session began with a furious Speaker throwing out Kenny MacAskill and Neale Hanvey, both from the Scottish pro-independence party Alba, after they launched a protest.

It was expected to be Johnson's penultimate appearance at the session of questions before parliament breaks for the summer, and the Tory party elects its new leader.

But Johnson suggested that leader may be elected "by acclamation" before next week, and that it could be his final appearance at the weekly ritual, if the remaining two candidates agree a deal between them. 

The contenders have already ruled that out, and Johnson's press secretary told reporters that he was merely noting "uncertainty" about the race.

Johnson's premiership began to crumble when he was found to have broken coronavirus regulations amid a series of parties at his Downing Street office.

Starmer said sarcastically his rival could "forget about following the rules".

"I am going to miss this weekly nonsense from him," the Labour leader joked, before turning to attack Sunak and current finance minister Nadhim Zahawi, a rival leadership contender, who is reportedly under investigation over his own tax affairs.

Johnson responded by saying that any of the eight candidates in the running to replace him would "wipe the floor with Captain Crasharooney Snooze Fest", a jibe at Starmer's lawyerly style of rhetoric.

The prime minister then touted his government's achievements on Brexit, vaccines and Ukraine, signing off by saying "I am also proud of the leadership that I have given and... I will be leaving with my head held high".

The first round of voting to whittle down the shortlist of candidates begins on Wednesday, with results expected around 1600 GMT.