Nick Gibb said the time had come for the Prime Minister to go, and suggested he had not been truthful in his explanations of parties reportedly held in No 10 and across Whitehall during Covid measures.
Mr Gibb, a former schools minister who served under three prime ministers, is among nine Tory MPs who have revealed that they have submitted letters of no confidence in Mr Johnson. Privately, the number is expected to be higher.
Writing in The Telegraph, the MP for Bognor Regis and Littlehampton said that the Conservative Party must face the “hard truths”, and added: “To restore trust, we need to change the Prime Minister.”
Meanwhile the Mirror reported that the Metropolitan Police had been handed a photograph of Mr Johnson holding a beer at an alleged gathering in June 2020 to mark the Prime Minister’s birthday.
The newspaper said it was one of the 300 photos handed to the Met in their investigation into 12 alleged gatherings that may have broken Covid restrictions.
No 10 said it could not comment while the Met Police’s investigation was ongoing.
On Friday, the beleaguered Prime Minister sought to rally Downing Street staff with a line from The Lion King, telling them “Change is good” after the departure of a number of aides.
He was hit with a fifth resignation in less than 24 hours when Elena Narozanski, a special adviser in the No 10 policy unit, walked out on Friday.
Health Secretary Sajid Javid made clear he disagreed with Mr Johnson’s attack on Sir Keir Starmer when he accused the Labour leader of failing to prosecute Jimmy Savile when he was director of public prosecutions.
Backbencher Aaron Bell also declared publicly he had submitted a letter calling for a vote of no confidence in his leader.
But on Saturday Culture Secretary Nadine Dorries defended the prime minister, suggesting those who were against him were Remainers.
She told Times Radio: "There are a small number of voices, whether they are people who were ardent supporters of Remain, who see this as their last opportunity to reverse Brexit."
Asked whether the moves against Boris Johnson were a "Remainer plot", Ms Dorries said: "There are a number of reasons actually, it's not just one, but that certainly is at play with a group."
She said it was "important that people don't get hung up" on a "small number" of MPs.
Ms Dorries suggested those in marginal seats were "working damn hard and they want Boris Johnson in place".
Mr Bell said on Friday in a statement on calling for the Prime Minister to go, he said: “The breach of trust that events in No 10 Downing Street represent, and the manner in which they have been handled, makes his position untenable.”
In an emotional question in the Commons on Monday following the publication of the Sue Gray report into lockdown parties, Mr Bell asked Mr Johnson if he thought he was a “fool” for following Covid restrictions at his grandmother’s funeral.
In his intervention, Mr Gibb said Mr Bell had been “brave” and “struck a chord”.
He wrote: “He expressed the anger and frustration of millions at rule-breaking by those most responsible for making the rules.
“He was not a fool for sticking by those rules. But, whether inadvertently or not, behaviour by people at the heart of government suggests that they think he was - or worse still, that the rules don’t apply to them.”
He said his constituents were “furious about the double standards”.
Mr Gibb said: “The Prime Minister accepted the resignation of Allegra Stratton for joking about a Christmas party that she hadn’t attended, but he won’t take responsibility for those that he did attend.
“I am sorry to say that it is hard to see how it can be the case that the Prime Minister told the truth.”
Other Tory MPs are expected to consider over the weekend whether to write to the chairman of the 1922 Committee Sir Graham in an attempt to force a leadership contest.
In an apparent move to head off further dissent, Mr Johnson has written to Tory MPs promising them a greater role in policymaking with a “direct line” into No 10.
The latest turmoil was triggered by the shock resignation on Thursday of Mr Johnson’s long-standing policy chief Munira Mirza over his refusal to withdraw his “scurrilous” accusation about Sir Keir.
Speaking on Friday, Mr Javid indicated that he agreed with Chancellor Rishi Sunak, who said he would not have made those remarks.
“Keir Starmer, when he was running the DPP, did a good job and he should be respected for it. It is a tough job and he deserved absolute respect for that,” he told reporters.
Mr Javid acknowledged Mr Johnson had subsequently “clarified” his comments and insisted the Prime Minister still had his full support.
However, his intervention will only fuel speculation over how much support the Prime Minister really enjoys from members of his top team.
Ms Mirza’s resignation was followed by the announcement that three more of the most senior figures in No 10 - chief of staff Dan Rosenfield, principal private secretary Martin Reynolds and director of communications Jack Doyle - were also leaving.
The Prime Minister’s official spokesperson said the departures had all been agreed before Ms Mirza tendered her resignation and that they were leaving by mutual consent.
All three were to some extent implicated in the controversy over parties during lockdown.
Mr Reynolds organised the notorious “bring your own booze” event in the Downing Street garden while Mr Doyle was reportedly at at least one of the gatherings now under investigation by the police.
While Mr Rosenfield did not join until after most of the events had taken place, he has been criticised over the Government’s initial response to the report that there had been no breach of the rules.