Party chairman Richard Holden yesterday hit out amid fears a revolt could mean the end of a third PM in just over a year.
Speculation was swirling in Westminster that some Tories have submitted letters of no confidence in Mr Sunak, although it is believed to be way off the 53 needed to trigger a leadership contest.
Asked if the rebels could force a contest before the next election, Mr Holden told reporters: “I think it would be insanity to do that.”
He went on: “It’s actually a challenge for my colleagues to decide whether they’re interested in being in government or whether they would prefer to sit in opposition.”
The PM earlier urged MPs to back his Rwanda plan.
He told a press conference: “I’m going to see this through.”
But he stopped short of insisting a Commons vote on the Safety of Rwanda Bill would be a seen as a confidence issue.
A vote next week on the principle of the Bill is expected to pass.
The final vote, where rebels are set to seek more concessions, is next year.
Mr Sunak is fighting to reassert his authority after his immigration minister quit.
Robert Jenrick claimed the proposed law aimed at starting migrant flights to East Africa was too weak.
Mr Sunak insisted the new Bill “blocks every single reason that has ever been used to stop removals to Kigali”.
And once again he warned meddling Euro judges he will “do whatever it takes” if they move to thwart flights.
Downing Street is awaiting the verdicts of two backbench factions whose views could make or break Mr Sunak.
Both the Brexiteer rump of MPs and the One Nation centrists have hired lawyers to trawl through the Bill.
Mr Jenrick, who The Sun understands decided to quit on Tuesday, has been replaced by two men.
Michael Tomlinson will be minister for illegal migration, while Tom Pursglove takes charge of legal migration.
Chances of earlier poll rising
By Kate Ferguson, Sun on Sunday Political Editor
DO the Tories have a death wish?
Once again they have exploded into civil war. Each day brings a new bloody bout.
This week, Immigration Minister Robert Jenrick stuck the knife in with a blistering attack on his old boss Rishi Sunak’s emergency Rwanda Bill.
It will fail to stop the boats, he raged, adding darkly: “The fortunes of the Conservative Party are at stake.”
More teddy bear than grizzly, Robert does not make a convincing assassin. Privately, Team Rishi is scathing about his departure.
“He’s a disgrace,” one Government insider said.
“He is just treacherous — throwing toys out of the pram to get headlines.”
But could his departure trigger another leadership election?
Could the Tories really axe their third leader in 18 months?
Surely MPs realise voters will not stomach another leader being dispatched and will demand a general election.
It takes 53 letters of no confidence to force a Tory leadership election.
Only one man knows how many are in — 1922 chairman Sir Graham Brady.
And he has them hidden in a safe in his parliamentary office.
Earlier this week, one Tory MP asked colleagues if the time had come to start sending in letters.
They were “shouted down” pretty quickly.
The bigger danger for Rishi is that one crisis leads to another and his power starts to slowly, accidentally, drain away.
One battle-hardened party grandee observed: “A leadership challenge is not yet likely. But it is more likely this week than last.”
Having staked so much on his Rwanda plan, if the PM loses the vote his authority will be battered and an early election looks inevitable.
Holding on until May may not even be possible.
Meanwhile, Labour are smiling like Cheshire Cats.
One Shadow Cabinet minister positively purred as they observed: “They are constantly fighting each other, not us — we just sit back and watch.”