Campaigners and MPs had blasted the leftie newspaper for printing the illustration.
Mr Sharp quit as BBC chairman on Friday after he was found to have broken rules by not telling appointees that he had helped secure an £800,000 loan to former PM Boris Johnson.
The cartoon, published online and in The Guardian’s Journal supplement yesterday, showed a heavily caricatured version of Mr Sharp departing the Beeb with a box marked Goldman Sachs, the investment bank where he used to work, containing a squid and a puppet of Rishi Sunak.
Former Home Secretary Sajid Javid said the cartoon “wouldn’t have looked out of place in a Nazi newspaper”.
Former Chancellor of the Exchequer Nadim Zahawi said: “This is pure antisemitism.”
Mike Katz, chairman of the Jewish Labour Movement, said it was “tawdry” with “obvious tropes”.
The Guardian pulled the cartoon from its website yesterday and apologised, saying it “does not meet our editorial standards”.
It added: “The Guardian apologises to Mr Sharp, to the Jewish community and to anyone offended.”
Martin Rowson, who created the illustration, also apologised, saying he had been rushing to meet a deadline and his work had gone “horribly wrong”.
He said: “This is on me, even if accidentally or, more precisely, thoughtlessly.”