In letters to members of Congress on Wednesday, Alito said his wife was responsible for flying an upside-down flag over their Virginia home in 2021 and an “Appeal to Heaven” flag at their New Jersey beach house last year.

Neither incident merits his recusal, he wrote.

“I am therefore duty-bound to reject your recusal request,” he wrote.

The court is considering two major cases related to the Jan. 6, 2021, attack by a mob of Trump supporters on the Capitol, including charges faced by the rioters and whether Trump has immunity from prosecution on election interference charges.

Alito has rejected calls from Democrats in the past to recuse himself on other issues.

The New York Times reported that an inverted American flag was seen at Alito’s home in Alexandria, Va., less than two weeks after the attack on the Capitol. The paper also reported that an Appeal to Heaven flag, featuring a pine tree on a white background, was flown outside the justice’s beach home in New Jersey last summer. Both flags were carried by rioters who violently stormed the Capitol in January 2021 echoing Trump’s false claims of election fraud.

Alito said he was unaware that the upside-down flag was flying above his house until it was called to his attention. “As soon as I saw it, I asked my wife to take it down, but for several days, she refused,” he wrote in nearly identical letters to Democrats in the House and Senate.