The suit alleges that the Stones owe the money through Drake Ventures, an LLC set up by the Stones. The DOJ says they did not pay taxes through this LLC from 2007 to 2011 and in 2018.
"They used Drake Ventures to receive payments that are payable to Roger Stone personally, pay their personal expenses, shield their assets, and avoid reporting taxable income to the IRS," the suit says.
Stone was then issued a full presidential pardon on Dec. 23, nullifying the conviction, just weeks before Trump left office.
The Justice Department said that after Stone was charged criminally they used Drake Ventures to purchase their condo.
The DOJ said that the Stones lived a "lavish lifestyle" despite having unpaid taxes.
"Although they used funds held in Drake Ventures accounts to pay some of their taxes, the Stones' use of Drake Ventures to hold their funds allowed them to shield their personal income from enforced collection and fund a lavish lifestyle despite owing nearly $2 million in unpaid taxes, interest and penalties."
Stone met with Trump at his club in West Palm Beach in late December to personally thank the president for commuting and pardoning him.
"My wife and I both had the opportunity to thank the president personally for righting the injustice of my conviction in a Soviet-style show trial, which featured the epic bias of the judge who withheld exculpatory evidence from my defense, misconduct by the jury forewoman and substantial misconduct by the prosecutors," Stone told ABC News at the time.
Stone made headlines again in the days following the Jan. 6 insurrection at the U.S. Capitol after he was seen in a video first unearthed by ABC News flanked by a man with ties to the Oath Keepers militia group outside a Washington, D.C., hotel before the riot. That man, Roberto Minuta, was later arrested for his part in the insurrection.
ABC News' Ali Dukakis, Alex Mallin, Aaron Katersky and Olivia Rubin contributed to this report.