But Biden went even one step further late Wednesday, saying Sen. Cory Booker should apologize for his own criticism.
The controversy began at a fundraiser in New York City Tuesday night when Biden spoke about the need for consensus to fix the "broken" political system, and recalled his time in the Senate serving alongside former Sens. James O. Eastland, of Mississippi, and Herman Talmadge, of Georgia. Eastland served as chair of the Senate Judiciary Committee when Biden arrived in the Senate and both Eastland and Talmadge were deeply opposed to desegregation.
Biden said of Eastland, "He never called me 'boy,' he always called me 'son.'"
The former vice president then brought up deceased Georgia Sen. Herman Talmage, "[He was] one of the meanest guys I ever knew, you go down the list of all these guys. Well, guess what? At least there was some civility. We got things done. We didn't agree on much of anything. We got things done. We got it finished. But today, you look at the other side and you're the enemy. Not the opposition, the enemy. We don't talk to each other anymore."
Biden reiterated that he did not agree with the views those senators held outside a fundraiser in Chevy Chase, Maryland, Wednesday night.
"I could not have disagreed with Jim Eastland more. He was a segregationist. I ran for the United States Senate because I disagreed with the views of the segregationist -- many of them in the Senate at the time," Biden said.
"The point I'm making is you don't have to agree. You don't have to like the people in terms of their views. But you just have to simply make the case you beat them, you beat them without changing the system," he added.
Biden's initial comments drew criticism from some of his 2020 rivals, including Booker.
"You don't joke about calling black men 'boys.' Men like James O. Eastland used words like that, and the racist policies that accompanied them, to perpetuate white supremacy and strip black Americans of our very humanity," Booker, who participated Wednesday in a hearing on reparations for the descendants of slaves, said in a statement.
"Vice President Biden's relationships with proud segregationists are not the model for how we make America a safer and more inclusive place for black people, and for everyone. I have to tell Vice President Biden, as someone I respect, that he is wrong for using his relationships with Eastland and Talmadge as examples of how to bring our country together. And frankly, I'm disappointed that he hasn't issued an immediate apology for the pain his words are dredging up for many Americans. He should," Booker continued.
But when asked if he would issue an apology, Biden made it clear Wednesday night he did not believe he had a reason to.
"Apologize for what?" Biden asked.
"Cory should apologize. He knows better. There's not a racist bone in my body. I've been involved in civil rights my whole career. Period. Period. Period."
In his comments Wednesday night Biden also pointed to his support for the Voting Rights Act, which prohibits racial discrimination in voting, and the extensions he was able to pass on the act while in the Senate. That position put Biden at odds with segregationist like Sen. Strom Thurmond, of South Carolina, who served as chair of the Senate Judiciary committee from 1981 to 1987.
Biden's comments Tuesday night were not the first time Biden spoke about his relationship with senators who supported segregation.
"When I first got started, it was a very different circumstance. The politics wasn't broken, but the American people were in overwhelming disagreement. On the war in Vietnam, the women's movement, the civil rights movement are bitter, bitter fights. When I got there, there were still five, seven segregationists from the south were part of the Democratic Party. ... But the politics wasn't broken in the sense that we still treat each other with some civility," Biden said in Concord, New Hampshire, earlier this month.
Biden has also recalled Eastland's offer to come and campaign for or against him in Delaware -- whichever would help more in his reelection efforts -- as he did in 2016.
"I was running for reelection in 1978. And I walked into the Senate dining room when we were trying to wind down everything. We had no appointments, just voting around the clock, Biden said during a speech at the Pittsburgh Labor Day parade in 2016.
"And I walked in -- true story -- and I got -- old Eastland looked at me. He never called me senator. He always called me ‘son.' He says, ‘Son, come over here and sit down a minute.' And I went over and sat down," Biden recalled at the time. "He said, ‘What can old Jim Eastland do for you in Delaware?' … I said, ‘Mr. Chairman, some places you'd help and some places you'd hurt.' "[Eastland] said, ‘Well, I'll come to Delaware and campaign for you or against you, whichever will help the most."
While on the campaign trail Biden has often talked about the need for consensus in order to make the political system work, which often requires working together to accomplish goals, despite differences in beliefs.
Booker was not the only 2020 Democrat to criticize Biden for the comments.
Sen. Kamala Harris, D-Calif., said Biden's comments on Eastland and Talmadge were "misinformed and wrong."
"I have a great deal of respect for Vice President Biden. He's done very good work and he's served our country in a very noble way. But to coddle the reputations of segregationists -- of people who, if they had their way, I would literally not be standing here as a member of the U.S. Senate, is, I think, is just misinformed and wrong," Harris told reporters outside the U.S. Capitol Wednesday. "Let's be very clear that the senators that he is speaking of with such adoration are individuals who made and built their reputation on segregation."
"I appreciate the importance of working with people and finding common ground. But to to suggest that individuals who literally made it their life's work to take America back on the issue of race is a real problem for me. And it's a very serious issue," she added.
Fellow candidate New York City Mayor Bill de Blasio slammed Biden for his comments, tweeting that Eastland believed his interracial family should be illegal.
"It's 2019 & @JoeBiden is longing for the good old days of 'civility' typified by James Eastland. Eastland thought my multiracial family should be illegal & that whites were entitled to "the pursuit of dead n******. It's past time for apologies or evolution from @JoeBiden. He repeatedly demonstrates that he is out of step with the values of the modern Democratic Party," de Blasio tweeted Wednesday.
Other presidential hopefuls refrained from hitting Biden directly but made clear that they did not agree with his views.
"I'm not here to criticize other Democrats, but it's never ok to celebrate segregationists. Never," Sen. Elizabeth Warren, D-Mass., said Wednesday avoiding mentioning Biden by name.
Former Maryland Rep. John Delaney was similarly reflective, saying, "Evoking an avowed segregationist is not the best way to make the point that we need to work together and is insensitive; we need to learn from history but we also need to be aggressive in dismantling structural racism that exists today."
Symone Sanders, a senior adviser for the Biden campaign, defended the former vice president's comments, making it clear the vice president was not praising Eastland and Talmadge and calling the insinuation he did "disingenuous."
"@JoeBiden did not praise a segregationist. That is a disingenuous take. He basically said sometimes in Congress, one has to work with terrible or downright racist folks to get things done. And then went on to say when you can't work with them, work around them," Sanders tweeted Wednesday afternoon.
"Joe Biden has been an ally in the fight for civil rights for years. I am all here for VALID CRITICISM, but suggesting that Joe Biden -- the man who literally ran for office against an incumbent at 29 because of the civil rights movement, the man who was at the forefront of marriage equality before it was politically popular, the man who served as President Obama's VP, the man who literally launched his 2020 campaign calling out Nazis in Charlottesville along with Trump's equivalency -- suggesting he is actively praising a segregationist is just a bad take and a willfully disingenuous act," Sanders continued.
House Majority Whip James Clyburn, of South Carolina, a senior member of the Congressional Black Caucus and the highest ranking black lawmaker, defended Biden in comments to Politico saying that the business of governance sometimes requires working with people with distasteful views. Biden is one of 22 candidates expected to attend Clyburn's famous fish fry before the South Carolina Democratic Convention -- a must-stop event on the campaign trail in a state with a high number of black voters.
"I worked with Strom Thurmond all my life," Clyburn said of Thurmond, who was known for his staunch segregationist views. "You don't have to agree with people to work with them."
ABC's Mariam Kahn contributed to this report