"I didn't like the sight or the feeling of families being separated," Trump said.
The president's surprise decision comes after images of children being separated from their parents erupted a national political firestorm. The Trump administration consistently said they had no choice but to separate families because of the law.
"You can't do it through an executive order," Trump said on Friday.
But the Trump administration's "zero tolerance" policy, enacted by Attorney General Jeff Sessions 75 days ago, led to family separations as it states that everyone who tries to cross the border illegally would be criminally prosecuted. As parents awaited trial, they were separated from their children.
Trump said the "zero tolerance" policy would continue but said his executive order is "about keeping families together while ensuring we have a powerful, very strong border."
"I think the word 'compassion' comes into it," he said. "My wife feels strongly about it. I feel strongly about it. Anybody with a heart would feel this way," he added.
"We have to maintain toughness, or our country will be overrun by people, by crime, by all of the things that we don't stand for, that we don't want," Trump said earlier Wednesday when he announced he would be signing the order.
The president's move comes amid growing outrage over the practice of separating families and on the eve of House votes Thursday to address immigration.
"I’ll be doing something that’s somewhat preemptive and ultimately will be matched by legislation I’m sure,” Trump said.
In the Cabinet Room, surrounded by Republican members of Congress, including Sen. Lindsay Graham, R-S.C., Sen. Tom Cotton, R-Ark., Sen. John Cornyn, R-Texas, Sen. Lamar Alexander, R-Tenn., and administration officials, including Treasury Secretary Steve Mnuchin, Trump continued to falsely place the blame squarely on the shoulders of Democrats in Congress.
"We're having a lot of problems with Democrats. They don't care about lack of security. They would like to have open borders where anybody in the world can just flow in, including from the Middle East, from anybody, anywhere, they can just flow into our country," Trump said.
"I think it's very important that we protect our border. We cannot allow a child to be a get out of free card and get into the U.S.," said Sen. Cotton.
The president said he's facing a dilemma between being weak and strong on the issue of immigration, and that it's hard both to have heart and to be strong.
"The dilemma is that if you're weak, if you're weak, which some people would like you to be, if you're really, really pathetically weak, the country is going to be overrun with millions of people. And if you're strong, then you don't have any heart. That's a tough dilemma," Trump said.
"Perhaps I would rather be strong, but that's a tough dilemma."
The president said images of children in detention “affect everyone” but said images from the Obama administration were worse.
“You have double standards,” Trump said. “This has been going on for 50 years.”
"We want to be strong at the border but we also want to be compassionate," Trump said.
First lady Melania Trump has been pushing behind the scenes for days to get her husband to end the child separation policy, according to a White House official.
The official said Mrs. Trump has been speaking with her husband often about this in recent days and has been encouraging him to do all he can to end the separations however he could.
"Ivanka feels very strongly, my wife feels very strongly, I feel very strongly," Trump said. "Anybody with a heart would feel strongly about it."
Last night, Trump told lawmakers his daughter Ivanka had shown him images of children in detention facilities and urged him to end the separation practice. Trump's daughter stood in the back of the Cabinet Room as the president made his announcement.
The directive prepared for the president instructs the Justice Department to allow children to be detained along with their family members while they await a hearing — even if that process takes more than 20 days.
Right now, under what's called the Flores consent decree, children can be detained for only 20 days. The president will instruct DOJ to challenge that decree and to not abide by it while it is being challenged.
Trump admitted during the signing of the executive order in the Oval Office that the move will almost certainly be challenged in court.
Trump announced that the White House congressional picnic -- a favorite summer social event for lawmakers on the Hill -- will be cancelled as the administration figures out its next steps.
"We have a congressional picnic tomorrow, and I was just walking over to the Oval Office and I said, you know, it doesn't feel right to have a picnic for Congress when we're working on doing something very important."
ABC News' Benjamin Siegel contributed to this report.