(Jacquelyn Martin/AP Photo)

Trying to change the subject after a week of political scandals, President Obama today said the nation's "focus cannot drift" from rebuilding the economy and that lawmakers should focus their attention on putting Americans back to work.

"I know it can seem frustrating sometimes when it seems like Washington's priorities aren't the same as your priorities," Obama told local residents and workers at Ellicott Dredges factory in Baltimore. "I know it often seems like folks down there are more concerned with their jobs than with yours.

"Others may get distracted by chasing every fleeting issue that passes by. But the middle class will always be my No. 1 folks, period," he said.

The president's day trip to Baltimore, the second stop on his "Middle Class Jobs and Opportunity Tour," came as the White House attempted to refocus the public's attention away from a trio of controversies and back to Obama's second-term agenda.

The president did not mention Benghazi, the IRS' targeting of conservative groups, or the Justice Department's probe into the Associated Press. Instead, he touted his plan to create jobs and boost the economy.

As a nation, "we've got to up our game when it comes to infrastructure," to help put people back to work, Obama said.

"The problem is, you know, we've had some trouble out of Congress," he added.

"I know: It's surprising, isn't it?" he said. "We've had a little difficulty getting our Republican friends to work with us, to find a steady funding source for these projects that everybody knows needs to happen."

In an attempt to "cut through red tape that keeps big construction projects from getting off the ground," Obama today signed a presidential memorandum intended to streamline the permitting process for infrastructure projects.

"I'm going to keep trying to work with both parties in Washington to make progress because our challenges are solvable," he said.

Before his remarks, the president visited Moravia Park Elementary School to highlight the type of early childhood education that he would like to make universal.

The president chatted with the 4- and 5-year-old students about school and what they were learning, and he even joined in one of the lessons.

"We were having to draw zoo animals and I've got to say, my tiger was not very good," he told the crowd at the factory.

"The kids were not impressed," he said to laughter. "They kind of looked at it, they said, 'That doesn't look like a tiger.'"