Schiano also will serve as the team's associate head coach. Schiano replaces Chris Ash, who was introduced as head coach at Rutgers earlier this week.
"Greg Schiano is an excellent coach," Ohio State coach Urban Meyer said. "He is someone I have known for quite some time now and someone who is going to align with our staff extremely well. I think he will be outstanding as a coach and mentor in our program, and I am pleased to have him on our staff."
Schiano was on South Carolina's interview list for its head-coaching position, which eventually went to Will Muschamp. Schiano was also linked to the head-coaching opening at Miami, sources told ESPN. The Hurricanes hired former Georgia coach Mark Richt.
Ash was the defensive coordinator at Ohio State the past two seasons, building the defense into a top-20 unit nationally.
Ohio State won the national championship last season and went 11-1 this season, boasting its best defense since 2010. The Buckeyes are tied for fourth in the nation in yards allowed per play (4.39) this season.
Schiano coached at Rutgers from 2001 to 2011, compiling a 68-67 record. He parlayed five bowl wins in six seasons into the Buccaneers job, where he went 11-21 before being fired after the 2013 season.
The Buccaneers allowed the second-fewest rushing yards per game (96.3) under Schiano during the 2012 and 2013 seasons, but Tampa Bay allowed the most passing yards per game (267.6) during his two seasons with the team.
ESPN Stats & Information and The Associated Press contributed to this report.