But Richardson, the No. 3 pick in the 2012 draft, has struggled the entire season.
He is only averaging 2.9 yards a carry and has yet to rush for more than 60 yards in a game this season.
The Colts are trying to jump start an offense that has been outscored 93-12 in the first half of the past four games.
Brown opened the season as the third-string running back but he's been the team's most consistent back this season. He had 80 yards and two touchdowns against the Titans on Nov. 14.
NFL Network had earlier reported Richardson's demotion.