The investigation, which will be handled by the Department of Justice's Civil Rights Division, will explore whether the Baltimore Police Department has engaged in a "pattern or practice" of discriminatory policing. A decision could be announced as early as Friday.
Gray died a week after he was taken into police custody. Six officers have since been charged in his death.
The Justice Department is already looking into whether those officers violated Gray's civil rights.
If, in the broader investigation, the Department of Justice finds that there was a "pattern or practice" of discriminatory policing, it will ask that the police department make sweeping changes. If the police department declines to do so, the matter could land in front of a federal judge, who could force changes within the department.