Chanting "white lives matter," "you will not replace us," and the Nazi-associated phrase "blood and soil," some of the white nationalists engaged with counterprotesters and scuffles broke out.
Police arrived on campus, declared it an unlawful assembly, and ordered the crowds to disperse.
It is unclear if anyone was injured or arrested.
Teresa A. Sullivan, president of the University of Virginia, said in a statement Friday, "I am deeply saddened and disturbed by the hateful behavior displayed by torch-bearing protestors that marched on our Grounds this evening. I strongly condemn the unprovoked assault on members of our community, including University personnel who were attempting to maintain order."
Sullivan added, "Law enforcement continues to investigate the incident, and it is my hope that any individuals responsible for criminal acts are held accountable. The violence displayed on Grounds is intolerable and is entirely inconsistent with the University’s values."
The Unite the Right rally, slated to take place in McIntire Park, is in part a response to the city's decision to remove a statue of Confederate Gen. Robert E. Lee from a municipal park. A judge on Friday ruled that the rally could continue at the permitted location, even though the city had asked it be moved to a larger venue.
In a statement Friday he said, "Many of the individuals coming to Charlottesville tomorrow are doing so in order to express viewpoints many people, including me, find abhorrent. As long as that expression is peaceful, that is their right. But it is also the right of every American to deny those ideas more attention than they deserve."
ABC News' Amanda Maile and Stephanie Ebbs contributed to this report.