The brazen incident is believed to be the first instance of rhino poaching in Europe.
The Thoiry Zoo, in the suburbs west of Paris, said in a statement that the poachers cut off part of the rhino's second horn too but appeared to have been interrupted or had defective equipment.
Vince, the 4-year-old male white rhino that was killed, mostly likely had his horn taken off with a chainsaw, according to the zoo.
The zoo's two other white rhinos — 37-year-old Gracie and 5-year-old Bruno — survived, the zoo said.
"White rhinos are a spectacular conservation success story, but the current poaching crisis could reverse all of our conservation gains," Matthew Lewis, an African-species expert writes on WWF's website.
The French government published a decree in August 2016 prohibiting the transport or sale of elephant ivory or rhinoceros horn.
No one has been arrested yet in connection to the incident, zoo officials said.
The Thoiry Zoo told ABC News that surveillance footage of the incident was handed over to investigators.
ABC News' Paul Pradier contributed to this report.