The Spaniard's only defeat at Roland Garros was in the fourth round against Robin Soderling five years ago. He was never in danger of losing this time, but was occasionally on the back foot as Thiem went for his shots throughout the match and broke the top-seeded Nadal's serve twice.
"It was a dangerous match, dangerous opponent today," Nadal said. "I am happy with the way I resisted. When I had to play long points I did well. When I had to attack and move him, I think I did well."
Although Nadal broke Thiem's serve seven times overall, Thiem's powerful groundstrokes caused Nadal problems and he applauded the 20-year-old Thiem as he left center court.
"He will have his chances to become a top star and fight for these tournaments," Nadal said. "I really fought back against his attacks."
With Nadal serving at 5-1 in the first set, Thiem saved a set point with a backhand volley at the net, then forced a break point with a forehand winner into the right corner and won the game with a replica stroke on the next point.
Nadal sealed the second set with a forehand winner down the line, but Thiem broke him again and then held for a 3-1 lead in the third.
That prompted Nadal to step up a gear and secure two consecutive breaks.
"I continued to deploy an attacking style and him move around the court," Nadal said. "I put pressure on him."
Fifth-seeded David Ferrer of Spain, last year's runner-up to Nadal, seventh-seeded Andy Murray of Great Britain and 12th-seeded Richard Gasquet of France also advanced to the third round in straight sets.
Ferrer beat Italy's Simone Bolelli 6-2, 6-3, 6-2; the two-time Grand Slam champion Murray followed up with a 6-3, 6-1, 6-3 win against Australia's Marinko Matosevic; and Gasquet won 7-6 (5), 6-4, 6-4 against Argentina's Carlos Berlocq.
"We had to fight against the wind," Ferrer said. "After four games, it was difficult and the balls were heavy."
Also, 19th-seeded Kevin Anderson of South Africa beat Axel Michon of France 6-2, 6-3, 6-2; American Donald Young won 6-1, 7-6 (1), 6-3 against Spaniard Feliciano Lopez and Spaniard Guillermo Garcia-Lopez beat Frenchman Adrian Mannarino 6-4, 6-3, 4-6, 6-0.
The Associated Press contributed to this report.