In the concluding set, Gulbis raced to a 3-0 lead, thanks largely to Federer miscues. In the second game, Federer netted backhands and forehands to offer up break points, then pushed a forehand wide to give Gulbis a lead he never relinquished.
After that miss, Federer grabbed a ball and swatted it in anger straight up in the air, a rare sign of exasperation from him.
"He's Roger Federer, but he also gets tight, you know," Gulbis said. "He's probably going to make (that forehand) seven out of 10 (times). Other guys are going to make two out of 10. Mistakes happen."
The result fit with the topsy-turvy nature of this tournament: Both reigning Australian Open champions, No. 3 Stan Wawrinka and No. 2 Li Na, lost in the first round; No. 1 Serena Williams left in the second round.
Earlier Sunday, reigning Wimbledon champion Andy Murray and No. 24 Fernando Verdasco set up a fourth-round meeting by finishing off victories in matches suspended Saturday night because of fading light.
Murray found his second wind to finally get the better of Philipp Kohlschreiber of Germany, needing 40 minutes to complete a 3-6, 6-3, 6-3, 4-6, 12-10 win.
"I was cramping yesterday. My muscles were obviously fatigued," Murray said. "I actually woke up [feeling] OK ... the nice thing about this surface, compared with the hard court, is when you have a tough match on the hard courts you wake up the next day and your hips are sore, your knees are sore."
Kohlschreiber missed a chance to break Murray in the 19th game of the decider.
"He came up with some great shots when he was behind in games today," Murray said. "I thought both of us served a little bit better. It was a good finish to the match."
Murray, a two-time Grand Slam champion, crouched down and clenched both fists after hitting a two-handed backhand winner on Kohlschreiber's second serve with the score at 15-40 in the 22nd game.
"Last night was extremely tough for me. I did lot of running,'' Murray said. "Considering the circumstances, today I thought we both played some good points. We both came out probably pretty nervous.''
Murray, the 2012 U.S. Open champion, made light of a disturbed night's sleep of barely five hours.
"I was kind of ready to play at 4 or 5 in the morning," he said. "When you know you have to come back and it's 7-all the next day and every single point counts, basically you need to get off to a big start. You're obviously going to be a bit anxious."
He will get a longer rest before playing Monday against Verdasco, who easily beat 12th-seeded Frenchman Richard Gasquet 6-3, 6-2, 6-3. Murray leads Verdasco 9-1 in their career head-to-heads but they have never met on clay.
The Associated Press contributed to this report.