At Stanford University, Ge Wang teaches computer music: how to make sounds from digits and turn them into melodies and rhythm.
So when the iPhone came out — a portable computer with a built-in microphone (for voice calls), and a graphic screen — a doctoral student with entrepreneurial experience suggested they start a company to bring their research in digital audio to the masses.
Their Ocarina, a 99-cent application that turns the iPhone into a virtual flute, has become one of the iPhone's best-selling apps — to the tune of nearly $800,000. Now out is the sequel, the Leaf Trombone World Stage.
"We believe in the potential of interactive sound; we believe that everyone is inherently creative; and we want to unlock that creativity in everyone," says Wang, 31. "We want to find new types of ways to connect people, using the technology we have before us."
Apple aapl launched the iPhone in June 2007. Its popularity surged even further in July 2008, when Apple introduced a new, faster model and an online App Store, offering software applications.
The apps — an array of fun, handy or just plain practical little programs — range from games and simulated aquariums to subway maps and more. Many are free; others are priced from 99 cents to $10 and up. The App Store — stocked with 15,000 titles — is on tap to move its 1 billionth application any day now.
Most of the developers are small outfits such as Smule, the company that Wang and Jeff Smith started to launch the Ocarina and several other apps. Smith, 35, is CEO.
"One of the biggest hurdles for game developers is getting published. But with the iPhone, you develop it, and Apple starts sending you money," says Bill Trost, who created the No. 2 best-seller Koi Pond (900,000 apps sold) with four friends.
The challenge for developers such as Trost and Wang, says Gartner analyst Van Baker, is building upon the initial success. "It was easier when there wasn't as much competition," he says.
Anything's an instrument
It's not hard to get attention with Ocarina and Leaf Trombone. Using the same iPhone microphone that's intended for voice calls, you blow into it, and through Wang's computer programming, music is created. Twist and turn the phone to add vibrato.
The App Store is crowded with virtual instruments such as Band, Pocket Guitar and Finger Piano. But unlike Ocarina, they didn't crack the top 20 best-selling apps chart.
Wang's marketing strategy focused on working the social Web via YouTube, Twitter and Facebook. He shared many videos of Ocarina in use by students and other performers on YouTube.
"We're making use of technology in ways that people haven't seen, and video really helps people to understand that," says Wang. "If you see some person holding a phone like a sandwich and have sound coming out, you get it."
To plug the new Leaf Trombone, Wang reached out to a popular YouTube performing duo named Rhett and Link, who have attracted a wide audience for their amusing musical videos. He dangled an advance copy of the Leaf Trombone application, and they spun up a new version of Movin' On Up, the theme song from The Jeffersons TV show — complete with trombone solo.
The Leaf "takes me back to my school days, where you could turn anything into an instrument," says the duo's Link Neal.
He found the virtual trombone a little tougher to play than the Ocarina, but once he had it down, "It's cool to pull something out and be the Pied Piper."
Interestingly, Smule's first product wasn't musical. It was Sonic Lighter, a virtual lighter.
"We thought it would be fun," says Smith, who is still enrolled at Stanford. "There were a lot of lighters, but none gave you the true physical experience of the lighter and let you blow out the flame."
Smule has sold 300,000 copies of the 99-cent lighter program. It followed up with a virtual firecracker (Sonic Boom) then Sonic Vox, which lets you alter your voice to sound like Darth Vader or high-pitched babies. Both sold around 150,000 copies.
Then came the Ocarina, named after an ancient Chinese flute-like instrument with four holes for notes. Its inspiration was an old Nintendo video game, The Legend of Zelda.
Wang remembered that one of the characters in the game carried around an ocarina. "Depending upon the melody the character played, it would transport you to a different time," Wang says.
The iPhone community got it, in a big way.
"Within four days of its release, it became the No. 1 best-selling app on the store," recalls Wang. "People were learning how to play music for the first time, on an iPhone."
Ocarina is currently the No. 1 best seller in the music category.
For the Leaf Trombone, Wang says, he wanted to create a "whimsical, wacky" instrument, like trying to make music from blowing on a leaf of grass.
"A trombone slide just made sense."
Making music on a laptop
Wang was born in China, where he lived with his grandparents before moving to Atlanta at age 9 to join his parents. His father now teaches statistics at a New Jersey college.
Wang studied computer science at Duke, figuring he'd be a video game designer. He worked in the software industry in Washington, D.C., before heading to Princeton for his master's and Ph.D. in computer music.
"It seemed like a powerful building block for everything I loved doing," he says. "It has music and graphics and networking — everything I loved about video games."
At Princeton, Wang and his adviser, Perry Cook, co-developed ChucK, the software Wang uses to make music on iPhones and laptops. The language is used at many universities, including CalArts, Georgia Tech, Princeton and Stanford, Wang says.
In 2007, he got the "dream job" of assistant professor at Stanford's CCRMA (Center for Computer Research in Music and Acoustics). Within a few months, he had organized the Stanford Laptop Orchestra and laid the bricks for the Stanford Mobile Phone Orchestra. Then Smith came to him with the idea for Smule.
"I was hardly sleeping at all, because so much was going on, but I had to do it, because it seemed like such an awesome research opportunity," Wang says. "It can bring the vision of computer music to a much wider audience."
Stanford has a long history of seeing outside projects developed by its researchers (think Google and Hewlett-Packard), and the university has a dedicated office in place to help.
"Ge is very cognizant about keeping Smule separate and not using Stanford resources for Smule," says Steve Sano, head of Stanford's music department. "He's attentive to what the boundaries are."
Smule has 12 employees working out of a facility in nearby Menlo Park, most of them Stanford students.
Smith, a former Silicon Valley entrepreneur who brought a company public (the now defunct Tumbleweed), then quit to focus on music, serves as CEO to Wang's chief technology officer.
Smith raised $5.5 million from local investors and says his target was to do $100,000 in the first six months.
Instead, Smule did $500,000. The sales target for the first year was $1.6 million, and Smith says that will be exceeded, too.
A musical connection
Stanford student Rob Hamilton (a teaching assistant who, like many in the CCRMA program, also works at Smule) helps out with the mobile phone and laptop orchestras.
Students use ChucK on Apple MacBooks to make sounds and tones, tilting and twisting them to alter the pitch.
"People come and listen to the crazy music we do, and come out of the concerts shaking their heads," says Hamilton.
Hamilton says that as a professor, Wang "brings a lot of energy to the classroom. He's really bubbly and gets students excited about their work. He shows them how to use technology to create whatever musical fantasies they have."
For Ocarina and Leaf Trombone, Wang's wildest dream has come true: He wanted to see the world latch onto an instrument and share the love of it in a social setting.
Within the iPhone app, you can listen to other folks playing the Ocarina and see where in the globe the music is coming from.
Fans have posted sheet music, showing how to play popular songs on the instrument.
"This is a new type of social fabric," says Wang.
Leaf Trombone takes it a step further. Wang has added American Idol-like competition — you can judge performances on the phone, or just watch others playing songs such as Battle Hymn of the Republic or The Blue Danube waltz.
"Here's an opportunity to combine music with technology, where anybody can play," says Smith. "You don't have to spend 10 hours in a practice room learning how to play. With the iPhone, anyone can do it."
And for Wang, this is only the beginning of music on mobile phones. He won't commit to a complete digital instrument portfolio, but says there is much more to come.
"The phone is such an intimate piece of technology that, for better or worse, it has become a natural extension of ourselves."