Holiday Sales Forecasts: Black Magic or Science?

— -- The nation's retailers, anticipating what forecasters believe will be a lackluster holiday shopping season, are launching big sales promotions earlier than ever before. But how accurate are forecasters' predictions?

A lot is riding on the accuracy of the predictions economists make about how much merchandise shoppers will scoop up between Thanksgiving and December 31: holiday sales traditionally account for up to 40 percent of a store's annual sales.

Jack Kleinhenz, chief economist for the National Retail Federation, tells ABC News that merchants rely on holiday predictions when deciding everything from how to manage inventory (how much to stock of what products) to how many extra sales people they should hire. His own prediction, he says, will be out October 4. "I'm working on it now."

How accurate are these predictions?

The business of making them has become so crowded in recent years that Joel Bines, a managing partner with consulting firm AlixPartners, calls it "a cottage industry." Over the past 10 years, he tells ABC News, a wide variety of advisory firms that benefit from publicity have realized that "making predictions is a good way to generate headlines." A cadre of service firms and suppliers, he says, now make them.

Their quality, he and other experts say, depends entirely on the assumptions on which they're based, the methodology used and the reliability of data. The results range between science and black magic.

Bines is dismissive of forecasts based on consumer surveys. "Consumers," he says, "have historically showed themselves to be not very good at telling us what they're going to do." AlixPartners' own forecast is based entirely on an analysis of historical data—20 years worth of retail sales, as compiled by the Commerce Department.

"At this point, all eyes are on the holidays," says AlixPartners' September newsletter. Referring to the welter of predictions flying around, it alludes to "all the navel-gazing and statistic-slinging that will soon start."

Unemployment, politics, gas prices, consumer confidence—these things are interesting, it says, "but, barring a major Black Swan event like 2008, they really don't matter." Why? Because an analysis of Department of Commerce sales data going back to 1993 shows (with the exception of 2008) that by August every year enough data had been collected to predict holiday sales, "based on cumulative year-to-date performance alone." Excluding 2008, it says, retail sales since 1993 through August have consistently accounted for between 65.6 percent and 66.6 percent of annual sales.

"Based on that calculus, we are predicting that, this year, holiday retail sales will increase between 4.1 percent and 4.9 percent." His prediction last year was 3.9 percent--exceeding the actual 3 percent.

ShopperTrak, which collects data from some 60,000 retail locations worldwide (about 80 percent of those in the U.S.) expects holiday sales to be up 2.4 percent from last year. ShopperTrak's founder, Bill Martin, tells ABC News his forecasts excludes online sales and take into account only those from brick and mortar stores.

ShopperTrak's prediction last year was an increase of 2.5 percent. Its 2011 prediction was for an increase of 3 percent, versus an actual increase of 4 percent.

Martin says his company has been making holiday predictions for 17 years. "We've seen a lot of other metrics come and go," he says, noting that Mastercard used to make a holiday prediction but no longer does. Those predictions for which he has the most respect are his own, the NRF's and the one done by the International Council of Shopping Centers.

"We capture data," he says. "Others take a survey of what consumers believe will happen."

Kleinhernz at the NRF says he uses a model that takes into account what's going on in the macro economy. Its components include monthly census retail sales reports, employment data, income data, credit card use, retail sales of food and beverage, and an estimate of e-commerce. He also factors in any geopolitical uncertainties that might cause shoppers to hold back.

How did NRF do last year? "We underestimated," says Kleinhernz. "We were close, but we didn't hit the nail on the head." In 2012 NRF's projected forecast was that holiday sales would increase 4.1 percent, versus the 3 percent final figure.

He views as suspect any predictions based on private, proprietary data. "It's important they be based on public data sets," he says. "With private data, how are you ever going to know if that data's right or wrong?"

He bridles at any suggestion that the phrase "cottage industry" applies to what he does. "I've been an economist for 30 years. I do surveys for the Fed—in Philadelphia and in Chicago."

There may be predictors dazzled by "Christmas trees and lights," but he isn't one of them. "I take great pride in our work. We don't always get it right, but we do the best we can."

Meanwhile, retailers are already gearing up for the season.

Toys R Us has already released its "hot-list" of what it hopes will be its best-selling toys, including a hugging Elmo doll by Hasbro ($59.99), a Razor Crazy go-cart ($399.99), and a loom that lets kids make rubber band bracelets ($14.99).

Kmart has aired its first holiday commercial—100 days before Christmas. The company, in a statement to ABC News, says it isn't so much jumping the holiday gun as it is trying to do shoppers a favor: "Customers can plan in advance in order to take advantage of layaway for holiday purchases."