Living by the Numbers: Big Data Knows What Your Future Holds

May 18, 2013— -- Forget Big Brother. Companies and countries are discovering that algorithms programmed to scour vast quantities of data can be much more powerful. They can predict your next purchase, forecast car thefts and maybe even help cure cancer. But there is a down side.

On balmy spring evenings, Hamburg's Köhlbrand Bridge offers an idyllic postcard view of the city's harbor. The Elbe River shimmers in the reddish glow of sunset, forklifts, cranes and trucks seem to move in slow motion, and occasionally a container ship glides by. But from the standpoint of Sebastian Saxe, the area is primarily an equation with many variables. For the past four-and-a-half years, the 57-year-old mathematician has been working on his trickiest computing task yet at the behest of the company that manages the Hamburg port.

The port covers an area of 7,200 hectares (about 28 square miles). Roughly 200 trains a day traverse its 300-kilometer (186-mile) network of rails and its 130 bridges to transport goods that have arrived by ship. Saxe, as chief information officer of the Hamburg Port Authority (HPA), faces the enormous task of optimizing this logistical nightmare.

The amount of land is finite, and further expansion is not possible. Nevertheless, the Hamburg Senate has announced its goal of almost tripling container transshipment volumes in the city by 2025. This will only work if Saxe and his 60-member IT team manage to optimally exploit another resource: data. He certainly has plenty of it.

The port is already filled with sensors today. Trucks and freight trains are constantly transmitting their positions while incoming container ships report their location and speed. Sensors that constantly monitor port traffic are built into the Köhlbrand Bridge.

"Our goal is a totally interconnected, intelligent port, a Smart-port," says Saxe. He envisions a port in which, for example, a railroad drawbridge would no longer open at specific times, but rather just before a ship actually reaches it. This eliminates unnecessary delays for the railroad and at the terminal. Even the Köhlbrand Bridge would become "intelligent," in that it would report its current condition and predict future maintenance needs, all through the use of sensors. The frequency of scheduled maintenance dates was recently increased because significantly larger numbers of heavy trucks were crossing the bridge than had been planned for. This was of interest to Saxe and the HPA, but also to police and the customs agency, because some of the trucks were carrying illegal loads.

Rediscovering Data

In the end, the complex harbor logistics will create a machine that controls itself. Saxe's vision of the future is a sort of port exchange, allowing shipping companies to predict, down to the minute, how quickly their containers will be moved from the water to the road.

Many other companies worldwide are in the same position as the HPA. They are rediscovering a raw material that they, their facilities and their customers produce in excess every day: data.