The invisible hand of consumer-facing models

The smart cities movement focused on how city governments would use digital technology. But urban tech's expanded purview includes consumer markets. There, advances in AI are powering more and better models of consumer behavior in a broad range of uses that will challenge urban policymakers and planners.

Many consumer-facing models work in the open, encouraging consumers to behave in ways consistent with policy goals. Walkscore, which rates street addresses in terms of access to amenities and transit, is a good example—it has reinforced the appeal of neighborhoods that support less car dependence. But even seemingly benevolent models like Walkscore raise questions about due process. Who designed them? How were they constructed? Do they contain biases or other structural flaws that will cause harm?

More dangerous models exert influence without public awareness or oversight. Their deployment, and the use of their predictions, will be concealed. They are, far more than markets, invisible hands at work shaping our choices and behaviors, often in ways that work against public interest and policy goals. Over the next decade, these models will rear their heads in more and more everyday consumer decisions that shape the urban environment—from housing to mobility to health and work. Will they become a form of "shadow government," regulating by algorithm?