The question was posed to me in 1981 by a young paper mill manager sometime between the fried catfish and the pecan pie on my first night in the small southern town that was home to his mammoth plant and would become my home periodically for the next six years. On that rainy night his words flooded my brain, drowning out the quickening tap tap tap of raindrops on the awning above our table. I recognized the oldest political questions: Home or exile? Lord or subject? Master or slave? These are eternal themes of knowledge, authority, and power that can never be settled for all time. There is no end of history; each generation must assert its will and imagination as new threats require us to retry the case in every age.

Information and communications technologies are more widespread than electricity, reaching three billion of the world's seven billion people. The entangled dilemmas of knowledge, authority, and power are no longer confined to workplaces as they were in the 1980s. Now their roots run deep through the necessities of daily life, mediating nearly every form of social participation.

It is in the nature of human attachment that every journey and expulsion sets into motion the search for home. That nostos, finding home, is among our most profound needs is evident by the price we are willing to pay for it. There is a universally shared ache to return to the place we left behind or to found a new home in which our hopes for the future can nest and grow.

Because our brains are larger than those of birds and sea turtles, we know that it is not always possible, or even desirable, to return to the same patch of earth. Home need not always correspond to a single dwelling or place. We can choose its form and location but not its meaning. Home is where we know and where we are known, where we love and are beloved. Home is mastery, voice, relationship, and sanctuary: part freedom, part flourishing… part refuge, part prospect.

In 2000 a group of computer scientists and engineers at Georgia Tech collaborated on a project called the “Aware Home.” It was meant to be a “living laboratory” for the study of “ubiquitous computing.” They imagined a “human-home symbiosis” in which many animate and inanimate processes would be captured by an elaborate network of “context aware sensors” embedded in the house and by wearable computers worn by the home’s occupants.

There were three working assumptions: first, the scientists and engineers understood that the new data systems would produce an entirely new knowledge domain. Second, it was assumed that the rights to that new knowledge and the power to use it to improve one's life would belong exclusively to the people who live in the house. Third, the team assumed that for all of its digital wizardry, the Aware Home would take its place as a modern incarnation of the ancient conventions that understand “home” as the private sanctuary of those who dwell within its walls.

All of this was expressed in the engineering plan. It emphasized trust, simplicity, the sovereignty of the individual, and the inviolability of the home as a private domain. The Aware Home information system was imagined as a simple “closed loop” with only two nodes and controlled entirely by the home's occupants. Because the house would be “constantly monitoring the occupants' whereabouts and activities… even tracing its inhabitants' medical conditions,” the team concluded, “there is a clear need to give the occupants knowledge and control of the distribution of this information.” All the information was to be stored on the occupants' wearable computers “to insure the privacy of an individual's information.”

By 2018, the global “smart-home” market was valued at $36 billion and expected to reach $151 billion by 2023. … As a result of the merger with Google, the thermostat, like other Nest products, will be built with Google's artificial intelligence capabilities, including its personal digital “assistant.” Like the Aware Home, the thermostat and its brethren devices create immense new stores of knowledge and therefore new power—but for whom?

Wi-Fi–enabled and networked, the thermostat's intricate, personalized data stores are uploaded to Google's servers. Each thermostat comes with a “privacy policy,” a “terms-of-service agreement,” and an “end-user licensing agreement.” These reveal oppressive privacy and security consequences in which sensitive household and personal information are shared with other smart devices, unnamed personnel, and third parties for the purposes of predictive analyses and sales to other unspecified parties. Nest takes little responsibility for the security of the information it collects and none for how the other companies in its ecosystem will put those data to use. A detailed analysis of Nest's policies by two University of London scholars concluded that were one to enter into the Nest ecosystem of connected devices and apps, each with their own equally burdensome and audacious terms, the purchase of a single home thermostat would entail the need to review nearly a thousand so-called contracts.

Should the customer refuse to agree to Nest's stipulations, the terms of service indicate that the functionality and security of the thermostat will be deeply compromised, no longer supported by the necessary updates meant to ensure its reliability and safety. The consequences can range from frozen pipes to failed smoke alarms to an easily hacked internal home system.

By 2018, the assumptions of the Aware Home were gone with the wind. Where did they go? What was that wind? The Aware Home, like many other visionary projects, imagined a digital future that empowers individuals to lead more-effective lives. What is most critical is that in the year 2000 this vision naturally assumed an unwavering commitment to the privacy of individual experience. Should an individual choose to render her experience digitally, then she would exercise exclusive rights to the knowledge garnered from such data, as well as exclusive rights to decide how such knowledge might be put to use. Today these rights to privacy, knowledge, and application have been usurped by a bold market venture powered by unilateral claims to others' experience and the knowledge that flows from it.

Eventually, surveillance capitalists discovered that the most-predictive behavioral data come from intervening in the state of play in order to nudge, coax, tune, and herd behavior toward profitable outcomes. Competitive pressures produced this shift, in which automated machine processes not only know our behavior but also shape our behavior at scale. With this reorientation from knowledge to power, it is no longer enough to automate information flows about us; the goal now is to automate us. In this phase of surveillance capitalism's evolution, the means of production are subordinated to an increasingly complex and comprehensive "means of behavioral modification." In this way, surveillance capitalism births a new species of power that I call instrumentarianism. Instrumentarian power knows and shapes human behavior toward others' ends. Instead of armaments and armies, it works its will through the automated medium of an increasingly ubiquitous computational architecture of "smart" networked devices, things, and spaces.

In the coming chapters we will follow the growth and dissemination of these operations and the instrumentarian power that sustains them. Indeed, it has become difficult to escape this bold market project, whose tentacles reach from the gentle herding of innocent Pokémon Go players to eat, drink, and purchase in the restaurants, bars, fast-food joints, and shops that pay to play in its behavioral futures markets to the ruthless expropriation of surplus from Facebook profiles for the purposes of shaping individual behavior, whether it’s buying pimple cream at 5:45 P.M. on Friday, clicking “yes” on an offer of new running shoes as the endorphins race through your brain after your long Sunday morning run, or voting next week. Just as industrial capitalism was driven to the continuous intensification of the means of production, so surveillance capitalists and their market players are now locked into the continuous intensification of the means of behavioral modification and the gathering might of instrumentarian power.

Surveillance capitalism runs contrary to the early digital dream, consigning the Aware Home to ancient history. Instead, it strips away the illusion that the networked form has some kind of indigenous moral content, that being “connected” is somehow intrinsically pro-social, innately inclusive, or naturally tending toward the democratization of knowledge. Digital connection is now a means to others’ commercial ends. At its core, surveillance capitalism is parasitic and self-referential. It revives Karl Marx’s old image of capitalism as a vampire that feeds on labor, but with an unexpected turn. Instead of labor, surveillance capitalism feeds on every aspect of every human’s experience.

Google invented and perfected surveillance capitalism in much the same way that a century ago General Motors invented and perfected managerial capitalism. Google was the pioneer of surveillance capitalism in thought and practice, the deep pocket for research and development, and the trailblazer in experimentation and implementation, but it is no longer the only actor on this path. Surveillance capitalism quickly spread to Facebook and later to Microsoft. Evidence suggests that Amazon has veered in this direction, and it is a constant challenge to Apple, both as an external threat and as a source of internal debate and conflict.

Surveillance capitalism is no longer confined to the competitive dramas of the large internet companies, where behavioral futures markets were first aimed at online advertising. Its mechanisms and economic imperatives have become the default model for most internet-based businesses. Eventually, competitive pressure drove expansion into the offline world, where the same foundational mechanisms that expropriate your online browsing, likes, and clicks are trained on your run in the park, breakfast conversation, or hunt for a parking space. Today’s prediction products are traded in behavioral futures markets that extend beyond targeted online ads to many other sectors, including insurance, retail, finance, and an ever-widening range of goods and services companies determined to participate in these new and profitable markets. Whether it’s a “smart” home device, what the insurance companies call “behavioral underwriting,” or any one of thousands of other transactions, we now pay for our own domination.

Surveillance capitalism’s products and services are not the objects of a value exchange. They do not establish constructive producer-consumer reciprocities. Instead, they are the “hooks” that lure users into their extractive operations in which our personal experiences are scraped and packaged as the means to others’ ends. We are not surveillance capitalism’s “customers.” Although the saying tells us “If it’s free, then you are the product,” that is also incorrect. We are the sources of surveillance capitalism’s crucial surplus: the objects of a technologically advanced and increasingly inescapable raw-material-extraction operation. Surveillance capitalism’s actual customers are the enterprises that trade in its markets for future behavior.

Surveillance capitalism operates through unprecedented asymmetries in knowledge and the power that accrues to knowledge. Surveillance capitalists know everything about us, whereas their operations are designed to be unknowable to us. They accumulate vast domains of new knowledge from us, but not for us. They predict our futures for the sake of others’ gain, not ours. As long as surveillance capitalism and its behavioral futures markets are allowed to thrive, ownership of the new means of behavioral modification eclipses ownership of the means of production as the fountainhead of capitalist wealth and power in the twenty-first century.

Just as industrial civilization flourished at the expense of nature and now threatens to cost us the Earth, an information civilization shaped by surveillance capitalism and its new instrumentarian power will thrive at the expense of human nature and will threaten to cost us our humanity. The industrial legacy of climate chaos fills us with dismay, remorse, and fear. As surveillance capitalism becomes the dominant form of information capitalism in our time, what fresh legacy of damage and regret will be mourned by future generations?

When we encounter something unprecedented, we automatically interpret it through the lenses of familiar categories, thereby rendering invisible precisely that which is unprecedented. A classic example is the notion of the “horseless carriage” to which people reverted when confronted with the unprecedented facts of the automobile. A tragic illustration is the encounter between indigenous people and the first Spanish conquerors. When the Taínos of the pre-Columbian Caribbean islands first laid eyes on the sweating, bearded Spanish soldiers trudging across the sand in their brocade and armor, how could they possibly have recognized the meaning and portent of that moment? Unable to imagine their own destruction, they reckoned that those strange creatures were gods and welcomed them with intricate rituals of hospitality. This is how the unprecedented reliably confounds understanding; existing lenses illuminate the familiar, thus obscuring the original by turning the unprecedented into an extension of the past. This contributes to the normalization of the abnormal, which makes fighting the unprecedented even more of an uphill climb.

I began to study the emergence of what I would eventually call surveillance capitalism in 2006, interviewing entrepreneurs and staff in a range of tech companies in the US and the UK. For several years I thought that the unexpected and disturbing practices that I documented were detours from the main road: management oversights or failures of judgment and contextual understanding.

Of course, the emergence of the unprecedented in economic history cannot be compared to a house fire. The portents of a catastrophic fire were unprecedented in my experience, but they were not original. In contrast, surveillance capitalism is a new actor in history, both original and sui generis. It is of its own kind and unlike anything else: a distinct new planet with its own physics of time and space, its sixty-seven-hour days, emerald sky, inverted mountain ranges, and dry water.

The unprecedented nature of surveillance capitalism has enabled it to elude systematic contest because it cannot be adequately grasped with our existing concepts. We rely on categories such as “monopoly” or “privacy” to contest surveillance capitalist practices. And although these issues are vital, and even when surveillance capitalist operations are also monopolistic and a threat to privacy, the existing categories nevertheless fall short in identifying and contesting the most crucial and unprecedented facts of this new regime.

Our effort to confront the unprecedented begins with the recognition that we hunt the puppet master, not the puppet. A first challenge to comprehension is the confusion between surveillance capitalism and the technologies it employs. Surveillance capitalism is not technology; it is a logic that imbues technology and commands it into action. Surveillance capitalism is a market form that is unimaginable outside the digital milieu, but it is not the same as the “digital.” As we saw in the story of the Aware Home, and as we shall see again in Chapter 2, the digital can take many forms depending upon the social and economic logics that bring it to life. It is capitalism that assigns the price tag of subjugation and helplessness, not the technology.

That surveillance capitalism is a logic in action and not a technology is a vital point because surveillance capitalists want us to think that their practices are inevitable expressions of the technologies they employ. For example, in 2009 the public first became aware that Google maintains our search histories indefinitely: data that are available as raw-material supplies are also available to intelligence and law-enforcement agencies. When questioned about these practices, the corporation’s former CEO Eric Schmidt mused, “The reality is that search engines including Google do retain this information for some time.”

In truth, search engines do not retain, but surveillance capitalism does. Schmidt’s statement is a classic of misdirection that bewilders the public by conflating commercial imperatives and technological necessity. It camouflages the concrete practices of surveillance capitalism and the specific choices that impel Google’s brand of search into action. Most significantly, it makes surveillance capitalism’s practices appear to be inevitable when they are actually meticulously calculated and lavishly funded means to self-dealing commercial ends. We will examine this notion of “inevitabilism” in depth in Chapter 7. For now, suffice to say that despite all the futuristic sophistication of digital innovation, the message of the surveillance capitalist companies barely differs from the themes once glorified in the motto of the 1933 Chicago World’s Fair: “Science Finds—Industry Applies—Man Conforms.”

In order to challenge such claims of technological inevitability, we must establish our bearings. We cannot evaluate the current trajectory of information civilization without a clear appreciation that technology is not and never can be a thing in itself, isolated from economics and society. This means that technological inevitability does not exist. Technologies are always economic means, not ends in themselves: in modern times, technology’s DNA comes already patterned by what the sociologist Max Weber called the “economic orientation.”

Surveillance capitalism employs many technologies, but it cannot be equated with any technology. Its operations may employ platforms, but these operations are not the same as platforms. It employs machine intelligence, but it cannot be reduced to those machines. It produces and relies on algorithms, but it is not the same as algorithms. Surveillance capitalism’s unique economic imperatives are the puppet masters that hide behind the curtain orienting the machines and summoning them to action. … Despite this, each generation stumbles into the quicksand of forgetting that technology is an expression of other interests. In modern times this means the interests of capital, and in our time it is surveillance capital that commands the digital milieu and directs our trajectory toward the future.

Surveillance capitalism is not an accident of overzealous technologists, but rather a rogue capitalism that learned to cunningly exploit its historical conditions to ensure and defend its success.

I argue that the significance of these developments is best understood as the privatization of the division of learning in society, the critical axis of social order in the twenty-first century.

For all of Google's technological prowess and computational talent, the real credit for its success goes to the radical social relations that the company declared as facts, beginning with its disregard for the boundaries of private human experience and the moral integrity of the autonomous individual. Instead, surveillance capitalists asserted their right to invade at will, usurping individual decision rights in favor of unilateral surveillance and the self-authorized extraction of human experience for others' profit. These invasive claims were nurtured by the absence of law to impede their progress, the mutuality of interests between the fledgling surveillance capitalists and state intelligence agencies, and the tenacity with which the corporation defended its new territories.

The evidence of our psychic numbing is that only a few decades ago US society denounced mass behavior-modification techniques as unacceptable threats to individual autonomy and the democratic order. Today the same practices meet little resistance or even discussion as they are routinely and pervasively deployed in the march toward surveillance revenues.

Finally, I consider surveillance capitalism’s operations as a challenge to the elemental right to the future tense, which accounts for the individual’s ability to imagine, intend, promise, and construct a future. It is an essential condition of free will and, more poignantly, of the inner resources from which we draw the will to will. I ask and answer the question How did they get away with it?

Here we explore this new reality business, as all aspects of human experience are claimed as raw-material supplies and targeted for rendering into behavioral data. Much of this new work is accomplished under the banner of “personalization,” a camouflage for aggressive extraction operations that mine the intimate depths of everyday life. As competition intensifies, surveillance capitalists learn that extracting human experience is not enough. The most-predictive raw-material supplies come from intervening in our experience to shape our behavior in ways that favor surveillance capitalists’ commercial outcomes.

I argue that instrumentarianism is an unprecedented species of power that has defied comprehension in part because it has been subjected to the “horseless-carriage” syndrome. Instrumentarian power has been viewed through the old lenses of totalitarianism, obscuring what is different and dangerous. Totalitarianism was a transformation of the state into a project of total possession. Instrumentarianism and its materialization in Big Other signal the transformation of the market into a project of total certainty, an undertaking that is unimaginable outside the digital milieu and the logic of surveillance capitalism.

The youngest members of our societies already experience many of these destructive dynamics in their attachment to social media, the first global experiment in the human hive. I consider the implications of these developments for a second elemental right: the right to sanctuary. The human need for a space of inviolable refuge has persisted in civilized societies from ancient times but is now under attack as surveillance capital creates a world of “no exit” with profound implications for the human future at this new frontier of power.

Surveillance capitalism and its rapidly accumulating instrumentarian power exceed the historical norms of capitalist ambitions, claiming dominion over human, societal, and political territories that range far beyond the conventional institutional terrain of the private firm or the market. As a result, surveillance capitalism is best described as a coup from above, not an overthrow of the state but rather an overthrow of the people's sovereignty and a prominent force in the perilous drift toward democratic deconsolidation that now threatens Western liberal democracies.

The top five internet companies—Apple, Google, Amazon, Microsoft, and Facebook—are often regarded as a single entity with similar strategies and interests, but when it comes to surveillance capitalism, this is not the case.

First, it is necessary to distinguish between capitalism and surveillance capitalism. As I discuss in more detail in Chapter 3, that line is defined in part by the purposes and methods of data collection. When a firm collects behavioral data with permission and solely as a means to product or service improvement, it is committing capitalism but not surveillance capitalism.

One important implication of these distinctions is that even when our societies address capitalist harms produced by the tech companies, such as those related to monopoly or privacy, those actions do not ipso facto interrupt a firm's commitment to and continued elaboration of surveillance capitalism. For example, calls to break up Google or Facebook on monopoly grounds could easily result in establishing multiple surveillance capitalist firms, though at a diminished scale, and thus clear the way for more surveillance capitalist competitors.

It is also the case that surveillance capitalism was invented in the United States: in Silicon Valley and at Google. This makes it an American invention, which, like mass production, became a global reality. For this reason, much of this text focuses on developments in the US, although the consequences of these developments belong to the world.

On August 9, 2011, three events separated by thousands of miles captured the bountiful prospects and gathering dangers of our emerging information civilization. First, Silicon Valley pioneer Apple promised a digital dream of new solutions to old economic and social problems, and finally surpassed Exxon Mobil as the world's most highly capitalized corporation. Second, a fatal police shooting in London sparked extensive rioting across the city, engulfing the country in a wave of violent protests. A decade of explosive digital growth had failed to mitigate the punishing austerity of neoliberal economics and the extreme inequality that it produced. Too many people had come to feel excluded from the future, embracing rage and violence as their only remedies. Third, Spanish citizens asserted their rights to a human future when they challenged Google by demanding "the right to be forgotten." This milestone alerted the world to how quickly the cherished dreams of a more just and democratic digital future were shading into nightmare, and it foreshadowed a global political contest over the fusion of digital capabilities and capitalist ambitions. We relive that August day every day as in some ancient fable, doomed to retrace this looping path until the soul of our information civilization is finally shaped by democratic action, private power, ignorance, or drift.

Apple thundered onto the music scene in the midst of a pitched battle between demand and supply. On one side were young people whose enthusiasm for Napster and other forms of music file sharing expressed a new quality of demand: consumption my way, what I want, when I want it, where I want it. On the other side were music-industry executives who chose to instill fear and to crush that demand by hunting down and prosecuting some of Napster's most-ardent users. Apple bridged the divide with a commercially and legally viable solution that aligned the company with the changing needs of individuals while working with industry incumbents. Napster hacked the music industry, but Apple appeared to have hacked capitalism.

It is easy to forget just how dramatic Apple's hack really was. The company's profits soared largely on the strength of its iPod/iTunes/iPhone sales. Bloomberg Businessweek described Wall Street analysts as "befuddled" by this mysterious Apple "miracle." As one gushed, "We can't even model out some of the possibilities.… It's like a religion." Even today the figures are staggering: three days after the launch of the Windows-compatible iTunes platform in October 2003, listeners downloaded a million copies of the free iTunes software and paid for a million songs, prompting Steve Jobs to announce, "In less than one week we've broken every record and become the largest online music company in the world." Within a month there were five million downloads, then ten million three months later, then twenty-five million three months after that. Four and a half years later, in January 2007, that number rose to two billion, and six years later, in 2013, it was 25 billion. In 2008 Apple surpassed Walmart as the world's largest music retailer. iPod sales were similarly spectacular, exploding from 1 million units per month after the music store's launch to 100 million less than four years later, when Apple subsumed the iPod's functions in its revolutionary iPhone, which drove another step-function of growth. A 2017 study of stock market returns concluded that Apple had generated more profit for investors than any other US company in the previous century.

Henry Ford reduced the price of an automobile by 60 percent with a revolutionary industrial logic that combined high volume and low unit cost. He called it “mass production,” summarized in his famous maxim “You can have any color car you want so long as it’s black.”

How should we understand this success? Apple’s “miracle” is typically credited to its design and marketing genius. Consumers’ eagerness to have “what I want, when, where, and how I want it” is taken as evidence of the demand for “convenience” and sometimes even written off as narcissism or petulance. In my view, these explanations pale against the unprecedented magnitude of Apple’s accomplishments. We have contented ourselves for too long with superficial explanations of Apple’s unprecedented fusion of capitalism and the digital rather than digging deeper into the historical forces that summoned this new form to life.

Just as Ford tapped into a new mass consumption, Apple was among the first to experience explosive commercial success by tapping into a new society of individuals and their demand for individualized consumption. The inversion implied a larger story of a commercial reformation in which the digital era finally offered the tools to shift the focus of consumption from the mass to the individual, liberating and reconfiguring capitalism's operations and assets. It promised something utterly new, urgently necessary, and operationally impossible outside the networked spaces of the digital. Its implicit promise of an advocacy-oriented alignment with our new needs and values was a confirmation of our inner sense of dignity and worth, ratifying the feeling that we matter. In offering consumers respite from an institutional world that was indifferent to their individual needs, it opened the door to the possibility of a new rational capitalism able to reunite supply and demand by connecting us to what we really want in exactly the ways that we choose.

As I shall argue in the coming chapters, the same historical conditions that sent the iPod on its wild ride summoned the emancipatory promise of the internet into our everyday lives as we sought remedies for inequality and exclusion. Of most significance for our story, these same conditions would provide important shelter for surveillance capitalism's ability to root and flourish. More precisely, the Apple miracle and surveillance capitalism each owes its success to the destructive collision of two opposing historical forces. One vector belongs to the longer history of modernization and the centuries-long societal shift from the mass to the individual. The opposing vector belongs to the decades-long elaboration and implementation of the neoliberal economic paradigm: its political economics, its transformation of society, and especially its aim to reverse, subdue, impede, and even destroy the individual urge toward psychological self-determination and moral agency.

Capitalism evolves in response to the needs of people in a time and place. Henry Ford was clear on this point: "Mass production begins in the perception of a public need." At a time when the Detroit automobile manufacturers were preoccupied with luxury vehicles, Ford stood alone in his recognition of a nation of newly modernizing individuals—farmers, wage earners, and shopkeepers—who had little and wanted much, but at a price they could afford. Their "demand" issued from the same conditions of existence that summoned Ford and his men as they discovered the transformational power of a new logic of standardized, high-volume, low-unit-cost production. Ford's famous "five-dollar day" was emblematic of a systemic logic of reciprocity. In paying assembly-line workers higher wages than anyone had yet imagined, he recognized that the whole enterprise of mass production rested upon a thriving population of mass consumers.

Although the market form and its bosses had many failings and produced many violent facts, its populations of newly modernizing individuals were valued as the necessary sources of customers and employees. It depended upon its communities in ways that would eventually lead to a range of institutionalized reciprocities. On the outside the drama of access to affordable goods and services was bound by democratic measures and methods of oversight that asserted and protected the rights and safety of workers and consumers. On the inside were durable employment systems, career ladders, and steady increases in wages and benefits. Indeed, considered from the vantage point of the last forty years, during which this market form was systematically deconstructed, its reciprocity with the social order, however vexed and imperfect, appears to have been one of its most-salient features.

What are these modernities and how do they matter to our story? The advent of the individual as the locus of moral agency and choice initially occurred in the West, where the conditions for this emergence first took hold. First let's establish that the concept of “individualization” should not be confused with the neoliberal ideology of “individualism” that shifts all responsibility for success or failure to a mythical, atomized, isolated individual, doomed to a life of perpetual competition and disconnected from relationships, community, and society. Neither does it refer to the psychological process of “individuation” that is associated with the lifelong exploration of self-development. Instead, individualization is a consequence of long-term processes of modernization.

Until the last few minutes of human history, each life was foretold in blood and geography, sex and kin, rank and religion. I am my mother's daughter. I am my father's son. The sense of the human being as an individual emerged gradually over centuries, clawed from this ancient vise. Around two hundred years ago, we embarked upon the first modern road where life was no longer handed down one generation to the next according to the traditions of village and clan. This “first modernity” marks the time when life became “individualized” for great numbers of people as they separated from traditional norms, meanings, and rules. That meant each life became an open-ended reality to be discovered rather than a certainty to be enacted. Even where the traditional world remains intact for many people today, it can no longer be experienced as the only possible story.

I often think about the courage of my great-grandparents. What mixture of sadness, terror, and exhilaration did they feel when in 1908, determined to escape the torments of the Cossacks in their tiny village outside of Kiev, they packed their five children, including my four-year-old grandfather Max, and all their belongings into a wagon and pointed the horses toward a steamer bound for America? Like millions of other pioneers of this first modernity, they escaped a still-feudal world and found themselves improvising a profoundly new kind of life. Max would later marry Sophie and build a family far from the rhythms of the villages that birthed them. The Spanish poet Antonio Machado captured the exhilaration and daring of these first-modernity individuals in his famous song: “Traveler, there is no road; the road is made as you go.” This is what “search” has meant: a journey of exploration and self-creation, not an instant swipe to already composed answers.

Still, the new industrial society retained many of the hierarchical motifs of the older feudal world in its patterns of affiliation based on class, race, occupation, religion, ethnicity, sex, and the leviathans of mass society: its corporations, workplaces, unions, churches, political parties, civic groups, and school systems. This new world order of the mass and its bureaucratic logic of concentration, centralization, standardization, and administration still provided solid anchors, guidelines, and goals for each life.

The hierarchical social compact and mass society of the first modernity promised predictable rewards, but their very success was the knife that cut us loose and sent us tumbling onto the shores of the second modernity, propelling us toward more-intricate and richly patterned lives. Education and knowledge work increased mastery of language and thought, the tools with which we create personal meaning and form our own opinions. Communication, information, consumption, and travel stimulated individual self-consciousness and imaginative capabilities, informing perspectives, values, and attitudes in ways that could no longer be contained by predefined roles or group identity. Improved health and longer life spans provided the time for a self-life to deepen and mature, fortifying the legitimacy of personal identity over and against a priori social norms.

Even when we revert to traditional roles, these are choices now rather than absolute truths imposed at birth. As the great clinician of identity, Erik Erikson, once described it, “The patient of today suffers most under the problem of what he should believe and who he should—or… might—be or become; while the patient of early psychoanalysis suffered most under inhibitions which prevented him from being what and who he thought he knew he was.” This new mentality has been most pronounced in wealthier countries, but research shows significant pluralities of second-modernity individuals in nearly every region of the world.

The first modernity suppressed the growth and expression of self in favor of collective solutions, but by the second modernity, the self is all we have. The new sense of psychological sovereignty broke upon the world long before the internet appeared to amplify its claims. We learn through trial and error how to stitch together our lives. Nothing is given. Everything must be reviewed, renegotiated, and reconstructed on the terms that make sense to us: family, religion, sex, gender, morality, marriage, community, love, nature, social connections, political participation, career, food…

Indeed, it was this new mentality and its demands that summoned the internet and the burgeoning information apparatus into our everyday lives. The burdens of life without a fixed destiny turned us toward the empowering information-rich resources of the new digital milieu as it offered new ways to amplify our voices and forge our own chosen patterns of connection. So profound is this phenomenon that one can say without exaggeration that the individual as the author of his or her own life is the protagonist of our age, whether we experience this fact as emancipation or affliction.

Western modernity had formed around a canon of principles and laws that confer inviolable individual rights and acknowledge the sanctity of each individual life. However, it was not until the second modernity that felt experience began to catch up with formal law. This felt truth has been expressed in new demands to make actual in everyday life what is already established in law.

In spite of its liberating potential, the second modernity was slated to become a hard place to live, and our conditions of existence today reflect this trouble. Some of the challenges of the second modernity arise from the inevitable costs associated with the creation and sustenance of one’s own life, but second-modernity instability is also the result of institutionalized shifts in economic and social policies and practices associated with the neoliberal paradigm and its rise to dominance. This far-reaching paradigm has been aimed at containing, rechanneling, and reversing the secular wave of second-modernity claims to self-determination and the habitats in which those claims can thrive.

Neoliberal economists had been waiting in the wings for this opportunity, and their ideas flowed into the “policy vacuum” that now bedeviled both governments. Led by the Austrian economist Friedrich Hayek, fresh from his 1974 Nobel Prize, and his American counterpart Milton Friedman, who received the Nobel two years later, they had honed their radical free-market economic theory, political ideology, and pragmatic agenda throughout the postwar period at the fringe of their profession, under the shadow of Keynesian domination, and now their time had come.

Hayek explained the necessity of absolute individual and collective submission to the exacting disciplines of the market as an unknowable "extended order" that supersedes the legitimate political authority vested in the state: "Modern economics explains how such an extended order...

In the “crisis of democracy” zeitgeist, the neoliberal vision and its reversion to market metrics was deeply attractive to politicians and policy makers, both as the means to evade political ownership of tough economic choices and because it promised to impose a new kind of order where disorder was feared. The absolute authority of market forces would be enshrined as the ultimate source of imperative control, displacing democratic contest and deliberation with an ideology of atomized individuals sentenced to perpetual competition for scarce resources. The disciplines of competitive markets promised to quiet unruly individuals and even transform them back into subjects too preoccupied with survival to complain.

As the old collectivist enemies had receded, new ones took their place: state regulation and oversight, social legislation and welfare policies, labor unions and the institutions of collective bargaining, and the principles of democratic politics. Indeed, all these were to be replaced by the market’s version of truth, and competition would be the solution to growth. The new aims would be achieved through supply-side reforms, including deregulation, privatization, and lower taxes.

Thirty-five years before Hayek and Friedman’s ascendance, the great historian Karl Polanyi wrote eloquently on the rise of the market economy. Polanyi’s studies led him to conclude that the operations of a self-regulating market are profoundly destructive when allowed to run free of countervailing laws and policies. He described the double movement: “a network of measures and policies… integrated into powerful institutions designed to check the action of the market relative to labor, land, and money.”

The double movement was scheduled for demolition under the neoliberal flag, and implementation began immediately. In 1976, the same year that Jensen and Meckling published their pathbreaking analysis, President Jimmy Carter initiated the first significant efforts to radically align the corporation with Wall Street’s market metrics, targeting the airline, transportation, and financial sectors with a bold program of deregulation. What began as a “ripple” turned into “a tidal wave that washed away controls from large segments of the economy in the last two decades of the twentieth century.”

Thus began the disaggregation and diminishment of the US public firm. The public corporation as a social institution was reinterpreted as a costly error, and its long-standing reciprocities with customers and employees were recast as destructive violations of market efficiency. Financial carrots and sticks persuaded executives to dismember and shrink their companies, and the logic of capitalism shifted from the profitable production of goods and services to increasingly exotic forms of financial speculation. The disciplines imposed by the new market operations stripped capitalism down to its raw core, and by 1989 Jensen confidently proclaimed the “eclipse of the public corporation.”

By 2000, US public corporations employed fewer than half as many Americans as they did in 1970.35 In 2009 there were only half as many public firms as in 1997.

These principles, culled from a once-extremist philosophy, were canonized as standard practice across commercial, financial, and legal domains. By 2000, US public corporations employed fewer than half as many Americans as they did in 1970.35 In 2009 there were only half as many public firms as in 1997. The public corporation had become “unnecessary for production, unsuited for stable employment and the provision of social welfare services, and incapable of proving a reliable long-term return on investment.”

On August 9, 2011, around the same time that cheers erupted in Apple’s conference room, 16,000 police officers flooded the streets of London, determined to quell “the most widespread and prolonged breakdown of order in London’s history since the Gordon riot of 1780.” The rioting had begun four nights earlier when a peaceful vigil triggered by the police shooting of a young man suddenly turned violent. In the days that followed, the number of rioters swelled as looting and arson spread to twenty-two of London’s thirty-two boroughs and other major cities across Britain. Over four days of street action, thousands of people caused property damage of over $50 million, and 3,000 people were arrested.

Even as Apple’s ascension appeared to ratify the claims of second-modernity individuals, the streets of London told the grim legacy of a three-decade experiment in economic growth through exclusion. One week after the rioting, an article by sociologist Saskia Sassen in the Daily Beast observed that “if there’s one underlying condition, it has to do with the unemployment and bitter poverty among people who desire to be part of the middle class and who are keenly aware of the sharp inequality between themselves and their country’s wealthy elite. These are in many ways social revolutions with a small ‘r,’ protests against social conditions that have become unbearable.”

What were the social conditions that had become so unbearable? Many analysts agreed that the tragedy of Britain’s riots was set into motion by neoliberalism’s successful transformation of society: a program that was most comprehensively executed in the UK and the US. Indeed, research from the London School of Economics based on interviews with 270 people who had participated in the rioting reported on the predominant theme of inequality: “no job, no money.” The terms of reference in nearly every study sound the same drumbeat: lack of opportunity, lack of access to education, marginalization, deprivation, grievance, hopelessness. And although the London riots differed substantially from other protests that preceded and followed, most notably the Indignados movement that began with a large-scale public mobilization in Madrid in May 2011 and the Occupy movement that would emerge on September 17 in Wall Street’s Zuccotti Park, they shared a point of origin in the themes of economic inequality and exclusion.

The US, the UK, and most of Europe entered the second decade of the twenty-first century facing economic and social inequalities more extreme than anything since the Gilded Age and comparable to some of the world’s poorest countries. Despite a decade of explosive digital growth that included the Apple miracle and the penetration of the internet into everyday life, dangerous social divisions suggested an even more stratified and antidemocratic future. “In the age of new consensus financial policy stabilization,” one US economist wrote, “the economy has witnessed the largest transfer of income to the top in history.” A sobering 2016 report from the International Monetary Fund warned of instability, concluding that the global trends toward neoliberalism “have not delivered as expected.” Instead, inequality had significantly diminished “the level and the durability of growth” while increasing volatility and creating permanent vulnerability to economic crisis.

The quest for effective life had been driven to the breaking point under the aegis of market freedom. Two years after the North London riots, research in the UK showed that by 2013, poverty fueled by lack of education and unemployment already excluded nearly a third of the population from routine social participation. Another UK report concluded, “Workers on low and middle incomes are experiencing the biggest decline in their living standards since reliable records began in the mid-19th Century.” By 2015, austerity measures had eliminated 19 percent, or 18 billion pounds, from the budgets of local authorities, had forced an 8 percent cut in child protection spending, and had caused 150,000 pensioners to no longer enjoy access to vital services. Buy 2014 nearly half of the US population lived in functional poverty, with the highest wage in the bottom half of earners at about $34,000.49 A 2012 US Department of Agriculture survey showed that close to 49 million people lived in “food-insecure” households.

In Capital in the Twenty-First Century, the French economist Thomas Piketty integrated years of income data to derive a general law of accumulation: the rate of return on capital tends to exceed the rate of economic growth. This tendency, summarized as r > g, is a dynamic that produces ever-more-extreme income divergence and with it a range of antidemocratic social consequences long predicted as harbingers of an eventual crisis of capitalism.

A précis of Piketty’s extensive research may be stated simply: capitalism should not be eaten raw. Capitalism, like sausage, is meant to be cooked by a democratic society and its institutions because raw capitalism is antisocial. As Piketty warns, “A market economy… if left to itself… contains powerful forces of divergence, which are potentially threatening to democratic societies and to the values of social justice on which they are based.” Many scholars have taken to describing these new conditions as neofeudalism, marked by the consolidation of elite wealth and power far beyond the control of ordinary people and the mechanisms of democratic consent. Piketty calls it a return to “patrimonial capitalism,” a reversion to a premodern society in which one’s life chances depend upon inherited wealth rather than meritocratic achievement.

We now have the tools to grasp the collision in all of its destructive complexity: what is unbearable is that economic and social inequalities have reverted to the preindustrial “feudal” pattern but that we, the people, have not. We are not illiterate peasants, serfs, or slaves. Whether “middle class” or “marginalized,” we share the collective historical condition of individualized persons with complex social experiences and opinions. We are hundreds of millions or even billions of second-modernity people whom history has freed both from the once-immutable facts of a destiny told at birth and from the conditions of mass society. We know ourselves to be worthy of dignity and the opportunity to live an effective life. This is existential toothpaste that, once liberated, cannot be squeezed back into the tube.

This is the existential contradiction of the second modernity that defines our conditions of existence: we want to exercise control over our own lives, but everywhere that control is thwarted. Individualization has sent each one of us on the prowl for the resources we need to ensure effective life, but at each turn we are forced to do battle with an economics and politics from whose vantage point we are but ciphers. We live in the knowledge that our lives have unique value, but we are treated as invisible. As the rewards of late-stage financial capitalism slip beyond our grasp, we are left to contemplate the future in a bewilderment that erupts into violence with increasing frequency. Our expectations of psychological self-determination are the grounds upon which our dreams unfold, so the losses we experience in the slow burn of rising inequality, exclusion, pervasive competition, and degrading stratification are not only economic. They slice us to the quick in dismay and bitterness because we know ourselves to be worthy of individual dignity and the right to a life on our own terms.

The Apple inversion implied trustworthy relationships of advocacy and reciprocity embedded in an alignment of commercial operations with consumers’ genuine interests. It held out the promise of a new digital market form that might transcend the collision: an early intimation of a third-modernity capitalism summoned by the self-determining aspirations of individuals and indigenous to the digital milieu. The opportunity for “my life, my way, at a price I can afford” was the human promise that quickly lodged at the very heart of the commercial digital project, from iPhones to one-click ordering to massive open online courses to on-demand services to hundreds of thousands of web-based enterprises, apps, and devices.

There were missteps, shortfalls, and vulnerabilities, to be sure. The potential significance of Apple’s tacit new logic was never fully grasped, even by the company itself. Instead, the corporation produced a steady stream of contradictions that signaled business as usual. Apple was criticized for extractive pricing policies, offshoring jobs, exploiting its retail staff, abrogating responsibility for factory conditions, colluding to depress wages via illicit noncompete agreements in employee recruitment, institutionalized tax evasion, and a lack of environmental stewardship—just to name a few of the violations that seemed to negate the implicit social contract of its own unique logic.

When it comes to genuine economic mutation, there is always a tension between the new features of the form and its mother ship. A combination of old and new is reconfigured in an unprecedented pattern. Occasionally, the elements of a mutation find the right environment in which to be “selected” for propagation. This is when the new form stands a chance of becoming fully institutionalized and establishes its unique migratory path toward the future. But it’s even more likely that potential mutations meet their fate in “transition failure,” drawn back by the gravitational pull of established practices.

Why did Google's Gmail, launched in 2004, scan private correspondence to generate advertising? As soon as the first Gmail user saw the first ad targeted to the content of her private correspondence, public reaction was swift. Many were repelled and outraged; others were confused. As Google chronicler Steven Levy put it, "By serving ads related to content, Google seemed almost to be reveling in the fact that users' privacy was at the mercy of the policies and trustworthiness of the company that owned the servers. And since those ads made profits, Google was making it clear that it would exploit the situation."

In 2007 Facebook launched Beacon, touting it as "a new way to socially distribute information." Beacon enabled Facebook advertisers to track users across the internet, disclosing users' purchases to their personal networks without permission. Most people were outraged by the company's audacity, both in tracking them online and in usurping their ability to control the disclosure of their own facts. Facebook founder Mark Zuckerberg shut the program down under duress, but by 2010 he declared that privacy was no longer a social norm and then congratulated himself for relaxing the company's "privacy policies" to reflect this self-interested assertion of a new social condition.

I purchased a diamond engagement ring set from overstock in preparation for a New Year's surprise for my girlfriend.… Within hours, I received a shocking call from one of my best friends of surprise and "congratulations" for getting engaged.(!!!) Imagine my horror when I learned that overstock had published the details of my purchase (including a link to the item and its price) on my public Facebook newsfeed, as well as notifications to all of my friends. ALL OF MY FRIENDS, including my girlfriend, and all of her friends, etc.… ALL OF THIS WAS WITHOUT MY CONSENT OR KNOWLEDGE. I am totally distressed that my surprise was ruined, and what was meant to be something special and a lifetime memory for my girlfriend and I was destroyed by a totally underhanded and infuriating privacy invasion. I want to wring the neck of the folks at overstock and facebook who thought that this was a good idea. It sets a terrible precedent on the net, and I feel that it ruined a part of my life.

Among the many violations of advocacy expectations, ubiquitous “terms-of-service agreements” were among the most pernicious. Legal experts call these “contracts of adhesion” because they impose take-it-or-leave-it conditions on users that stick to them whether they like it or not. Online “contracts” such as terms-of-service or terms-of-use agreements are also referred to as “click-wrap” because, as a great deal of research shows, most people get wrapped in these oppressive contract terms by simply clicking on the box that says “I agree” without ever reading the agreement.

Scholars point out that these digital documents are excessively long and complex in part to discourage users from actually reading the terms, safe in the knowledge that most courts have upheld the legitimacy of click-wrap agreements despite the obvious lack of meaningful consent. US Supreme Court Chief Justice John Roberts admitted that he “doesn’t read the computer fine print.” Adding insult to injury, terms of service can be altered unilaterally by the firm at any time, without specific user knowledge or consent, and the terms typically implicate other companies (partners, suppliers, marketers, advertising intermediaries, etc.) without stating or accepting responsibility for their terms of service. These “contracts” impose an unwinnable infinite regress upon the user that law professor Nancy Kim describes as “sadistic.”

Legal scholar Margaret Radin observes the Alice-in-Wonderland quality of such “contracts.” Indeed, the sacred notions of “agreement” and “promise” so critical to the evolution of the institution of contract since Roman times have devolved to a “talismanic” signal “merely indicating that the firm deploying the boilerplate wants the recipient to be bound.” Radin calls this “private eminent domain,” a unilateral seizure of rights without consent.

Kim points out that paper documents once imposed natural restraints on contracting behavior simply by virtue of their cost to produce, distribute, and archive. Paper contracts require a physical signature, limiting the burden a firm is likely to impose on a customer by requiring her to read multiple pages of fine print. Digital terms, in contrast, are “weightless.” They can be expanded, reproduced, distributed, and archived at no additional cost. Once firms understood that the courts were disposed to validate their click-wrap and browse-wrap agreements, there was nothing to stop them from expanding the reach of these degraded contracts “to extract from consumers additional benefits unrelated to the transaction.” … In 2008 two Carnegie Mellon professors calculated that a reasonable reading of all the privacy policies that one encounters in a year would require 76 full workdays at a national opportunity cost of $781 billion. The numbers are much higher today. Still, most users remain unaware of these “rapacious” terms that, as Kim puts it, allow firms “to acquire rights without bargaining and to stealthily establish and embed practices before users, and regulators, realize what has happened.”

Although the Apple miracle contained the seeds of economic reformation, it was poorly understood: a mystery even to itself. Long before the death of its legendary founder, Steve Jobs, its frequent abuses of user expectations raised questions about how well the corporation understood the deep structure and historic potential of its own creations. The dramatic success of Apple’s iPod and iTunes instilled internet users with a sense of optimism toward the new digital capitalism, but Apple never did seize the reins on developing the consistent, comprehensive social and institutional processes that would have elevated the iPod’s promise to an explicit market form, as Henry Ford and Alfred Sloan had once done.

At first, it had seemed that the new internet companies had simply failed to grasp the moral, social, and institutional requirements of their own economic logic. But with each corporate transgression, it became more difficult to ignore the possibility that the pattern of violations signaled a feature, not a bug.

The credo of digital innovation quickly turned to the language of disruption and an obsession with speed, its campaigns conducted under the flag of “creative destruction.” That famous, fateful phrase coined by evolutionary economist Joseph Schumpeter was seized upon as a way to legitimate what Silicon Valley euphemistically calls “permissionless innovation.” Destruction rhetoric promoted what I think of as a “boys and their toys” theory of history, as if the winning hand in capitalism is about blowing things up with new technologies. Schumpeter’s analysis was, in fact, far more nuanced and complex than modern destruction rhetoric suggests.

Although Schumpeter regarded capitalism as an “evolutionary” process, he also considered that relatively few of its continuous innovations actually rise to the level of evolutionary significance. These rare events are what he called “mutations.” These are enduring, sustainable, qualitative shifts in the logic, understanding, and practice of capitalist accumulation, not random, temporary, or opportunistic reactions to circumstances. Schumpeter insisted that this evolutionary mechanism is triggered by new consumer needs, and alignment with those needs is the discipline that drives sustainable mutation: “The capitalist process, not by coincidence but by virtue of its mechanism, progressively raises the standard of life of the masses.”

If a mutation is to be reliably sustained, its new aims and practices must be translated into new institutional forms: “The fundamental impulse that sets and keeps the capitalist engine in motion comes from the new consumers’ goods, the new methods of production or transportation, the new markets, the new forms of industrial organization that capitalist enterprise creates.” Note that Schumpeter says “creates,” not “destroys.” As an example of mutation, Schumpeter cites “the stages of organizational development from the craft shop to the factory to a complex corporation like U.S. Steel.…”

The significance of a “mutation” in Schumpeter’s reckoning implies a high threshold, one that is crossed in time through the serious work of inventing new institutional forms embedded in the new needs of new people. Relatively little destruction is creative, especially in the absence of a robust double movement. This is illustrated in Schumpeter’s example of US Steel, founded by some of the Gilded Age’s most notorious “robber barons,” including Andrew Carnegie and J. P. Morgan. Under pressure from an increasingly insistent double movement, US Steel eventually institutionalized fair labor practices through unions and collective bargaining as well as internal labor markets, career ladders, professional hierarchies, employment security, training, and development, all while implementing its technological advances in mass production.

This new market form is a unique logic of accumulation in which surveillance is a foundational mechanism in the transformation of investment into profit.

Surveillance capitalism commandeered the wonders of the digital world to meet our needs for effective life, promising the magic of unlimited information and a thousand ways to anticipate our needs and ease the complexities of our harried lives. We welcomed it into our hearts and homes with our own rituals of hospitality. As we shall explore in detail throughout the coming chapters, thanks to surveillance capitalism the resources for effective life that we seek in the digital realm now come encumbered with a new breed of menace. Under this new regime, the precise moment at which our needs are met is also the precise moment at which our lives are plundered for behavioral data, and all for the sake of others' gain. The result is a perverse amalgam of empowerment inextricably layered with diminishment. In the absence of a decisive societal response that constrains or outlaws this logic of accumulation, surveillance capitalism appears poised to become the dominant form of capitalism in our time.

For now we can recognize that over the centuries we have imagined threat in the form of state power. This left us wholly unprepared to defend ourselves from new companies with imaginative names run by young geniuses that seemed able to provide us with exactly what we yearn for at little or no cost.

We may yet see the founding of a new synthesis for a third modernity in which a genuine inversion and its social compact are institutionalized as principles of a new rational digital capitalism aligned with a society of individuals and supported by democratic institutions.

The new harms we face entail challenges to the sanctity of the individual, and chief among these challenges I count the elemental rights that bear on individual sovereignty, including the right to the future tense and the right to sanctuary. Each of these rights invokes claims to individual agency and personal autonomy as essential prerequisites to freedom of will and to the very concept of democratic order.

In 2011, however, neither the Montes nor their customers and friends would take their August holidays. The economic crisis had ripped through the country like the black plague, shrinking consumption and driving unemployment to 21 percent, the highest in the EU, and to an astonishing 46 percent among people under twenty-four years old. In Catalonia, the region that includes Barcelona, 18 percent of its 7.5 million people had fallen below the poverty line. In the summer of 2011, few could afford the simple pleasure of an August spent by the sea or in the mountains.

Just three months earlier, Juan Pablo and Maria had made the pilgrimage to Madrid to join thousands of protesters at the Puerta del Sol, where a month-long encampment established Los Indignados, the 15M, as the new voice of a people who had finally been pushed to the breaking point by the economics of contempt. All that was left to say was “Ya. No mas!” Enough already! The convergence of so many citizens in Madrid led to a wave of protests across the nation, and eventually those protests would give way to new political parties, including Podemos.

“The bankers may not know it,” Fito Montes reflected, “but the future will need the past. It will need these marble floors and the sweet taste of my gypsy cakes. They treat us like figures in a ledger, like they are reading the number of casualties in a plane crash. They believe the future belongs only to them. But we each have our story. We each have our life. It is up to us to proclaim our right to the future. The future is our home too.”

The Spanish Data Protection Agency had chosen to champion the claims of ninety ordinary citizens who, like the Montes family, were determined to preserve inherited meaning for a world bent on change at the speed of light. In the name of "the right to be forgotten," the Spaniards had stepped into the bullring brandishing red capes, resolved to master the fiercest bull of all: Google, the juggernaut of surveillance capitalism. When the agency ordered the internet firm to stop indexing the contested links of these ninety individuals, the bull received one of its first and most significant blows.

In the assertion of a right to be forgotten, the complexity of human existence, with its thousand million shades of gray, was pitted against surveillance capitalism's economic imperatives that produced the relentless drive to extract and retain information. It was there, in Spain, that the right to the future tense was on the move, insisting that the operations of surveillance capitalism and its digital architecture are not, never were, and never would be inevitable. Instead, the opposition asserted that even Google's capitalism was made by humans to be unmade and remade by democratic processes, not commercial decree. Google's was not to be the last word on the human or the digital future.

One had been terrorized by her former husband and didn’t want him to find her address online. Informational privacy was essential to her peace of mind and her physical safety. A middle-aged woman was embarrassed by an old arrest from her days as a university student. Informational privacy was essential to her identity and sense of dignity. One was an attorney, Mario Costeja González, who years earlier had suffered the foreclosure of his home. Although the matter had long been resolved, a Google search of his name continued to deliver links to the foreclosure notice, which, he argued, damaged his reputation. While the Spanish Data Protection Agency rejected the idea of requiring newspapers and other originating sites to remove legitimate information—such information, they reasoned, would exist somewhere under any circumstances—it endorsed the notion that Google had responsibility and should be held to account. After all, Google had unilaterally undertaken to change the rules of the information life cycle when it decided to crawl, index, and make accessible personal details across the world wide web without asking anyone’s permission.

Google’s mission to “organize the world’s information and make it universally accessible and useful”—starting with the web—changed all of our lives. There have been enormous benefits, to be sure. But for individuals it has meant that information that would normally age and be forgotten now remains forever young, highlighted in the foreground of each person’s digital identity. The Spanish Data Protection Agency recognized that not all information is worthy of immortality. Some information should be forgotten because that is only human.

The Court of Justice’s decision, so often reduced to the legal and technical considerations related to the deletion or de-linking of personal data, was in fact a key inflection point at which democracy began to claw back rights to the future tense from the powerful forces of a new surveillance capitalism determined to claim unilateral authority over the digital future. Instead, the court’s analysis claimed the future for the human way, rejecting the inevitability of Google’s search-engine technology and recognizing instead that search results are the contingent products of the specific economic interests that drive the action from within the belly of the machine: "The operator of a search engine is liable to affect significantly the fundamental rights to privacy and to the protection of personal data. In the light of the potential seriousness of the interference" with those interests, "it cannot be justified by merely the economic interest which the operator of such an engine has in that processing."

In reasserting the right to be forgotten, the court declared that decisive authority over the digital future rests with the people, their laws, and their democratic institutions. It affirmed that individuals and democratic societies can fight for their rights to the future tense and can win, even in the face of a great private power.

Page defended Google’s unprecedented information power with an extraordinary statement suggesting that people should trust Google more than democratic institutions: “In general, having the data present in companies like Google is better than having it in the government with no due process to get that data, because we obviously care about our reputation. I’m not sure the government cares about that as much.”

In fact, the smart money appeared not to be all that smart. In the year following the EU decision, a national poll of US adults found that 88 percent supported a law similar to the right to be forgotten. That year, Pew Research found that 93 percent of Americans believed that it was important to have control of “who can get information about you.” A series of polls echoed these findings.

The smart money says that US law will never abandon its allegiance to the surveillance capitalists over the people. But the next decades may once again prove that the smart money can be wrong.

The point for us is that every successful vaccine begins with a close understanding of the enemy disease. The mental models, vocabularies, and tools distilled from past catastrophes obstruct progress. We smell smoke and rush to close doors to rooms that are already fated to vanish. The result is like hurling snowballs at a smooth marble wall only to watch them slide down its facade, leaving nothing but a wet smear: a fine paid here, an operational detour there, a new encryption package there.

What is crucial now is that we identify this new form of capitalism on its own terms and in its own words. This pursuit necessarily returns us to Silicon Valley, where things move so fast that few people know what just happened. It is the habitat for progress “at the speed of dreams,” as one Google engineer vividly describes it. My aim here is to slow down the action in order to enlarge the space for such debate and unmask the tendencies of these new creations as they amplify inequality, intensify social hierarchy, exacerbate exclusion, usurp rights, and strip personal life of whatever it is that makes it personal for you or for me.

Google is to surveillance capitalism what the Ford Motor Company and General Motors were to mass-production–based managerial capitalism. New economic logics and their commercial models are discovered by people in a time and place and then perfected through trial and error. In our time Google became the pioneer, discoverer, elaborator, experimenter, lead practitioner, role model, and diffusion hub of surveillance capitalism. GM and Ford’s iconic status as pioneers of twentieth-century capitalism made them enduring objects of scholarly research and public fascination because the lessons they had to teach resonated far beyond the individual companies. Google’s practices deserve the same kind of examination, not merely as a critique of a single company but rather as the starting point for the codification of a powerful new form of capitalism.

Google is a notoriously secretive company, and one is hard-pressed to imagine a Drucker equivalent freely roaming the scene and scribbling in the hallways. Its executives carefully craft their messages of digital evangelism in books and blog posts, but its operations are not easily accessible to outside researchers or journalists. In 2016 a lawsuit brought against the company by a product manager alleged an internal spying program in which employees are expected to identify coworkers who violate the firm’s confidentiality agreement: a broad prohibition against divulging anything about the company to anyone. The closest thing we have to a Buck Weaver or James Couzens codifying Google’s practices and objectives is the company’s longtime chief economist, Hal Varian, who aids the cause of understanding with scholarly articles that explore important themes. Varian has been described as “the Adam Smith of the discipline of Googlenomics” and the “godfather” of its advertising model. It is in Varian’s work that we find hidden-in-plain-sight important clues to the logic of surveillance capitalism and its claims to power.

Google’s invention of targeted advertising paved the way to financial success, but it also laid the cornerstone of a more far-reaching development: the discovery and elaboration of surveillance capitalism. Its business is characterized as an advertising model, and much has been written about Google’s automated auction methods and other aspects of its inventions in the field of online advertising. With so much verbiage, these developments are both over-described and under-theorized. Our aim in this chapter and those that follow in Part I is to reveal the "laws of motion" that drive surveillance competition, and in order to do this we begin by looking freshly at the point of origin, when the foundational mechanisms of surveillance capitalism were first discovered.

Any confrontation with the unprecedented requires new language, and I introduce new terms when existing language fails to capture a new phenomenon. Sometimes, however, I intentionally repurpose familiar language because I want to stress certain continuities in the function of an element or process. This is the case with "laws of motion," borrowed from Newton's laws of inertia, force, and equal and opposite reactions.

Over the years historians have adopted this term to describe the "laws" of industrial capitalism. For example, economic historian Ellen Meiksins Wood documents the origins of capitalism in the changing relations between English property owners and tenant farmers, as the owners began to favor productivity over coercion: "The new historical dynamic allows us to speak of 'agrarian capitalism' in early modern England, a social form with distinctive 'laws of motion' that would eventually give rise to capitalism in its mature, industrial form."

The critical factor in the divergence of capitalism from all other forms of "commercial society" was the development of certain social property relations that generated market imperatives and capitalist "laws of motion"… competitive production and profit-maximization, the compulsion to reinvest surpluses, and the relentless need to improve labour-productivity associated with capitalism.… Those laws of motion required vast social transformations and upheavals to set them in train. They required a transformation in the human metabolism with nature, in the provision of life's basic necessities.

For example, in addition to key words, each Google search query produces a wake of collateral data such as the number and pattern of search terms, how a query is phrased, spelling, punctuation, dwell times, click patterns, and location.

His work with these data logs persuaded him that detailed stories about each user—thoughts, feelings, interests—could be constructed from the wake of unstructured signals that trailed every online action. These data, he concluded, actually provided a “broad sensor of human behavior” and could be put to immediate use in realizing cofounder Larry Page’s dream of Search as a comprehensive artificial intelligence.

Other search engines in the 1990s had the chance to do the same, but did not pursue it. Around 2000 Yahoo! saw the potential, but nothing came of the idea. It was Google that recognized the gold dust in the detritus of its interactions with its users and took the trouble to collect it up.… Google exploits information that is a by-product of user interactions, or data exhaust, which is automatically recycled to improve the service or create an entirely new product.

What had been regarded as waste material—“data exhaust” spewed into Google's servers during the combustive action of Search—was quickly reimagined as a critical element in the transformation of Google's search engine into a reflexive process of continuous learning and improvement.

At that early stage of Google's development, the feedback loops involved in improving its Search functions produced a balance of power: Search needed people to learn from, and people needed Search to learn from. This symbiosis enabled Google's algorithms to learn and produce ever-more relevant and comprehensive search results. More queries meant more learning; more learning produced more relevance. More relevance meant more searches and more users. By the time the young company held its first press conference in 1999, to announce a $25 million equity investment from two of the most revered Silicon Valley venture capital firms, Sequoia Capital and Kleiner Perkins, Google Search was already fielding seven million requests each day.

During this early period, behavioral data were put to work entirely on the user's behalf. User data provided value at no cost, and that value was reinvested in the user experience in the form of improved services: enhancements that were also offered at no cost to users. Users provided the raw material in the form of behavioral data, and those data were harvested to improve speed, accuracy, and relevance and to help build ancillary products such as translation. I call this the behavioral value reinvestment cycle, in which all behavioral data are reinvested in the improvement of the product or service (see Figure 1).

The cycle emulates the logic of the iPod; it worked beautifully at Google but with one critical difference: the absence of a sustainable market transaction. In the case of the iPod, the cycle was triggered by the purchase of a high-margin physical product. Subsequent reciprocities improved the iPod product and led to increased sales. Customers were the subjects of the commercial process, which promised alignment with their “what I want, when I want, where I want” demands. At Google, the cycle was similarly oriented toward the individual as its subject, but without a physical product to sell, it floated outside the marketplace, an interaction with “users” rather than a market transaction with customers.

This helps to explain why it is inaccurate to think of Google’s users as its customers: there is no economic exchange, no price, and no profit. Nor do users function in the role of workers. When a capitalist hires workers and provides them with wages and means of production, the products that they produce belong to the capitalist to sell at a profit. Not so here. Users are not paid for their labor, nor do they operate the means of production, as we’ll discuss in more depth later in this chapter. Finally, people often say that the user is the “product.” This is also misleading, and it is a point that we will revisit more than once. For now let’s say that users are not products, but rather we are the sources of raw-material supply. As we shall see, surveillance capitalism’s unusual products manage to be derived from our behavior while remaining indifferent to our behavior. Its products are about predicting us, without actually caring what we do or what is done to us.

To summarize, at this early stage of Google’s development, whatever Search users inadvertently gave up that was of value to the company they also used up in the form of improved services. In this reinvestment cycle, serving users with amazing Search results “consumed” all the value that users created when they provided extra behavioral data. The fact that users needed Search about as much as Search needed users created a balance of power between Google and its populations. People were treated as ends in themselves, the subjects of a nonmarket, self-contained cycle that was perfectly aligned with Google’s stated mission “to organize the world’s information, making it universally accessible and useful.”

By 1999, despite the splendor of Google’s new world of searchable web pages, its growing computer science capabilities, and its glamorous venture backers, there was no reliable way to turn investors’ money into revenue. The behavioral value reinvestment cycle produced a very cool search function, but it was not yet capitalism. The balance of power made it financially risky and possibly counterproductive to charge users a fee for search services. Selling search results would also have set a dangerous precedent for the firm, assigning a price to indexed information that Google’s web crawler had already taken from others without payment. Without a device like Apple’s iPod or its digital songs, there were no margins, no surplus, nothing left over to sell and turn into revenue.

Google had relegated advertising to steerage class: its AdWords team consisted of seven people, most of whom shared the founders’ general antipathy toward ads. The tone had been set in Sergey Brin and Larry Page’s milestone paper that unveiled their search engine conception, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” presented at the 1998 World Wide Web Conference: “We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. This type of bias is very difficult to detect but could still have a significant effect on the market… we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.”

Google’s first revenues depended instead on exclusive licensing deals to provide web services to portals such as Yahoo! and Japan’s BIGLOBE. It also generated modest revenue from sponsored ads linked to search query keywords. There were other models for consideration. Rival search engines such as Overture, used exclusively by the then-giant portal AOL, or Inktomi, the search engine adopted by Microsoft, collected revenues from the sites whose pages they indexed. Overture was also successful in attracting online ads with its policy of allowing advertisers to pay for high-ranking search listings, the very format that Brin and Page scorned.

This changed abruptly in April 2000, when the legendary dot-com economy began its steep plunge into recession, and Silicon Valley’s Garden of Eden unexpectedly became the epicenter of a financial earthquake.

Startups with outsized valuations just months earlier were suddenly forced to shutter. Prominent articles such as “Doom Stalks the Dotcoms” noted that the stock prices of Wall Street’s most-revered internet “high flyers” were “down for the count,” with many of them trading below their initial offering price: “With many dotcoms declining, neither venture capitalists nor Wall Street is eager to give them a dime.…” The news brimmed with descriptions of shell-shocked investors. The week of April 10 saw the worst decline in the history of the NASDAQ, where many internet companies had gone public, and there was a growing consensus that the “game” had irreversibly changed.

Many Google investors began to express doubts about the company’s prospects, and some threatened to withdraw support. Pressure for profit mounted sharply, despite the fact that Google Search was widely considered the best of all the search engines, traffic to its website was surging, and a thousand résumés flooded the firm’s Mountain View office each day. Page and Brin were seen to be moving too slowly, and their top venture capitalists, John Doerr from Kleiner Perkins and Michael Moritz from Sequoia, were frustrated.

Studies of pre-bubble investment patterns showed a "big-score" mentality in which bad results tended to stimulate increased investing as funders chased the belief that some young company would suddenly discover the elusive business model destined to turn all their bets into rivers of gold. Startup mortality rates in Silicon Valley outstripped those for other venture capital centers such as Boston and Washington, DC, with impatient money producing a few big wins and many losses. Impatient money is also reflected in the size of Silicon Valley startups, which during this period were significantly smaller than in other regions, employing an average of 68 employees as compared to an average of 112 in the rest of the country. This reflects an interest in quick returns without spending much time on growing a business or deepening its talent base, let alone developing the institutional capabilities that Joseph Schumpeter would have advised.

The declaration of a state of exception functions in politics as cover for the suspension of the rule of law and the introduction of new executive powers justified by crisis. At Google in late 2000, it became a rationale for annulling the reciprocal relationship that existed between Google and its users, steeling the founders to abandon their passionate and public opposition to advertising. As a specific response to investors’ anxiety, the founders tasked the tiny AdWords team with the objective of looking for ways to make more money. Page demanded that the whole process be simplified for advertisers. In this new approach, he insisted that advertisers "shouldn't even get involved with choosing keywords—Google would choose them."

Operationally, this meant that Google would turn its own growing cache of behavioral data and its computational power and expertise toward the single task of matching ads with queries. New rhetoric took hold to legitimate this unusual move. If there was to be advertising, then it had to be “relevant” to users. Ads would no longer be linked to keywords in a search query, but rather a particular ad would be “targeted” to a particular individual. Securing this holy grail of advertising would ensure relevance to users and value to advertisers.

The raw materials that had been solely used to improve the quality of search results would now also be put to use in the service of targeting advertising to individual users. Some data would continue to be applied to service improvement, but the growing stores of collateral signals would be repurposed to improve the profitability of ads for both Google and its advertisers. These behavioral data available for uses beyond service improvement constituted a surplus, and it was on the strength of this behavioral surplus that the young company would find its way to the “sustained and exponential profits” that would be necessary for survival. Thanks to a perceived emergency, a new mutation began to gather form and quietly slip its moorings in the implicit advocacy-oriented social contract of the firm’s original relationship with users.

Google’s declared state of exception was the backdrop for 2002, the watershed year during which surveillance capitalism took root. The firm’s appreciation of behavioral surplus crossed another threshold that April, when the data logs team arrived at their offices one morning to find that a peculiar phrase had surged to the top of the search queries: “Carol Brady’s maiden name.” Why the sudden interest in a 1970s television character? It was data scientist and logs team member Amit Patel who recounted the event to the New York Times, noting, “You can’t interpret it unless you know what else is going on in the world.”

The team went to work to solve the puzzle. First, they discerned that the pattern of queries had produced five separate spikes, each beginning at forty-eight minutes after the hour. Then they learned that the query pattern occurred during the airing of the popular TV show Who Wants to Be a Millionaire? The spikes reflected the successive time zones during which the show aired, ending in Hawaii. In each time zone, the show’s host posed the question of Carol Brady’s maiden name, and in each zone the queries immediately flooded into Google’s servers.

As the New York Times reported, “The precision of the Carol Brady data was eye-opening for some.” Even Brin was stunned by the clarity of Search’s predictive power, revealing events and trends before they “hit the radar” of traditional media. As he told the Times, “It was like trying an electron microscope for the first time. It was like a moment-by-moment barometer.” Google executives were described by the Times as reluctant to share their thoughts about how their massive stores of query data might be commercialized. “There is tremendous opportunity with this data,” one executive confided.

Schmidt later boasted that as a result of their close quarters over the course of several months, he had instant access to better revenue figures than did his own financial planners. We do not know (and may never know) what other insights Schmidt might have gleaned from Patel about the predictive power of Google's behavioral data stores, but there is no doubt that a deeper grasp of the predictive power of data quickly shaped Google's specific response to financial emergency, triggering the crucial mutation that ultimately turned AdWords, Google, the internet, and the very nature of information capitalism toward an astonishingly lucrative surveillance project.

Google's earliest ads had been considered more effective than most online advertising at the time because they were linked to search queries and Google could track when users actually clicked on an ad, known as the "click-through" rate. Despite this, advertisers were billed in the conventional manner according to how many people viewed an ad. As Search expanded, Google created the self-service system called AdWords, in which a search that used the advertiser's keyword would include that advertiser's text box and a link to its landing page. Ad pricing depended upon the ad's position on the search results page.

Rival search startup Overture had developed an online auction system for web page placement that allowed it to scale online advertising targeted to keywords. Google would produce a transformational enhancement to that model, one that was destined to alter the course of information capitalism. As a Bloomberg journalist explained in 2006, "Google maximizes the revenue it gets from that precious real estate by giving its best position to the advertiser who is likely to pay Google the most in total, based on the price per click multiplied by Google's estimate of the likelihood that someone will actually click on the ad." That pivotal multiplier was the result of Google's advanced computational capabilities trained on its most significant and secret discovery: behavioral surplus. From this point forward, the combination of ever-increasing machine intelligence and ever-more-vast supplies of behavioral surplus would become the foundation of an unprecedented logic of accumulation. Google's reinvestment priorities would shift from merely improving its user offerings to inventing and institutionalizing the most far-reaching and technologically advanced raw-material supply operations that the world had ever seen. Henceforth, revenues and growth would depend upon more behavioral surplus.

In other words, Google would no longer mine behavioral data strictly to improve service for users but rather to read users’ minds for the purposes of matching ads to their interests, as those interests are deduced from the collateral traces of online behavior. With Google’s unique access to behavioral data, it would now be possible to know what a particular individual in a particular time and place was thinking, feeling, and doing. That this no longer seems astonishing to us, or perhaps even worthy of note, is evidence of the profound psychic numbing that has inured us to a bold and unprecedented shift in capitalist methods.

The techniques described in the patent meant that each time a user queries Google's search engine, the system simultaneously presents a specific configuration of a particular ad, all in the fraction of a moment that it takes to fulfill the search query. The data used to perform this instant translation from query to ad, a predictive analysis that was dubbed "matching," went far beyond the mere denotation of search terms. New data sets were compiled that would dramatically enhance the accuracy of these predictions. These data sets were referred to as "user profile information" or "UPI." These new data meant that there would be no more guesswork and far less waste in the advertising budget. Mathematical certainty would replace all of that.

Of critical importance to our story is the scientists' observation that the most challenging sources of friction here are social, not technical. Friction arises when users intentionally fail to provide information for no other reason than that they choose not to. "Unfortunately, user profile information is not always available," the scientists warn. Users do not always "voluntarily" provide information, or "the user profile may be incomplete… and hence not comprehensive, because of privacy considerations, etc."

A clear aim of the patent is to assure its audience that Google scientists will not be deterred by users' exercise of decision rights over their personal information, despite the fact that such rights were an inherent feature of the original social contract between the company and its users. Even when users do provide UPI, the inventors caution, "it may be intentionally or unintentionally inaccurate, it may become stale.… UPI for a user… can be determined (or updated or extended) even when no explicit information is given to the system.… An initial UPI may include some expressly entered UPI information, though it doesn't need to."

The scientists thus make clear that they are willing—and that their inventions are able—to overcome the friction entailed in users' decision rights. Google's proprietary methods enable it to surveil, capture, expand, construct, and claim behavioral surplus, including data that users intentionally choose not to share. Recalcitrant users will not be obstacles to data expropriation. No moral, legal, or social constraints will stand in the way of finding, claiming, and analyzing others' behavior for commercial purposes.

A one-way mirror embodies the specific social relations of surveillance based on asymmetries of knowledge and power. The new mode of accumulation invented at Google would derive, above all, from the firm’s willingness and ability to impose these social relations on its users. Its willingness was mobilized by what the founders came to regard as a state of exception; its ability came from its actual success in leveraging privileged access to behavioral surplus in order to predict the behavior of individuals now, soon, and later. The predictive insights thus acquired would constitute a world-historic competitive advantage in a new marketplace where low-risk bets about the behavior of individuals are valued, bought, and sold.

Behavioral data, whose value had previously been “used up” on improving the quality of Search for users, now became the pivotal—and exclusive to Google—raw material for the construction of a dynamic online advertising marketplace. Google would now secure more behavioral data than it needed to serve its users. That surplus, a behavioral surplus, was the game-changing, zero-cost asset that was diverted from service improvement toward a genuine and highly lucrative market exchange.

These capabilities were and remain inscrutable to all but an exclusive data priesthood among whom Google is the übermensch. They operate in obscurity, indifferent to social norms or individual claims to self-determining decision rights. These moves established the foundational mechanisms of surveillance capitalism.

The state of exception declared by Google’s founders transformed the youthful Dr. Jekyll into a ruthless, muscular Mr. Hyde determined to hunt his prey anywhere, anytime, irrespective of others’ self-determining aims. The new Google ignored claims to self-determination and acknowledged no a priori limits on what it could find and take. It dismissed the moral and legal content of individual decision rights and recast the situation as one of technological opportunism and unilateral power. This new Google assures its actual customers that it will do whatever it takes to transform the natural obscurity of human desire into scientific fact. This Google is the superpower that establishes its own values and pursues its own purposes above and beyond the social contracts to which others are bound.

The first new pricing metric was based on “click-through rates,” or how many times a user clicks on an ad through to the advertiser’s web page, rather than pricing based on the number of views that an ad receives. The click-through was interpreted as a signal of relevance and therefore a measure of successful targeting, operational results that derive from and reflect the value of behavioral surplus.

This new pricing discipline established an ever-escalating incentive to increase behavioral surplus in order to continuously upgrade the effectiveness of predictions. Better predictions lead directly to more click-throughs and thus to revenue. Google learned new ways to conduct automated auctions for ad targeting that allowed the new invention to scale quickly, accommodating hundreds of thousands of advertisers and billions (later it would be trillions) of auctions simultaneously. Google's unique auction methods and capabilities earned a great deal of attention, which distracted observers from reflecting on exactly what was being auctioned: derivatives of behavioral surplus. Click-through metrics institutionalized "customer" demand for these prediction products and thus established the central importance of economies of scale in surplus supply operations. Surplus capture would have to become automatic and ubiquitous if the new logic was to succeed, as measured by the successful trading of behavioral futures.

Another key metric called the "quality score" helped determine the price of an ad and its specific position on the page, in addition to advertisers' own auction bids. The quality score was determined in part by click-through rates and in part by the firm's analyses of behavioral surplus. "The clickthrough rate needed to be a predictive thing," one top executive insisted, and that would require "all the information we had about the query right then." It would take enormous computing power and leading-edge algorithmic programs to produce powerful predictions of user behavior that became the criteria for estimating the relevance of an ad. Ads that scored high would sell at a lower price than those that scored poorly. Google's customers, its advertisers, complained that the quality score was a black box, and Google was determined to keep it so. Nonetheless, when customers followed its disciplines and produced high-scoring ads, their click-through rates soared.

AdWords quickly became so successful that it inspired significant expansion of the surveillance logic. Advertisers demanded more clicks. The answer was to extend the model beyond Google's search pages and convert the entire internet into a canvas for Google's targeted ads. This required turning Google's newfound skills at "data extraction and analysis," as Hal Varian put it, toward the content of any web page or user action by employing Google's rapidly expanding semantic analysis and artificial intelligence capabilities to efficiently "squeeze" meaning from them. Only then could Google accurately assess the content of a page and how users interact with that content. This "content-targeted advertising" based on Google's patented methods was eventually named AdSense. By 2004, AdSense had achieved a run rate of a million dollars per day, and by 2010, it produced annual revenues of more than $10 billion.

With click-through rates as the measure of relevance accomplished, behavioral surplus was institutionalized as the cornerstone of a new kind of commerce that depended upon online surveillance at scale. Insiders referred to Google’s new science of behavioral prediction as the “physics of clicks.” Mastery of this new domain required a specialized breed of click physicists who would secure Google’s preeminence within the nascent priesthood of behavioral prediction.

Page was haunted by the example of the brilliant but impoverished scientist Nikola Tesla, who died without ever benefiting financially from his inventions. “You need to do more than just invent things,” Page reflected. … Exceptional threats to their financial and social status appear to have awakened a survival instinct in Page and Brin that required exceptional adaptive measures. The Google founders’ response to the fear that stalked their community effectively declared a “state of exception” in which it was judged necessary to suspend the values and principles that had guided Google’s founding and early practices.

The first 12 months of Google were not a cakewalk, because the company didn’t start off in the business that it eventually tapped. At first it went in a different direction, which was selling its technology—selling licenses for its search engines to larger internet properties and to corporations.… Cash was going out of the window at a feral rate during the first six, seven months. And then, very ingeniously, Larry… and Sergey… and others fastened on a model that they had seen this other company, Overture, develop, which was ranked advertisements. They saw how it could be improved and enhanced and made it their own, and that transformed the business.

By 1996, the function of cookies had become a contested public policy issue. Federal Trade Commission workshops in 1996 and 1997 discussed proposals that would assign control of all personal information to users by default with a simple automated protocol. Advertisers bitterly contested this scheme, collaborating instead to avert government regulation by forming a “self-regulating” association known as the Network Advertising Initiative. Still, in June 2000 the Clinton administration banned cookies from all federal websites, and by April 2001, three bills before Congress included provisions to regulate cookies.

Google brought new life to these practices. As had occurred at Ford a century earlier, the company’s engineers and scientists were the first to conduct the entire commercial surveillance symphony, integrating a wide range of mechanisms from cookies to proprietary analytics and algorithmic software capabilities in a sweeping new logic that enshrined surveillance and the unilateral expropriation of behavioral data as the basis for a new market form. The impact of this invention was just as dramatic as Ford’s. In 2001, as Google’s new systems to exploit its discovery of behavioral surplus were being tested, net revenues jumped to $86 million (more than a 400 percent increase over 2000), and the company turned its first profit. By 2002, the cash began to flow and has never stopped, definitive evidence that behavioral surplus combined with Google’s proprietary analytics were sending arrows to their marks. Revenues leapt to $347 million in 2002, then $1.5 billion in 2003, and $3.5 billion in 2004, the year the company went public. The discovery of behavioral surplus had produced a stunning 3,590 percent increase in revenue in less than four years.

It is important to note the vital differences for capitalism in these two moments of originality at Ford and Google. Ford’s inventions revolutionized production. Google’s inventions revolutionized extraction and established surveillance capitalism’s first economic imperative: the extraction imperative. The extraction imperative meant that raw-material supplies must be procured at an ever-expanding scale. Industrial capitalism had demanded economies of scale in production in order to achieve high throughput combined with low unit cost. In contrast, surveillance capitalism demands economies of scale in the extraction of behavioral surplus.

The scale of surplus expropriation that was possible at Google would soon eliminate all serious competitors to its core search business as the windfall earnings from leveraging behavioral surplus were used to continuously draw more users into its net, thus establishing its de facto monopoly in Search. On the strength of Google's inventions, discoveries, and strategies, it became the mother ship and ideal type of a new economic logic based on fortune-telling and selling—an ancient and eternally lucrative craft that has fed on humanity's confrontation with uncertainty from the beginning of the human story.

Once Google's leadership understood the commercial power of behavioral surplus, Schmidt instituted what he called the “hiding strategy.” Google employees were told not to speak about what the patent had referred to as its “novel methods, apparatus, message formats and/or data structures” or confirm any rumors about flowing cash. Hiding was not a post hoc strategy; it was baked into the cake that would become surveillance capitalism.

As Schmidt told the New York Times, “You need to win, but you are better off winning softly.” The scientific and material complexity that supported the capture and analysis of behavioral surplus also enabled the hiding strategy, an invisibility cloak over the whole operation. “Managing search at our scale is a very serious barrier to entry,” Schmidt warned would-be competitors.

What might the response have been back then if the public were told that Google’s magic derived from its exclusive capabilities in unilateral surveillance of online behavior and its methods specifically designed to override individual decision rights? Google policies had to enforce secrecy in order to protect operations that were designed to be undetectable because they took things from users without asking and employed those unilaterally claimed resources to work in the service of others’ purposes.

This power is a crucial illustration of the difference between “decision rights” and “privacy.” Decision rights confer the power to choose whether to keep something secret or to share it. One can choose the degree of privacy or transparency for each situation. US Supreme Court Justice William O. Douglas articulated this view of privacy in 1967: “Privacy involves the choice of the individual to disclose or to reveal what he believes, what he thinks, what he possesses.…”

Surveillance capitalism lays claim to these decision rights. The typical complaint is that privacy is eroded, but that is misleading. In the larger societal pattern, privacy is not eroded but redistributed, as decision rights over privacy are claimed for surveillance capital. Instead of people having the rights to decide how and what they will disclose, these rights are concentrated within the domain of surveillance capitalism.

These euphemisms operate in exactly the same way as those found on the earliest maps of the North American continent, in which whole regions were labeled with terms such as “heathens,” “infidels,” “idolaters,” “primitives,” “vassals,” and “rebels.” On the strength of those euphemisms, native peoples—their places and claims—were deleted from the invaders’ moral and legal equations, legitimating the acts of taking and breaking that paved the way for church and monarchy.

The intentional work of hiding naked facts in rhetoric, omission, complexity, exclusivity, scale, abusive contracts, design, and euphemism is another factor that helps explain why during Google’s breakthrough to profitability, few noticed the foundational mechanisms of its success and their larger significance. In this picture, commercial surveillance is not merely an unfortunate accident or occasional lapse. It is neither a necessary development of information capitalism nor a necessary product of digital technology or the internet. It is a specifically constructed human choice, an unprecedented market form, an original solution to emergency, and the underlying mechanism through which a new asset class is created on the cheap and converted to revenue. Surveillance is the path to profit that overrides “we the people,” taking our decision rights without permission and even when we say “no.” The discovery of behavioral surplus marks a critical turning point not only in Google’s biography but also in the history of capitalism.

In the years following its IPO in 2004, Google’s spectacular financial breakthrough first astonished and then magnetized the online world. Silicon Valley investors had doubled down on risk for years, in search of that elusive business model that would make it all worthwhile. When Google’s financial results went public, the hunt for mythic treasure was officially over.

The new logic of accumulation spread first to Facebook, which launched the same year that Google went public. CEO Mark Zuckerberg had rejected the strategy of charging users a fee for service as the telephone companies had done in an earlier century. “Our mission is to connect every person in the world. You don’t do that by having a service people pay for,” he insisted. In May 2007 he introduced the Facebook platform, opening up the social network to everyone, not just people with a college e-mail address. Six months later, in November, he launched his big advertising product, Beacon, which would automatically share transactions from partner websites with all of a user’s “friends.” These posts would appear even if the user was not currently logged into Facebook, without the user’s knowledge or an opt-in function. The howls of protest—from users but also from some of Facebook’s partners such as Coca-Cola—forced Zuckerberg to back down swiftly. By December, Beacon became an opt-in program. The twenty-three-year-old CEO understood the potential of surveillance capitalism, but he had not yet mastered Google’s facility in obscuring its operations and intent.

With Google in the lead, surveillance capitalism rapidly became the default model of information capitalism on the web and, as we shall see in coming chapters, gradually drew competitors from every sector. This new market form declares that serving the genuine needs of people is less lucrative, and therefore less important, than selling predictions of their behavior. Google discovered that we are less valuable than others' bets on our future behavior. This changed everything.

Behavioral surplus defines Google's earnings success. In 2016, 89 percent of the revenues of its parent company, Alphabet, derived from Google's targeted advertising programs. The scale of raw-material flows is reflected in Google's domination of the internet, processing over 40,000 search queries every second on average: more than 3.5 billion searches per day and 1.2 trillion searches per year worldwide in 2017.

The logic: Google and other surveillance platforms are sometimes described as "two-sided" or "multi-sided" markets, but the mechanisms of surveillance capitalism suggest something different. Google had discovered a way to translate its nonmarket interactions with users into surplus raw material for the fabrication of products aimed at genuine market transactions with its real customers: advertisers. The translation of behavioral surplus from outside to inside the market finally enabled Google to convert investment into revenue. The corporation thus created out of thin air and at zero marginal cost an asset class of vital raw materials derived from users' nonmarket online behavior. At first those raw materials were simply "found," a by-product of users' search actions. Later those assets were hunted aggressively and procured largely through surveillance. The corporation simultaneously created a new kind of marketplace in which its proprietary "prediction products" manufactured from these raw materials could be bought and sold.

The summary of these developments is that the behavioral surplus upon which Google's fortune rests can be considered as surveillance assets. These assets are critical raw materials in the pursuit of surveillance revenues and their translation into surveillance capital. The entire logic of this capital accumulation is most accurately understood as surveillance capitalism, which is the foundational framework for a surveillance-based economic order: a surveillance economy. The big pattern here is one of subordination and hierarchy, in which earlier reciprocities between the firm and its users are subordinated to the derivative project of our behavioral surplus captured for others' aims. We are no longer the subjects of value realization. Nor are we, as some have insisted, the "product" of Google's sales. Instead, we are the objects from which raw materials are extracted and expropriated for Google's prediction factories. Predictions about our behavior are Google's products, and they are sold to its actual customers but not to us. We are the means to others' ends.

Industrial capitalism transformed nature's raw materials into commodities, and surveillance capitalism lays its claims to the stuff of human nature for a new commodity invention. Now it is human nature that is scraped, torn, and taken for another century's market project. It is obscene to suppose that this harm can be reduced to the obvious fact that users receive no fee for the raw material they supply. That critique is a feat of misdirection that would use a pricing mechanism to institutionalize and therefore legitimate the extraction of human behavior for manufacturing and sale. It ignores the key point that the essence of the exploitation here is the rendering of our lives as behavioral data for the sake of others' improved control of us. The remarkable questions here concern the facts that our lives are rendered as behavioral data in the first place; that ignorance is a condition of this ubiquitous rendition; that decision rights vanish before one even knows that there is a decision to make; that there are consequences to this diminishment of rights that we can neither see nor foretell; that there is no exit, no voice, and no loyalty, only helplessness, resignation, and psychic numbing; and that encryption is the only positive action left to discuss when we sit around the dinner table and casually ponder how to hide from the forces that hide from us.

Google's machine intelligence capabilities feed on behavioral surplus, and the more surplus they consume, the more accurate the prediction products that result. Wired magazine's founding editor, Kevin Kelly, once suggested that although it seems like Google is committed to developing its artificial intelligence capabilities to improve Search, it's more likely that Google develops Search as a means of continuously training its evolving AI capabilities. This is the essence of the machine intelligence project. As the ultimate tapeworm, the machine's intelligence depends upon how much data it eats. In this important respect the new means of production differs fundamentally from the industrial model, in which there is a tension between quantity and quality. Machine intelligence is the synthesis of this tension, for it reaches its full potential for quality only as it approximates totality.

Machine intelligence processes behavioral surplus into prediction products designed to forecast what we will feel, think, and do: now, soon, and later. These methodologies are among Google's most closely guarded secrets. The nature of its products explains why Google repeatedly claims that it does not sell personal data. What? Never! Google executives like to claim their privacy purity because they do not sell their raw material. Instead, the company sells the predictions that only it can fabricate from its world-historic private hoard of behavioral surplus.

The quality and competitiveness of the product are a function of its approximation to certainty: the more predictive the product, the lower the risks for buyers and the greater the volume of sales. Google has learned to be a data-based fortune-teller that replaces intuition with science at scale in order to tell and sell our fortunes for profit to its customers, but not to us.

Surveillance capitalism's profits derive primarily from these behavioral futures markets. Although advertisers were the dominant players in the early history of this new kind of marketplace, there is no reason why such markets are limited to this group. The new prediction systems are only incidentally about ads, in the same way that Ford's new system of mass production was only incidentally about automobiles. In both cases the systems can be applied to many other domains.

Google's former CEO Eric Schmidt credits Hal Varian's early examination of the firm's ad auctions with providing the eureka moment that clarified the true nature of Google's business: "All of a sudden, we realized we were in the auction business." Larry Page is credited with a very different and far more profound answer to the question "What is Google?" Douglas Edwards recounts a 2001 session with the founders that probed their answers to that precise query. It was Page who ruminated, "If we did have a category, it would be personal information.… The places you've seen. Communications.… Sensors are really cheap.… Storage is cheap. Cameras are cheap. People will generate enormous amounts of data.… Everything you've ever heard or seen or experienced will become searchable. Your whole life will be searchable."

In our time of pro-market ideology and practice, this cycle has become so pervasive that we eventually fail to notice its audacity or contest its claims. For example, you can now "purchase" human blood and organs, someone to have your baby or stand in line for you or hold a public parking space, a person to comfort you in your grief, and the right to kill an endangered animal. The list grows longer each day.

Social theorist David Harvey builds on Arendt's insight with his notion of "accumulation by dispossession": "What accumulation by dispossession does is to release a set of assets… at very low (and in some instances zero) cost. Overaccumulated capital can seize hold of such assets and immediately turn them to profitable use." He adds that entrepreneurs who are determined to "join the system" and enjoy "the benefits of capital accumulation" are often the ones who drive this process of dispossession into new, undefended territories.

Page grasped that human experience could be Google’s virgin wood, that it could be extracted at no extra cost online and at very low cost out in the real world, where “sensors are really cheap.” Once extracted, it is rendered as behavioral data, producing a surplus that forms the basis of a wholly new class of market exchange. Surveillance capitalism originates in this act of digital dispossession, brought to life by the impatience of over-accumulated investment and two entrepreneurs who wanted to join the system.

Today’s owners of surveillance capital have declared a fourth fictional commodity expropriated from the experiential realities of human beings whose bodies, thoughts, and feelings are as virgin and blameless as nature’s once-plentiful meadows and forests before they fell to the market dynamic. In this new logic, human experience is subjugated to surveillance capitalism’s market mechanisms and reborn as “behavior.” These behaviors are rendered into data, ready to take their place in a numberless queue that feeds the machines for fabrication into predictions and eventual exchange in the new behavioral futures markets.

The commodification of behavior under surveillance capitalism pivots us toward a societal future in which market power is protected by moats of secrecy, indecipherability, and expertise. Even when knowledge derived from our behavior is fed back to us as a quid pro quo for participation, as in the case of so-called “personalization,” parallel secret operations pursue the conversion of surplus into sales that point far beyond our interests. We have no formal control because we are not essential to this market action.

In this future we are exiles from our own behavior, denied access to or control over knowledge derived from its dispossession by others for others. Knowledge, authority, and power rest with surveillance capital, for which we are merely “human natural resources.” We are the native peoples now whose tacit claims to self-determination have vanished from the maps of our own experience.

Digital dispossession is not an episode but a continuous coordination of action, material, and technique, not a wave but the tide itself. Google’s leaders understood from the start that their success would require continuous and pervasive fortifications designed to defend their “repetitive sin” from contest and constraint. They did not want to be bound by the disciplines typically imposed by the private market realm of corporate governance or the democratic realm of law. In order for them to assert and exploit their freedom, democracy would have to be kept at bay.

A third set of answers requires an appreciation of the political and cultural circumstances and strategies that advanced surveillance capitalism’s claims and protected them from fatal challenge. It is this third domain that we pursue in the sections that follow. No single element is likely to have done the job, but together a convergence of political circumstances and proactive strategies helped enrich the habitat in which this mutation could root and flourish. These include (1) the relentless pursuit and defense of the founders’ “freedom” through corporate control and an insistence on the right to lawless space; (2) the shelter of specific historical circumstances, including the policies and juridical orientation of the neoliberal paradigm and the state’s urgent interest in the emerging capabilities of behavioral surplus analysis and prediction in the aftermath of the September 2001 terror attacks; and (3) the intentional construction of fortifications in the worlds of politics and culture, designed to protect the kingdom and deflect any close scrutiny of its practices.

Page and Brin were the first to introduce a dual-class share structure to the tech sector with Google’s 2004 public offering. The two would control the super-class “B” voting stock, shares that each carried ten votes, as compared to the “A” class of shares, which each carried only one vote. This arrangement inoculated Page and Brin from market and investor pressures, as Page wrote in the “Founder’s Letter” issued with the IPO: “In the transition to public ownership, we have set up a corporate structure that will make it harder for outside parties to take over or influence Google.… The main effect of this structure is likely to leave our team, especially Sergey and me, with increasingly significant control over the company’s decisions and fate, as Google shares change hands.”

In the absence of standard checks and balances, the public was asked to simply “trust” the founders. Schmidt would voice this theme on their behalf whenever challenged on the subject. For example, at the Cato Institute in December 2014, Schmidt was asked about the possibility of abuse of power at Google. He simply assured the audience of the continuity of the firm’s dynastic line. Page had succeeded Schmidt as CEO in 2011, and the current leaders would handpick future leaders: “We’re fine with Larry… same circus, same clowns… it’s the same people… all of us who built Google have the same view, and I am sure our successors will have the same view.”

By that year, Page and Brin had a 56 percent majority vote, which they used to impose a new tri-class share structure, adding a “C” class of zero-voting-rights stock. As Bloomberg Businessweek observed, “The neutered ‘C’ shares ensure Page and Brin retain control far into the future.…” By 2017, Brin and Page controlled 83 percent of the super-voting-class “B” shares, which translated into 51 percent of the voting power.

Many Silicon Valley founders followed Google’s lead. By 2015, 15 percent of IPOs were introduced with a dual-class structure, compared to 1 percent in 2005, and more than half of those were for technology companies. Most significantly, Facebook’s 2012 IPO featured a two-tiered stock structure that left founder Mark Zuckerberg in control of voting rights. The company then issued nonvoting class “C” shares in 2016, solidifying Zuckerberg’s personal control over every decision.

While financial scholars and investors debated the consequences of these share structures, absolute corporate control enabled the Google and Facebook founders to aggressively pursue acquisitions, establishing an arms race in two critical arenas. State-of-the-art manufacturing depended on machine intelligence, compelling Google and later Facebook to acquire companies and talent representing its disciplines: facial recognition, "deep learning," augmented reality, and more. But machines are only as smart as the volume of their diet allows. Thus, Google and Facebook vied to become the ubiquitous net positioned to capture the swarming schools of behavioral surplus flowing from every computer-mediated direction. To this end the founders paid outsized premiums for the chance to corner behavioral surplus through acquisitions of an ever-expanding roster of key supply routes.

Facebook's Zuckerberg pursued the same strategies, paying "astronomical" prices for a "fast and furious" parade of typically unprofitable startups like virtual reality firm Oculus ($2 billion) and the messaging application WhatsApp ($19 billion), thus ensuring Facebook's ownership of the gargantuan flows of human behavior that would pour through these pipes. Consistent with the extraction imperative, Zuckerberg told investors that he would not consider driving revenue until the service reaches "billions" of users. As one tech journalist put it, "There's no real need for Zuckerberg to chat with the board… there's no way for shareholders to check Zuckerberg's antics.…"

It's worth noting that an understanding of this logic of accumulation would have usefully contributed to the EU Commission's deliberations on the WhatsApp acquisition, which was permitted based on assurances that data flows from the two businesses would remain separate. The commission would discover later that the extraction imperative and its necessary of economies of scale in supply operations compel the integration of surplus flows in the quest for better prediction products.

Google's founders constructed a corporate form that gave them absolute control in the market sphere, and they also pursued freedom in the public sphere. A key element of Google's freedom strategy was its ability to discern, construct, and stake its claim to unprecedented social territories that were not yet subject to law. Cyberspace is an important character in this drama, celebrated on the first page of Eric Schmidt and Jared Cohen's book on the digital age: “The online world is not truly bound by terrestrial laws… it's the world's largest ungoverned space.” They celebrate their claim to operational spaces beyond the reach of political institutions: the twenty-first-century equivalent of the “dark continents” that drew nineteenth-century European speculators to their shores.

This is an Andy Grove formula.… “High tech runs three-times faster than normal businesses. And the government runs three-times slower than normal businesses. So we have a nine-times gap.… And so what you want to do is you want to make sure that the government does not get in the way and slow things down.”

Both Brin and Page are even more candid in their contempt for law and regulation. CEO Page surprised a convocation of developers in 2013 by responding to questions from the audience, commenting on the “negativity” that hampered the firm’s freedom to “build really great things” and create “interoperable” technologies with other companies: “Old institutions like the law and so on aren’t keeping up with the rate of change that we’ve caused through technology.… The laws when we went public were 50 years old. A law can’t be right if it’s 50 years old, like it’s before the internet.” When asked his thoughts on how to limit “negativity” and increase “positivity,” Page reflected, “Maybe we should set aside a small part of the world… as technologists we should have some safe places where we can try out some new things and figure out what is the effect on society, what’s the effect on people, without having to deploy kind of into the normal world.”

It is important to understand that surveillance capitalists are impelled to pursue lawlessness by the logic of their own creation. Google and Facebook vigorously lobby to kill online privacy protection, limit regulations, weaken or block privacy-enhancing legislation, and thwart every attempt to circumscribe their practices because such laws are existential threats to the frictionless flow of behavioral surplus.

Extraction quarry must be both unprotected and available at zero cost if this logic of accumulation is to succeed. These requirements are also an Achilles heel. Code is law for Google now, but the risk of new laws in its established and anticipated territories remains a persistent danger to surveillance capitalism. If new laws were to outlaw extraction operations, the surveillance model would implode. This market form must either gird itself for perpetual conflict with the democratic process or find new ways to infiltrate, seduce, and bend democracy to its ends if it is to fulfill its own inner logic. The survival and success of surveillance capitalism depend upon engineering collective agreement through all available means while simultaneously ignoring, evading, contesting, reshaping, or otherwise vanquishing laws that threaten free behavioral surplus.

These claims to lawless space are remarkably similar to those of the robber barons of an earlier century. Like the men at Google, the late-nineteenth-century titans claimed undefended territory for their own interests, declared the righteousness of their self-authorizing prerogatives, and defended their new capitalism from democracy at any cost. At least in the US case, we have been here before.

Economic historians describe the dedication to lawlessness among the Gilded Age “robber barons” for whom Herbert Spencer’s social Darwinism played the same role that Hayek, Jensen, and even Ayn Rand play for today’s digital barons. In the same way that surveillance capitalists excuse their corporations’ unprecedented concentrations of information and wealth as the unavoidable result of “network effects” and “winner-take-all” markets, the Gilded Age industrialists cited Spencer’s specious, pseudoscientific “survival of the fittest” as proof of a divine plan intended to put society’s wealth in the hands of its most aggressively competitive individuals.

The Gilded Age millionaires, like today’s surveillance capitalists, stood on the frontier of a vast discontinuity in the means of production with nothing but blank territory in which to invent a new industrial capitalism free from constraints on the use of labor, the nature of working conditions, the extent of environmental destruction, the sourcing of raw materials, or even the quality of their own products. And like their twenty-first-century counterparts, they did not hesitate to exploit the very law that they despised, flying the banner of “private property” and “freedom of contract,” much as surveillance capitalists march under the flag of freedom of speech as the justification for unobstructed technological “progress,” a topic to which we shall return.

Imbued with the conviction that “the state had neither right nor reason to interfere in the workings of the economy,” the Gilded Age millionaires joined forces to defend the “rights of capital” and limit the role of elected representatives in setting policy or developing legislation. There was no need for law, they argued, when one had the “law of evolution,” the “laws of capital,” and the “laws of industrial society.” John Rockefeller insisted that his outsized oil fortune was the result of “the natural law of trade development.” Jay Gould, when questioned by Congress on the need for federal regulation of railroad rates, replied that rates were already regulated by “the laws of supply and demand, production and consumption.”

The bottom line for Gilded Age business elites was that the most effective way to protect the original sin of that economic era was, as David Nasaw put it, “to circumscribe democracy.” They did this by lavishly funding their own political candidates as well as through the careful honing and aggressive dissemination of an ideological attack on the very notion of democracy’s right to interfere in the economic realm. Their industries were to be “self-regulating”: free to follow their own evolutionary laws. “Democracy,” they preached, “had its limits, beyond which voters and their elected representatives dared not trespass lest economic calamity befall the nation.”

A fascinating study by University of California law professor Jodi Short empirically illustrates the role of neoliberal ideology as one important explanation for Google's ambitions and successful defense of lawless territory. Short analyzed 1,400 law review articles on the subject of regulation, all published between 1980 and 2005. As the influence of Hayek and Friedman predicts, the dominant theme of this literature was "the coercive nature of administrative government" and the systematic conflation of industry regulation with "tyranny" and "authoritarianism." According to this worldview, all regulation is burdensome, and bureaucracy must be repudiated as a form of human domination. Short observes that during the sample decades these fears were even more influential in shaping regulatory approaches than rational arguments about cost and efficiency, and she identifies two points of origin for these anxieties.

The first source was in the US business community's opposition to New Deal reforms, which, not unlike the propaganda of the Gilded Age millionaires, cast resistance to regulation as a "righteous fight to defend democracy from dictatorship." A second source was the dread of totalitarianism and collectivism incited by World War II and the cold war, a direct inheritance from Hayek. These defensive themes infiltrated and reshaped US political thought, and they gradually transformed policy makers' assumptions about the regulatory role of the state.

Short found several suggested remedies for "coercive" governmental regulation in the literature, but the most salient, especially after 1996—the very years when digital technology and the internet were becoming mainstream—was "self-regulation." The idea here is that firms set their own standards, monitor their own compliance with those standards, and even judge their own conduct in order to "voluntarily report and remediate violations." By the time of Google's public offering in 2004, self-regulation was fully enshrined within government and across the business community as the single most effective tool for regulation without coercion and the antidote to any inclination toward collectivism and the centralization of power.

This neoliberal legacy was a windfall for the new surveillance capitalists. As another legal scholar, Frank Pasquale, observed, it produced a model that treated privacy as a competitive good, assuming that "consumers" engage only with services that offer the level of privacy they seek. Regulatory interference, according to this view, would only undermine competitive diversity. It also credits the "notice and consent" model—click-wrap and its "sadistic" relatives—as accurate signals of individual privacy choices.

The neoliberal zeitgeist also favored Google's leaders, and later their fellow travelers in the surveillance project, as they sought shelter for their inventions beneath claims of First Amendment rights to freedom of expression. This is a complex and contested arena in which constitutional law and political ideology are thoroughly entangled, and I point out just a few elements here in order to better understand the habitat that nurtured the new surveillance market form.

The key dynamic here is that First Amendment jurisprudence, especially over the last two decades, has reflected a “conservative-libertarian” interpretation of First Amendment rights. As constitutional law scholar Steven Heyman suggests, “In recent decades, the First Amendment has become one of the most important means by which judges have sought to advance a conservative-libertarian agenda.” This has produced many dramatic judicial decisions, including the US Supreme Court’s rejection of any constraints on the role of money in election campaigns, its rejection of restrictions on hate speech and pornography, and its holding that the right to free association takes precedence over state civil rights laws that bar discrimination.

As many legal scholars observe, the ideological orientation of contemporary First Amendment judicial reasoning asserts a close connection between free speech and property rights. The logic that links ownership to an absolute entitlement to freedom of expression has led to a privileging of corporate action as “speech” deserving of constitutional protection. Some scholars regard this as a dangerous reversion to the feudal doctrines from which corporate law evolved in the seventeenth century. Those medieval legal principles limited the sovereign’s authority over “the corporations of Aristocracy, Church, guilds, universities, and cities… who asserted the right to rule themselves.” One result is that US courts have been “quick to see the possibilities of governmental overreach, but much less willing to see the problems of ‘private,’ let alone corporate, power.”

In this context, surveillance capitalists vigorously developed a “cyberlibertarian” ideology that Frank Pasquale describes as “free speech fundamentalism.” Their legal teams aggressively assert First Amendment principles to fend off any form of oversight or externally imposed constraints that either limit the content on their platforms or the “algorithmic orderings of information” produced by their machine operations. As one attorney who has represented many of the leading surveillance capitalists puts it, “The lawyers working for these companies have business reasons for supporting free expression. Indeed, all of these companies talk about their businesses in the language of free speech.”

This is one respect in which the surveillance capitalists are not unprecedented. Adam Winkler, a historian of corporate rights, reminds us, “Throughout American history the nation’s most powerful corporations have persistently mobilized to use the Constitution to fight off unwanted government regulations.” Although today’s mobilizations are not original, Winkler’s careful account demonstrates the effects of past mobilizations on the distribution of power and wealth in US society and the strength of democratic values and principles in each era.

Four years later, in 1995, an early provider of web services called Prodigy was sued for a defamatory anonymous posting on one of its message boards. This time a New York state court came to the opposite conclusion. The key problem as the court saw it was that Prodigy had exercised editorial control by moderating its message boards. The company established content guidelines and deleted posts that violated those standards. The court concluded that Prodigy was a publisher, not merely a distributor, because it had taken responsibility for the content on its site. Were the court’s ruling to stand, internet companies would face “a paradoxical no-win situation: the more an ISP tried to keep obscene or harmful material away from its users, the more it would be liable for that material.” Internet companies faced a binary choice: “free speech savior or shield for scoundrels?”

According to Senator Ron Wyden, Section 230 was intended to resolve that contradiction by encouraging internet companies to exercise some control over content without the risk of legal sanctions. The very first sentence of the statute mentions “protection for 'good samaritan' blocking and screening of offensive material.” What Wyden and his colleagues could not have anticipated, and still do not grasp, is that the logic of this early controversy no longer holds. Neither CompuServe nor Prodigy was a surveillance capitalist, but many of today’s internet intermediaries are committed to the pursuit of surveillance revenues.

There is nothing neutral about the surveillance intermediary now, as the extraction imperative and its demand for economies of scale in surplus supply mean that the surveillance capitalists must use every means to attract a never-ending tide of content to their shores. They no longer merely host content but aggressively, secretly, and unilaterally extract value from that content. As we shall see in Chapter 18, economic imperatives require them to forgo as few of these raw materials as possible. That means moderating only those extremes that threaten the volume and velocity of surplus by repelling users or attracting regulatory scrutiny. This is the reason that firms such as Facebook, Google, and Twitter have been reluctant to remove even the most egregious content from their landscapes, and it helps to explain why "lawyers for tech companies litigate ferociously to prevent even a sliver of erosion" in Section 230.

The commissioners proceeded to outline federal legislation that would have protected consumers online despite the dominant bias against regulation and in favor of treating internet operations as free speech. The recommendations demanded “clear and conspicuous” notice of information practices; consumer choice over how personal information is used; access to all personal information, including rights to correct or delete; and enhanced security of personal information. Had these been translated into law, it is quite possible that many of the foundational elements of surveillance capitalism would have been either plainly illegal or at least subject to public examination and contest.

For all its secrecy, even the NSA was subject to the temporalities and legal restrictions of a democratic state. The tempos of democracy are slow by design, weighted by redundancies, checks and balances, laws and rules. The agencies sought methods of deployment that could rapidly bypass legal and bureaucratic restrictions.

Google’s mission was to “organize and make accessible the world’s information,” and by late 2001 the intelligence community established “information dominance” in the public’s house, quickly institutionalizing it in hundreds of billions of dollars’ worth of state-sponsored global technology infrastructure, personnel, and practice. The contours of a new interdependency between public and private agents of information dominance began to emerge, one that is best understood through the lens of what the sociologist Max Weber once called an “elective affinity” born of a mutual magnetism that originates in shared meanings, interests, and reciprocities.

The elective affinity between public intelligence agencies and the fledgling surveillance capitalist Google blossomed in the heat of emergency to produce a unique historical deformity: surveillance exceptionalism. The 9/11 attacks transformed the government’s interest in Google, as practices that just hours earlier were careening toward legislative action were quickly recast as mission-critical necessities. Both institutions craved certainty and were determined to fulfill that craving in their respective domains at any price. These elective affinities sustained surveillance exceptionalism and contributed to the fertile habitat in which the surveillance capitalism mutation would be nurtured to prosperity.

Surveillance exceptionalism thrived in that hyperactivity. Poindexter’s Total Information Awareness program did not obtain congressional support, but an analysis in the MIT Technology Review showed that many of the TIA initiatives were quietly reassigned to the Pentagon’s Advanced Research and Development Activity (ARDA), which in 2002 received $64 million to fund a research program in “novel intelligence from massive data.” In 2004 the US General Accounting Office surveyed 199 data-mining projects across dozens of federal agencies and more than 120 programs developed to collect and analyze personal data to predict individual behavior. The New York Times reported in 2006 that the intelligence agencies, backed by a $40 billion annual budget, regularly fielded secretive shopping expeditions to Silicon Valley in search of new data-mining and analysis technologies.

By late summer 2003, Google was awarded a $2.07 million contract to outfit the agency with Google search technology. According to documents obtained by Consumer Watchdog under the Freedom of Information Act, the NSA paid Google for a “search appliance capable of searching 15 million documents in twenty-four languages.” Google extended its services for another year at no cost in April 2004.

In 2006 General Keith Alexander outlined his vision for a search tool called ICREACH that “would allow unprecedented volumes of… metadata to be shared and analyzed across the many agencies in the Intelligence Community.” By late 2007, the program was piloted, boosting the number of communications events it shared from 50 billion to more than 850 billion. The system was designed with a “Google-like” search interface that enabled analysts to run searches against meta-data “selectors” and to extract vital behavioral surplus for analyses that could reveal “social networks,” “patterns of life,” and “habits,” and in general “predict future behavior.”

The government’s need to evade constitutional oversight, argues legal scholar Jon Michaels, leads to secret public-private intelligence collaborations that tend to be “orchestrated around handshakes rather than legal formalities, such as search warrants, and may be arranged this way to evade oversight and, at times, to defy the law.” He observed that intelligence agencies are irresistibly drawn to “and in some respects dependent upon” firms’ privately held data resources.

McConnell made clear that Google's surveillance-based operations in data capture, extraction, and analysis were both taken for granted and coveted. Here the boundaries of private and public melt in the intense heat of new threats and their high-velocity demands that must be met in "milliseconds." In McConnell's future there is one "seamless" surveillance empire in which the requirements of self-preservation leave no opportunity for the amenities of democracy, with its time-wasting practices of due process, evidence, warrants, and law.

An effective partnership with the private sector must be formed so information can move quickly back and forth from public to private and classified to unclassified… to protect the nation's critical infrastructure. Recent reports of possible partnership between Google and the government point to the kind of joint efforts—and shared challenges—that we are likely to see in the future… such arrangements will muddy the waters between the traditional roles of the government and the private sector.… Cyberspace knows no borders, and our defensive efforts must be similarly seamless.

Cate and Dempsey note the "expansive aggregation" of personal data in the hands of private companies: "Governments understandably want access to this data.… Essentially every government in the world claims the power to compel disclosure of this data by the companies that hold it." Had it not been for surveillance exceptionalism, it is possible that these data would not even exist, at least not in their current volume and detail.

Surveillance exceptionalism helped to shape the evolutionary course of information capitalism by creating an environment in which Google's budding surveillance practices were coveted rather than contested. Once again, history offers us no control groups, and we cannot know with certainty whether information capitalism might have developed in a different direction had it not been for the sudden new interest in surveillance capabilities. For now, it appears that one unanticipated consequence of this public-private "elective affinity" was that the fledgling practices of surveillance capitalism were allowed to root and grow with little regulatory or legislative challenge, emboldening Google's young leaders to insist on lawlessness as a natural right and, in ways that are even more opaque, emboldening the state to grant them that freedom.

In the context of new military purpose, the digital capabilities that were aimed toward the advocacy-oriented values of the behavioral value reinvestment cycle flowed toward surveillance without impediment. Surveillance assets thrived without risk of sanction and attracted surveillance capital. Revenue followed. The situation recalls the auto, steel, and machine tool industries at mid-century, when military orders kept plants operating at full capacity. In the end, however, this turned out to be more curse than blessing. Military demand distorted and suppressed the innovation process and drove a wedge between these industries and their civilian customers, leaving them vulnerable to foreign competitors in the globalizing markets of the late 1970s and early 1980s.

Similarly, under the conditions of surveillance exceptionalism, Google's leaders were not compelled to undertake the arduous and risky work of inventing an exchange-based advocacy-oriented market form when the surveillance model was so lucrative. Why risk experimentation with more-organic paths to monetization when surveillance and extraction operations were safe from law and hugely profitable? Eventually, it wasn't just Google asking these questions; every other internet business faced the same choices. Once surveillance revenues set the bar for venture capitalists and Wall Street analysts, it became that much easier for internet companies to go with the flow. Then it became onerous not to.

Why is it that so many years after the events that triggered the mania for information dominance, surveillance capitalism still runs with relatively little impediment, especially in the US? The intervening years have seen the proliferation of thousands of institutional facts that normalized surveillance capitalism's practices and made them appear necessary and inevitable: the discovery of behavioral surplus and the massive accumulations of capital and material that followed, the proliferation of devices and services, the integration of data flows, and the institutionalization of futures markets in human behavior.

Although many elements of this fortification strategy have been well publicized, their importance for our story lies in the fact that each is one aspect of a multipronged effort that deflects scrutiny from core operations in order to maintain the flow of free, unregulated behavioral surplus. Fortifications have been erected in four key arenas to protect Google, and eventually other surveillance capitalists, from political interference and critique: (1) the demonstration of Google's unique capabilities as a source of competitive advantage in electoral politics; (2) a deliberate blurring of public and private interests through relationships and aggressive lobbying activities; (3) a revolving door of personnel who migrated between Google and the Obama administration, united by elective affinities during Google's crucial growth years of 2009–2016; and (4) Google's intentional campaign of influence over academic work and the larger cultural conversation so vital to policy formation, public opinion, and political perception. The results of these four arenas of defense contribute to an understanding of how surveillance capitalism's facts came to stand and why they continue to thrive.

Google demonstrated that the same predictive knowledge derived from behavioral surplus that had made the surveillance capitalists wealthy could also help candidates win elections. To make the point, Google was ready to apply its magic to the red-hot core of twenty-first-century campaigning, beginning with the 2008 Obama presidential campaign. Schmidt had a leading role in organizing teams and guiding the implementation of cutting-edge data strategies that would eclipse the traditional political arts with the science of behavioral prediction. Indeed, "At Obama's Chicago headquarters… they remodeled the electorate in every battleground state each weekend… field staff could see the events' impact on the projected behaviors and beliefs of every voter nationwide."

Research by media scholars Daniel Kreiss and Philip Howard indicates that the 2008 Obama campaign compiled significant data on more than 250 million Americans, including "a vast array of online behavioral and relational data collected from use of the campaign's web site and third-party social media sites such as Facebook.…" Journalist Sasha Issenberg, who documented these developments in his book The Victory Lab, quotes one of Obama's 2008 political consultants who likened predictive modeling to the tools of a fortune-teller: "We knew who… people were going to vote for before they decided."

Obama used his proximity to Schmidt to cement his own identity as the innovation candidate poised to disrupt business as usual in Washington. Once elected, Schmidt joined the Transition Economic Advisory Board and appeared next to Obama at his first postelection press conference. According to Politico, "The image alone of Schmidt standing elbow-to-elbow with Obama's top economic thinkers was enough to send shivers up the spine of Google's competitors. 'This terrifies Microsoft,' said a Democratic lobbyist familiar with the industry. 'There's a reason why people are scared to death of Google.'"

Political correspondent Jim Rutenberg’s New York Times account of the data scientists’ seminal role in the 2012 Obama victory offers a vivid picture of the capture and analysis of behavioral surplus as a political methodology. The campaign knew “every single wavering voter in the country that it needed to persuade to vote for Obama, by name, address, race, sex, and income,” and it had figured out how to target its television ads to these individuals. One breakthrough was the “persuasion score” that identified how easily each undecided voter could be persuaded to vote for the Democratic candidate.

Second, with the 2012 election in sight, an interview with the Washington Post in 2011 found Schmidt boasting about another fortification strategy: “The staffers are young—the staffers get it.… So that’s what we depend on. And of course we’ve hired ex-staffers as well. They all know each other. So that’s how it really works.” Google’s political utilities paved the way for the unusually crowded and fast-spinning revolving door between the East Coast and West Coast centers of power. The Google Transparency Project analyzed the movement of staff between the Googlesphere (the company plus its affiliates and its law and lobbying firms) and the government (including the White House, Congress, government agencies, federal commissions, and national political campaigns) during the Obama years. It found that by April 2016, 197 individuals had migrated from the government into the Googlesphere, and 61 had moved in the other direction. Among these, 22 White House officials went to work for Google, and 31 Googlesphere executives joined the White House or federal advisory boards with direct relevance to Google’s business.

Third, just to be on the safe side, Google shared its largesse throughout the political system. In Schmidt’s 2014 book, coauthored with longtime Google executive Jonathan Rosenberg, the CEO aggressively developed the theme of government as the shill of incumbents colluding to inhibit change, with Google on the outside: an upstart and a disrupter. The authors voiced their disdain for politicians and lobbyists, writing, “This is the natural path of politicians since incumbents tend to have a lot more money than disrupters and are quite expert in using it to bend the political will of any democratic government.”

In the fourth arena of fortifications the corporation learned to infiltrate and influence academic research and civil society advocacy in ways that softened or in some cases thwarted the examination of its practices. The Washington Post describes Google as a "master of Washington influence" and notes the subtlety with which the firm grasps and directs its own narrative. Schmidt was hands-on in this work as well. Already a board member of the New America Foundation, a public-policy think tank that played an influential role in shaping the Obama administration’s approach to economic issues, he assumed the chairmanship in 2013 with a personal donation of $1 million, a significant percentage of its $12.9 million annual budget that year. Between 1999 and 2016, when Schmidt left the board, the foundation received $21 million from a combination of Google, Schmidt, and Schmidt’s family foundation.

The Washington Post published an elaborate exposé of Google’s meticulous work in this fourth domain, illustrated in the backstage intrigues that accompanied a three-part series on internet search competition held at George Mason University’s Law and Economics Center, a "free-market–oriented" academic center that had received significant funding from Google. The meetings occurred in May 2012, just as the FTC was investigating the Google antitrust case. Reporters found that company staffers worked closely with the center, choosing pro-Google speakers and participants, many of whom were Google employees. Their efforts included "sending the center’s staff a detailed spreadsheet listing members of Congress, FTC commissioners, and senior officials with the Justice Department and state attorney general’s office." Reporters noted that the conference’s panels were dominated by "leading technology and legal experts" who forcefully rejected the need for government action against Google, "making their arguments before some of the very regulators who would help determine its fate." Many participants had no idea that Google was involved in crafting the meetings because Google and center staffers had agreed to conceal the corporation’s backstage involvement.

The FTC antitrust investigation appears to have heightened Google's fears of a regulatory threat to surveillance capitalism. That year, Google's grant-making operation aimed at civil society organizations took an aggressive turn. According to the Center for Media and Democracy's investigatory research report, "The Googlization of the Far Right," the corporation's 2012 list of grantees featured a new group of antigovernment groups known for their opposition to regulation and taxes and their support for climate-change denial, including Grover Norquist's Americans for Tax Reform, the Koch brothers–funded Heritage Action, and other antiregulatory groups such as the Federalist Society and the Cato Institute. The corporation also quietly acknowledged its membership in the corporate lobbying group ALEC, known for its opposition to gun control and emissions curbs, and for its support for voter-suppression schemes, tobacco industry tax breaks, and other far-right causes. Meanwhile, a list of Google Policy Fellows for 2014 included individuals from a range of nonprofit organizations whom one would expect to be leading the fight against that corporation's concentrations of information and power, including the Center for Democracy and Technology, the Electronic Frontier Foundation, the Future of Privacy Forum, the National Consumers League, the Citizen Lab, and the Asociación por los Derechos Civiles.

In July 2017 the Wall Street Journal reported that since 2009, Google had actively sought out and provided funding to university professors for research and policy papers that support Google's positions on matters related to law, regulation, competition, patents, and so forth. In many cases, Google weighed in on the papers before publication, and some of the authors did not disclose Google as a source of funding. Although Google publicly claimed that "the check came with no requirements," another case in 2017 belied that notion. That summer, one of the New America Foundation's most highly regarded scholars and a specialist in digital monopolies, Barry Lynn, posted a statement praising the EU's historic decision to levy a $2.7 billion fine on Google as the result of a multiyear antitrust investigation. According to the New York Times and Lynn's own account, New America's director bent to pressure from Schmidt, firing Lynn and his Open Markets team of ten researchers. "Google is very aggressive in throwing its money around Washington and Brussels, and then pulling strings," Lynn told the New York Times. "People are so afraid of Google now."

With Google in the lead, surveillance capitalism vastly expanded the market dynamic as it learned to expropriate human experience and translate it into coveted behavioral predictions. Google and this larger surveillance project have been birthed, sheltered, and nurtured to success by the historical conditions of their era—second-modernity needs, the neoliberal inheritance, and the realpolitik of surveillance exceptionalism—as well as by their own purpose-built fortifications designed to protect supply chain operations from scrutiny through political and cultural capture.

Surveillance capitalism’s ability to keep democracy at bay produced these stark facts. Two men at Google who do not enjoy the legitimacy of the vote, democratic oversight, or the demands of shareholder governance exercise control over the organization and presentation of the world’s information. One man at Facebook who does not enjoy the legitimacy of the vote, democratic oversight, or the demands of shareholder governance exercises control over an increasingly universal means of social connection along with the information concealed in its networks.

In this chapter we follow the consequences of the extraction imperative as it drives the elaboration of the new market form and its competitive dynamics. The imperative elevates surplus supply operations to a defining role in every aspect of the surveillance capitalist enterprise. This begins with a continuous parade of innovations aimed at cornering raw-material supplies. Cornering is not simply a technological achievement. Sustainable dispossession requires a highly orchestrated and carefully phased amalgam of political, communicative, administrative, legal, and material strategies that audaciously asserts and tirelessly defends claims to new ground. The success of these strategies, first at Google and later at Facebook, established both their feasibility and their rewards, drawing new competitors into an increasingly ruthless cycle of kidnapping human experience, cornering surplus supplies, and competing in new behavioral futures markets.

When Harvard Business School's Benjamin Edelman researched these hidden mechanisms in 2010, he found that the "enhanced features" option of a product called Google Toolbar—a plug-in for Microsoft's Internet Explorer web browser that lets users search without having to go to google.com—transmits to the company "the full URL of every page view, including searches at competing search engines." Edelman discovered that it was "strikingly easy" to activate this option but impossible to disable it. Even when a user specifically instructed that the toolbar be disabled, and even when it appeared to be disabled because it had disappeared from view, the toolbar continued to track browsing behavior.

The extraction imperative demands that everything be possessed. In this new context, goods and services are merely surveillance-bound supply routes. It's not the car; it's the behavioral data from driving the car. It's not the map; it's the behavioral data from interacting with the map. The ideal here is continuously expanding borders that eventually describe the world and everything in it, all the time.

Traditionally, monopolies on goods and services disfigure markets by unfairly eliminating competition in order to raise prices at will. Under surveillance capitalism, however, many of the practices defined as monopolistic actually function as means of cornering user-derived raw-material supplies. There is no monetary price for the user to pay, only an opportunity for the company to extract data. Cornering practices are not designed to protect product niches but rather to protect critical supply routes for the unregulated commodity that is behavioral surplus. In another time, rogue market actors might corner markets in copper or magnesium, but in our time it is behavioral surplus. The corporation unfairly impedes competitors in Search in order to protect the dominance of its most important supply route, not primarily to fix prices.

It is important to say—and we will revisit this theme more than once—that regulatory interventions designed to constrain Google's monopoly practices are likely to have little effect on the fundamental operations of this market form. New supply routes are continuously discovered, opened, and secured. Dispossession activities are compelled to circumvent every obstacle and will continue to do so, short of a genuine existential threat.

Google fiercely defends threatened supply routes. Any disruption of its extraction operations and its exclusive claims to raw material is the line that cannot be breached. In 2009 Android manufacturer Motorola chose to replace Google's free location services with those of Skyhook Wireless, which, Motorola believed, produced more-reliable results. A Google product manager admitted Skyhook's superiority but expressed his concerns in an e-mail to a Google executive, noting that if other manufacturers switched to Skyhook, it “would be awful for Google, because it will cut off our ability to continue collecting data” for the company's Wi-Fi location database. Court documents from Skyhook's eventual lawsuit against Motorola (and Samsung) include an e-mail from Google's senior vice president of Mobile to Motorola's CEO, insisting that the interruption of Google's data collection was a “stop-ship issue.”

Another 2015 analysis, this one of the top one million websites, by Timothy Libert of the University of Pennsylvania, found that 90 percent leak data to an average of nine external domains that track, capture, and expropriate user data for commercial purposes. Among these websites, 78 percent initiate third-party transfers to a domain owned by one company: Google. Another 34 percent transfer to a Facebook-owned domain. Steven Englehardt and Arvind Narayanan from Princeton University reported in 2016 on the results of their measurement and analysis of tracking data from one million websites. They identified 81,000 third parties, but only 123 of those are present on more than 1 percent of sites. Of that group, the top five third parties and twelve of the top twenty are Google-owned domains. “In fact,” they conclude, “Google, Facebook, and Twitter are the only third-party entities present on more than 10% of sites.”

Two themes stand out in the research report: ubiquity and intensification. First, there is hardly an innocent app; if it’s not tracking you now, it may be doing so in the next week or month: “There is an entire industry based upon these trackers, and apps identified as ‘clean’ today may contain trackers that have not yet been identified. Tracker code may also be added by developers to new versions of apps in the future.” Second is that even the most innocent-seeming applications such as weather, flashlights, ride sharing, and dating apps are “infested” with dozens of tracking programs that rely on increasingly bizarre, aggressive, and illegible tactics to collect massive amounts of behavioral surplus ultimately directed at ad targeting.

For example, the ad tracker FidZup developed “communication between a sonic emitter and a mobile phone.…” It can detect the presence of mobile phones and therefore their owners by diffusing a tone, inaudible to the human ear, inside a building: “Users installing ‘Bottin Gourmand,’ a guide to restaurants and hotels in France, would thus have their physical location tracked via retail outlet speakers as they move around Paris.

Google is under enormous pressure from the financial community to increase the “effectiveness” of its tracking, so that it can increase revenues and profits. Giving a user the ability to control his own privacy information (and to protect himself from malware) by blocking invisible connections to problematic sites constitutes an existential threat to Google.

The first stage of successful dispossession is initiated by unilateral incursion into undefended space: your laptop, your phone, a web page, the street where you live, an e-mail to your friend, your walk in the park, browsing online for a birthday gift, sharing photos of your kids, your interests and tastes, your digestion, your tears, your attention, your feelings, your face. The incursion is when dispossession operations rely on their virtual capabilities to kidnap behavioral surplus from the nonmarket spaces of everyday life where it lives. The incursion initiates Google’s most basic and prolific form of dispossession: Arendt’s repeated “original sin of simple robbery.” Incursion moves down the road without looking left or right, continuously laying claim to decision rights over whatever is in its path. “I’m taking this,” it says. “These are mine now.”

The company has learned to launch incursions and proceed until resistance is encountered. It then seduces, ignores, overwhelms, or simply exhausts its adversaries. Seduction means a cascade of golden enticements: unprecedented storage, access to new qualities of information, new conveniences. When necessary, the company can just as easily pivot toward harsher tactics that aim to deplete its adversaries’ time, money, and grit. There are hundreds of cases launched against Google by countries, states, groups, and individuals, and there are many more cases that never become public.

Legal opposition and social protest have surfaced in relation to the digitalization of books, the collection of personal information through Street View’s Wi-Fi and camera capabilities, the capture of voice communications, the bypassing of privacy settings, the manipulation of search results, the extensive retention of search data, the tracking of smartphone location data, wearable technologies and facial-recognition capabilities, the secret collection of student data for commercial purposes, and the consolidation of user profiles across all Google’s services and devices, just to name several instances. Expect to see drones, body sensors, neurotransmitters, “digital assistants,” and other sensored devices on this list in the years to come.

In a second stage the aim is habituation. Whereas lawsuits and investigations unwind at the tedious pace of democratic institutions, Google continues the development of its contested practices at high velocity. During the elapsed time of FTC and FCC inquiries, court cases, judicial reviews, and EU Commission investigations, the new contested practices become more firmly established as institutional facts, rapidly bolstered by growing ecosystems of stakeholders. People habituate to the incursion with some combination of agreement, helplessness, and resignation. The sense of astonishment and outrage dissipates. The incursion itself, once unthinkable, slowly worms its way into the ordinary. Worse still, it gradually comes to seem inevitable. New dependencies develop. As populations grow numb, it becomes more difficult for individuals and groups to complain.

In a third stage of the cycle, when Google is occasionally forced to alter its practices, its executives and engineers produce superficial but tactically effective adaptations that satisfy the immediate demands of government authorities, court rulings, and public opinion. Meanwhile, in a final stage the corporation regroups to cultivate new rhetoric, methods, and design elements that redirect contested supply operations just enough so that they appear to be compliant with social and legal demands. The creativity, financial resources, and determination brought to the task of managing this staged process are flexible and dynamic. In contrast, the operational necessity for economies of scale in the capture of behavioral surplus is a perpetual-motion machine whose implacable rhythms offer no room for divergence.

The theory and practice of dispossession were developed and refined as the company learned how to counter and transform public resistance as an essential condition for the protection and expansion of its behavioral surplus franchise. Google’s launch of Gmail on April Fool’s Day, 2004, provided an early occasion to climb this learning curve as the corporation faced down public outrage over the automated scanning of e-mail content intended as a fresh source of surplus for targeted ads. Eventually, the dispossession cycle was refined as an explicit theory of change that framed a tactical game plan, which is by now regularly evoked as the surveillance capitalist corporation’s battle-tested response to societal resistance.

The dispossession cycle at Google was so successful in facing down the threats to Gmail that it was replicated and further elaborated in the battles over Google Street View, the street-mapping operation launched in 2007. Once again, the company did not ask permission. It simply repeated the “original sin of simple robbery” and took what it wanted, waiting for resistance to run its course as it devoured and datafied the world’s public spaces, streets, buildings, and homes.

Street View first entered public awareness with an apparently benign blog post. Peter Fleischer, Google’s “privacy counsel,” helped launch the new “service” by writing a paean celebrating America’s “noble tradition” of public spaces, where, he claimed, “people don’t have the same expectations of privacy as they do in their homes.” As a lawyer, Fleischer knows the work that words do in establishing contracts and setting precedents, so it pays to read his 2007 words with care. His casually declarative prose undertakes some extraordinary work as he asserts that all public spaces are fair game for Google’s taking. In his account, any public space is a fitting subject for the firm’s new breed of incursion without authorization, knowledge, or agreement. Homes, streets, neighborhoods, villages, towns, cities: they are no longer local scenes where neighbors live and walk, where residents meet and talk. Google Street View, we are informed, claims every place as just another object among objects in an infinite grid of GPS coordinates and camera angles.

In his declaration, Fleischer intends to establish Google's prerogative to empty every place of the subjective meanings that unite the human beings who gather there. Yes, when we leave our home we know we will be seen, but we expect to be seen by one another in spaces that we choose. Instead, it's all impersonal spectacle now. My house, my street, my neighborhood, my favorite café: each is redefined as a living tourist brochure, surveillance target, and strip mine, an object for universal inspection and commercial expropriation.

Google had already taken everything on the web, but Street View and Google's other mapping operations, Google Maps and Google Earth (the company's 3-D view of the world using satellite and aerial imagery), announced an even-more-ambitious vision. Everything in the world was to be known and rendered by Google, accessed through Google, and indexed by Google in its infinite appetite for behavioral surplus. The presumption is that nothing is beyond Google's borders. The world is vanquished now, on its knees, and brought to you by Google.

Fleischer similarly assures his audience of friendly terms. Street View, which used cartoonishly wrapped cars with a large 360-degree camera mount on the roof to capture the imagery it sought, was designed to "respect the privacy of people who happen to be walking down a public street," Fleischer wrote. "That's why we designed a simple process for anyone to contact us and have their image removed," and he promised that it would respect laws and customs "in other parts of the world."

Resistance came swiftly and often. By January 2009, Street View was facing opposition in Germany and Japan. … He told a reporter that it was all simply part of a "cycle of people understanding exactly what it is and what it isn't and what they shouldn't really be concerned about"—in other words, the dispossession cycle. … He cleverly equated any resistance to Google's incursions with the anti-freedom-of-expression interests of authoritarian governments and their "closed information societies." This would become a standard rhetorical device for Google and its allies as they executed their offense.

Was Hanke surprised, then, when in April 2009 residents of the quiet English village of Broughton blocked a Street View car that tried to breach the village perimeter, calling it an unwelcome intrusion? This was "the West," after all, but the debate over privacy, self-determination, and decision rights was anything but dead. Privacy International submitted a formal complaint to the UK privacy authority, citing more than 200 reports from people who were identifiable on Street View images and demanded that the service be suspended.

Google's ideal society is a population of distant users, not a citizenry. It idealizes people who are informed, but only in the ways that the corporation chooses. It means for us to be docile, harmonious, and, above all, grateful.

Google denied the charge, insisting that it was gathering only publicly broadcast Wi-Fi network names and the identifying addresses of Wi-Fi routers, but not personal information sent over the network. Within days, an independent analysis by German security experts proved decisively that Street View's cars were extracting unencrypted personal information from homes. Google was forced to concede that it had intercepted and stored "payload data," personal information grabbed from unencrypted Wi-Fi transmissions. As its apologetic blog post noted, "In some instances entire emails and URLs were captured, as well as passwords."

By 2012, there were multiple investigations in twelve countries, including most of Europe, the North Atlantic, and Australia, and Google had been found guilty of violating laws in at least nine countries.

In the US, attorneys general from thirty-eight states launched a probe into Google's Street View practices. Private citizens filed numerous class-action suits, eight of which were consolidated in the Northern California US District Court. The head of Privacy International said that Google was becoming "Big Brother." The Electronic Privacy Information Center championed substantial legal resistance in the US against Google's efforts to avoid repercussions in the wake of the Spy-Fi scandal, and it maintained a detailed and continuously updated online overview of the worldwide outrage, protests, investigations, litigation, and settlements in response to Google Street View and its extraction tactics.

Google characterized Street View's "privacy violations" as a "mistake" made by a single engineer working on an "experimental" project, whose code had inadvertently made it into Street View's software. The firm refused to release the identity of the mystery engineer and insisted that the project's leaders were unaware of the data capture and "had no intention" of using those data. As Eric Schmidt told the Financial Times, "We screwed up," noting that the engineer in question would face an internal investigation for his clear "violation" of Google's policies. Unbowed, Schmidt insisted on the validity of Google's mission to index all the world's information.

A 2012 investigation by the Federal Communications Commission described the case as "a deliberate software-design decision by one of the Google employees working on the Street View project." The engineer had been selected for the team because of his unique expertise in Wi-Fi "wardriving," the practice of driving around using equipment to locate wireless networks. His design notes indicated that user traffic and location data would be logged along with "information about what they are doing" that would "be analyzed offline for use in other initiatives." The notes identified but then dismissed "privacy considerations."

The FCC found evidence that contradicted Google's scapegoating narrative. The records showed that the engineer had e-mailed links to his software documentation to project leaders, who then shared them with the entire Street View team. It also found evidence that on at least two occasions, the engineer told his colleagues that Street View was collecting personal data. Despite these facts along with evidence of the company's exhaustive internal software reviews and testing procedures and the regular transfer of payload data from Street View's hard disks to Google's Oregon data center, Google's engineers denied any knowledge of personal data collection.

The thirty-eight attorneys general did not fare much better. When the group's leader, Connecticut's Richard Blumenthal, issued a civil investigative demand (equivalent to a subpoena) to get access to the infamous private data, “Google ignored it.” The company finally agreed to settle with the states in 2013, accepting a mere $7 million fine and a series of agreements regarding “aggressive” self-policing. The New York Times announced that Google had finally admitted that “it had violated people's privacy during its Street View mapping project, when it casually scooped up… personal information,” as if this scandal had been the only contested element in the whole affair. State officials crowed that “the industry giant… is committing to change its corporate culture to encourage sensitivity to issues of personal data privacy.” In light of the fact that the extraction imperative is what makes this giant a giant, one doesn't know whether to laugh or cry at the confidence of the attorneys general in Google's commitment to privacy self-regulation.

There are two key elements here that illuminate habituation tactics. The first is the simple fact of the elapsed time between Street View’s initial incursion in 2007, the 2010 scandal, the 2012 conclusion of the FCC inquiry, and the 2013 conclusion of the states’ investigation. The German investigation also closed in late 2012, with little to show for its trouble. Other contests and lawsuits lumbered on. Despite all the sound and fury, Google continued to operate Street View during those years. Between 2008 and 2010, 600 billion bytes of personal information were collected “illegitimately” around the world, 200 billion of those in the US. The corporation said that it had discontinued personal data collection. Had it? Can anyone say with certainty? Even if it had, the original incursion that was Street View continued unscathed.

Google achieved exactly what Hanke had predicted. Street View’s fundamental audacity, the unprecedented astonishing incursion that drew English villagers into the streets to block a Google camera car, enjoyed six more years of rooting in global consciousness. The corporation’s strategic discipline when it comes to stonewalling, snubbing, and exploiting democracy brought six more years of people using Street View data and six more years to build the tacit case for Google’s inevitability and our helplessness. There were six more years for this simple robbery of decision rights to shade into normalcy and even to be reckoned as “convenient,” “useful,” or “marvelous.”

In Germany, Google allowed residents to request that their homes be blurred in any Street View images. Nearly 250,000 households made opt-out requests in 2009–2010, requiring Google to hire 200 temporary programmers to meet the demand. Google was fined 145,000 euros by the Hamburg data-protection supervisor who had first discovered the Street View illicit data gathering, just short of the 150,000 euro fee he could have imposed. It was the largest fine ever to have been levied by European regulators for privacy concerns.

This helps to explain why, as one 2015 article celebrating the history of Google Maps observes, "Google Maps was attracting all sorts of privacy controversies… people were freaked out.… But that doesn't mean Street View has been tamped down as a project. It's now available in 65 of Google Maps' 200-some countries."

It had been two years since the supposedly irreparable damage that he inflicted on Google and his "clear violation" of policy, yet he continued to be employed at the firm in its YouTube operations. Later that year, he would be one of six inventors on a team led by John Hanke to patent "A System and Method for Transporting Virtual Objects in a Parallel Reality Game."

The invention in which Milner participated was related to a virtual reality game called Ingress, also developed by Hanke and his team at Google. (Hanke would eventually establish his own shop, Niantic Labs, within Google's new Alphabet holding company.) Ingress became a test bed for many of the foundational concepts that reappeared in another "game," Pokémon Go, a prototype of a second phase of surveillance capitalist expansion that we examine closely in Part II. In this next phase, Google's maps are a critical resource for the expansion of digital dispossession from the virtual world to the one that we call "real."

Google’s closely guarded “Ground Truth” project, initiated in 2008 but only publicly revealed just four months after the FCC report in 2012, exemplifies the point. Ground Truth is the “deep map” that contains the detailed “logic of places”: walking paths, goldfish ponds, highway on-ramps, traffic conditions, ferry lines, parks, campuses, neighborhoods, buildings, and more. Getting these details right is a source of competitive advantage in the contest for behavioral surplus accrued from mobile devices. The construction of the deep map draws on public resources such as geographic databases from the US Census Bureau and the US Geological Survey, but what distinguishes Google’s maps from all others is the integration of its exclusive proprietary data from Street View. In other words, data compiled through public investments are augmented with data taken from a unilateral transfer of surplus behavior and decision rights. The composite results are then reclassified as private assets.

Street View cars are likened to Google Search’s early web crawlers, which quietly commandeered web pages for indexing and access in the corporation’s original act of dispossession. By 2012, Street View data also provided street signs and addresses. Soon, Madrigal wrote, “any word that is visible from a road will become a part of Google’s index of the physical world” thanks to Street View. Madrigal’s look at the Ground Truth operation concludes, “The geographic data Google has assembled is not likely to be matched by any other company.… They’ve built this whole playground as an elaborate lure for you.”

As one project leader put it, “The challenge of deciding you’re going to map the world is that you can’t ever stop.” So it was that by 2016, Google’s Street View website celebrated its successful evolution by stating, “We’ve come a long way since our initial U.S. launch in 2007; today we’ve expanded our 360-degree panoramic views to include locations on all seven continents.” Street View’s fleet of surveillance-gathering tools had been augmented to include a wearable backpack, a three-wheeled pedicab, a snowmobile, and a trolley, all of which were designed to capture places that Street View cars could not traverse. Tourist boards and nonprofits were offered the use of the company’s Trekker equipment (the backpack camera) to “collect views of remote and unique places” that were, literally and figuratively, “off the grid.”

If you imagine a satellite above your office then the old resolution could probably make out your desk. The new imagery—where each pixel measures around 31 cm—can now make out what’s on your desk. When you reach this sort of frequency you can begin to add in what we call “pattern of life” analysis. This means looking at activity in terms of movement—not just identification.

Building interiors had eluded Street View and the extraction imperative; few homeowners were likely to invite those cameras indoors. Instead, Cartographer's capabilities were bundled into the larger Street View redirection campaign and pitched to businesses as a way to enhance consumer trust, allay anxiety, and substantially increase revenues.

Google exhorted consumer-facing businesses to “invite customers inside.” With its “Business View,” consumers would be able to see inside thousands of hotels, restaurants, and other destinations. Search listings would feature the new Street View content. Hotel listings would offer a virtual tour of the properties. “Give them the confidence they’re seeking,” Google told its business marketplace, by allowing consumers “to experience your location before they arrive.” Google asserted that virtual tours “double bookings,” and it instated a certification program that enabled businesses to hire a Google-approved freelance photographer to produce images for Street View. These extraordinary new redirection tactics aimed to flip the old pattern. They reframed Street View from an edgy incursion circumventing resistance through stealth to an opulent VIP tent where businesses scrambled for an entry pass.

Street View's redirection and elaboration announced a critical shift in the orientation and ambition of the surveillance program: it would no longer be only about routes, but about routing. We will examine this new episode of dispossession in the chapters that follow. For now, suffice to say that Street View and the larger project of Google Maps illustrate the new and even more ambitious goals toward which this cycle of dispossession would soon point: the migration from an online data source to a real-world monitor to an advisor to an active shepherd—from knowledge to influence to control.

In 2016, for example, the corporation introduced a new Maps app feature called “Driving Mode” that suggests destinations and travel times without users even selecting where they want to go. If you searched for a hammer online, then “Driving Mode” can send you to a hardware store when you buckle up your seat belt. … Predictions about where and why a person might spend money are derived from Google’s exclusive access to behavior surplus and its equally exclusive analytic capabilities: “Eat here.” “Buy this.” Google’s surplus analysis can predict that you are likely to buy an expensive woolen suit, and its real-time location data can trigger the proprietor or advertiser’s real-time prompt, matched to your profile and delivered at the very moment that you are within sight of the flannels, tweeds, and cashmeres. Push and pull, suggest, nudge, cajole, shame, seduce: Google wants to be your copilot for life itself. Each human response to each commercial prompt yields more data to refine into better prediction products. The prompts themselves are bought and paid for in a novel iteration of Google’s online ad markets: real-time, real-world trading in behavioral futures. Your future.

Google’s insistence reflects the authoritarian politics of the extraction imperative as well as the corporation’s own enslavement to the implacable demands of its economics.

Google discovered by chance or intention the source of every mapmaker's power. The great historian of cartography, John B. Harley, said it succinctly: "Maps created empire." They are essential for the effective "pacification, civilization, and exploitation" of territories imagined or claimed but not yet seized in practice. Places and people must be known in order to be controlled. "The very lines on the map," wrote Harley, were a language of conquest in which "the invaders parcel the continent among themselves in designs reflective of their own complex rivalries and relative power." The first US rectangular land survey captured this language perfectly in its slogan: "Order upon the Land." The cartographer is the instrument of power as the author of that order, reducing reality to only two conditions: the map and oblivion. The cartographer's truth crystallizes the message that Google and all surveillance capitalists must impress upon all humans: if you are not on our map, you do not exist.

The project was seen as a precursor to more flexible and less overt forms of wearable computation and surplus capture. John Hanke described its familiar shape in the form of eyewear as suitable for "the early adoption phases" of wearable technology in much the same way that the first automobiles resembled horse-drawn buggies. In other words, the "glasses" were intended to disguise what was in fact unprecedented: "Ultimately we will want these technologies, wherever they are on your body, to be totally optimized based on the job they're doing, not on what is more socially acceptable at that first moment of creation, just because it reminds people of something they've seen in the past."

Privacy advocates protested the “always on” but “undetectable” recording of people and places that eliminates a person’s reasonable expectation of privacy and/or anonymity. They warned of new risks as facial-recognition software is applied to these new data streams and predicted that technologies like Glass would fundamentally alter how people behave in public. By May 2013, a congressional privacy caucus asked CEO Larry Page for assurances on privacy safeguards for Glass, even as a Google conference was held to coach developers on creating apps for the new device. In April 2014 Pew Research announced that 53 percent of Americans thought that smart wearables were “a change for the worse,” including 59 percent of American women.

Google continued to tough it out, waiting for habituation to kick in. That June it announced that Glass would offer the Livestream video-sharing app, enabling Glass users to stream everything around them to the internet in real time. When asked about these controversial and intrusive capabilities in the hands of any owner of the device, Livestream’s CEO reckoned, “Google is ultimately in charge of… setting the rules.” Sergey Brin made it clear that any resistance would be categorically rejected when he told the Wall Street Journal, “People always have a natural aversion to innovation.”

Adaptation began in 2015 with the announcement that Glass would no longer be available. The company said nothing to acknowledge the public’s revulsion or the social issues that Glass had raised. A short blog post announced, “Now we’re ready to put on our big kid shoes and learn how to run… you’ll start to see future versions of Glass when they’re ready.” An eyewear designer was tasked with transforming the look from a futuristic device to something more beautiful.

In July 2017 the redirection phase went public with a blog post introducing a new iteration of Google Glass to the world, now as “Glass Enterprise Edition.” This time there would be no frontal attack on public space. Instead, it was to be a tactical retreat to the workplace—the gold standard of habituation contexts, where invasive technologies are normalized among captive populations of employees. “Workers in many fields, like manufacturing, logistics, field services, and healthcare find it useful to consult a wearable device for information and other resources while their hands are busy,” wrote the project’s leader, and most press accounts lauded the move, citing productivity and efficiency increases in factories that deployed the new Glass. There was little acknowledgment that habituation to Glass at work was most certainly a back door to Glass in our streets or that the intrusive surveillance properties of the device would, with equal certainty, be imposed on the women and men required to use them as a condition of their employment.

Dispossession may be an act of “simple robbery” in theory, but in fact it is a complex, highly orchestrated political and material process that exhibits discernible stages and predictable dynamics. The theory of change exhibited here systematically transfers knowledge and rights from the many to the few in a glorious fog of Page’s “automagic.” It catalogues public contest as the unfortunate but predictable outcry of foolish populations who exhibit a knee-jerk “resistance to change,” wistfully clinging to an irretrievable past while denying an inevitable future: Google’s future, surveillance capitalism’s future. The theory indicates that opposition must simply be weathered as the signature of the first difficult phases of incursion. It assumes that opposition is fleeting, like the sharp yelp of pain when a Novocain needle first pierces the flesh, before numbness sets in.

With Facebook's unflinching assertions that it did not track users, even in the face of many robust facts, specialists grew more frustrated and the public more confused. This appears to have been the point. By denying every accusation and pledging its commitment to user well-being, Facebook secured a solid year and a half with which to habituate the world to its "Like" button, institutionalizing that iconic thumb turned toward the sky as an indispensable prosthetic of virtual communication.

This solid achievement paved the way for the adaptation stage of the dispossession cycle, when in late November 2011, Facebook consented to a settlement with the FTC over charges that it had systematically “deceived consumers by telling them that they could keep their Facebook information private, and then repeatedly allowing it to be shared and made public.” The complaint brought by EPIC and a coalition of privacy advocates in 2009 initiated an FTC investigation that yielded plenty of evidence of the corporation’s broken promises. These included website changes that made private information public, third-party access to users’ personal data, leakage of personal data to third-party apps, a “verified apps” program in which nothing was verified, enabling advertisers to access personal information, allowing access to personal data after accounts were deleted, and violations of the Safe Harbor Framework, which governs data transfers between the United States and the EU. In the parallel universe of surveillance capitalism, each one of these violations was worthy of a five-star rating from the extraction imperative. … The FTC order barred the company from making further privacy misrepresentations, required users’ affirmative consent to new privacy policies, and mandated a comprehensive privacy program to be audited every two years for twenty years. … But Leibowitz was not up against a company; he was up against a new market form with distinct and intractable imperatives whose mandates can be fulfilled only at the expense of user privacy.

In 2012 the company announced it would target ads based on mobile app use, as it worked with Datalogix to determine when online ads result in a real-world purchase. This gambit required mining personal information, including e-mail addresses, from user accounts. In 2012 Facebook also gave advertisers access to targeting data that included users’ e-mail addresses, phone numbers, and website visits, and it admitted that its system scans personal messages for links to third-party websites and automatically registers a “like” on the linked web page. By 2014, the corporation announced that it would be tracking users across the internet using, among its other digital widgets, the “Like” button, in order to build detailed profiles for personalized ad pitches. Its “comprehensive privacy program” advised users of this new tracking policy, reversing every assertion since April 2010 with a few lines inserted into a dense and lengthy terms-of-service agreement. No opt-out privacy option was offered. The truth was finally out: the bug was a feature.

Meanwhile, Google maintained the pledge that had been critical to the FTC’s approval of its 2001 acquisition of the ad-tracking behemoth DoubleClick when it agreed not to combine data from the tracking network with other personally identifiable information in the absence of a user’s opt-in consent. In this case, Google appears to have waited for Facebook to extend the surveillance capitalist frontier and bear the brunt of incursion and habituation. Later, in the summer of 2016, Google crossed that frontier with an announcement that a user’s DoubleClick browsing history “may be” combined with personally identifiable information from Gmail and other Google services. Its promised opt-in function for this new level of tracking was presented with the headline “Some new features for your Google account.” One privacy scholar characterized the move as the final blow to the last “tiny semblance” of privacy on the web. A coalition of privacy groups presented a new complaint to the FTC, implicitly recognizing the logic of the dispossession cycle: “Google has done incrementally and furtively what would plainly be illegal if done all at once.”

Facebook’s IPO in 2012 was notoriously botched when last-minute downward revisions of its sales projections, precipitated by the rapid shift to mobile devices, led to some unsavory dealings among its investment bankers and their clients. But Zuckerberg, Sheryl Sandberg, and their team quickly mastered the nuances of the dispossession cycle, this time to steer the company toward mobile ads. They learned to be skilled and ruthless hunters of behavioral surplus, capturing supplies at scale, evading and resisting law, and upgrading the means of production to improve prediction products.

Surveillance revenues flowed fast and furiously, and the market lavishly rewarded the corporation’s shareholders. By 2017, the Financial Times hailed the company’s 71 percent earnings surge with the headline “Facebook: The Mark of Greatness” as Facebook’s market capitalization rose to just under $500 billion, with 2 billion average monthly active users. Facebook ranked seventh in one important tally of the top 100 companies in the first quarter of 2017, when just a year earlier it hadn’t figured anywhere in the top 100. Advertising, primarily mobile, accounted for nearly every dollar of the company’s revenue in the second quarter of 2017: $9.2 billion of a total $9.3 billion and a 47 percent increase over the prior year.

The Guardian reported that Google and Facebook accounted for one-fifth of global ad spending in 2016, nearly double the figure of 2012, and by one accounting Google and Facebook owned almost 90 percent of the growth in advertising expenditures in 2016.

Among the remaining three of the largest internet companies, Microsoft, Apple, and Amazon, it was Microsoft that first and most decisively turned toward surveillance capitalism as the means to restore its leadership in the tech sector, with the appointment of Satya Nadella to the role of CEO in February 2014. Microsoft had notoriously missed several key opportunities to compete with Google in the search business and develop its targeted advertising capabilities. As early as 2009, when Nadella was a senior vice president and manager of Microsoft’s search business, he publicly criticized the company’s failure to recognize the financial opportunities associated with that early phase of surveillance capitalism. “In retrospect,” he lamented, “it was a terrible decision” to end the search-ad service: “None of us saw the paid-search model in all its glory.” Nadella recognized then that Microsoft’s Bing search engine could not compete with Google because it lacked scale in behavioral surplus capture, the critical factor in the fabrication of high-quality prediction products: “When you look at search… it’s a game of scale. Clearly we don’t have sufficient scale today and that hinders… the quality of the ad relevance which is perhaps the bigger issue we have today."

It concluded that “companies taking advantage of their data have the potential to raise an additional $1.6 trillion in revenue over companies that don’t,” and Nadella was determined to make landfall on the far shores of this rich new space. Microsoft would reap the advantages of its own data, and it would specialize in “empowering” its clients to do the same. Nadella composed a blog to signal the new direction, writing, “The opportunity we have in this new world is to find a way of catalyzing this data exhaust from ubiquitous computing and converting it into fuel for ambient intelligence.” As a video outlining the new “data vision” explains, “Data that was once untapped is now an asset.”

Bing’s search engineering team built its own model of the digital and physical world with a technology it calls Satori: a self-learning system that is adding 28,000 DVDs of content every day. According to the project’s senior director, “It’s mind-blowing how much data we have captured over the last couple of years. The line would extend to Venus and you would still have 7 trillion pixels left over.” All those pixels were being put to good use. In its October 2015 earnings call, the company announced that Bing had become profitable for the first time, thanks to around $1 billion in search ad revenue from the previous quarter.

According to an analysis by the Electronic Frontier Foundation (EFF), even users who opted out of Cortana were subject to an “unprecedented” amount of information capture, including text, voice, and touch input; web tracking; and telemetry data on their general use, programs, session duration, and more. The EFF also found that the company chose to hold security functions hostage to personal data flows, claiming that security updates for the operating system would not function properly if users chose to limit location reporting.

The unprecedented successes of Google, Facebook, and then Microsoft exerted a palpable magnetism on the global economy, especially in the US, where the politics of lawlessness were most firmly entrenched. It did not take long before companies from established sectors with roots far from Silicon Valley demonstrated their determination to compete for surveillance revenues. Among the first in this second wave were the telecom and cable companies that provide broadband service to millions of individuals and households. … “Armed with their expansive view over the entire web, internet providers may even be in a position to out-Facebook Facebook, or out-Google Google,” observed the Washington Post.

By the fall of 2014, Verizon's stealthy new claim on free raw material was outed by Jacob Hoffman-Andrews, a technologist with the Electronic Frontier Foundation. An article in Wired called attention to Hoffman-Andrews' analysis of Verizon's surveillance program and his additional discovery that AT&T was using a similar tracking ID. The article quoted a Verizon spokesperson admitting, "There's no way to turn it off." Hoffman-Andrews observed that even when customers opt out of Verizon's targeted ads, its tracking ID persists, as the corporation bypasses or overrides all signals of a user's intentions, including the Do Not Track setting, Incognito and other private browsing modes, and cookie deletion. The ID is then broadcast to every "unencrypted website a Verizon customer visits from a mobile device. It allows third-party advertisers and websites to assemble a deep, permanent profile of visitors' web browsing habits without their consent." Alarmed by the threat of fresh competition, Google, posing as a privacy advocate, launched a campaign for a new internet protocol that would prevent "header injections" such as Verizon's PrecisionID.

In May 2015 Verizon agreed to purchase AOL for $4.4 billion. As many analysts quickly appreciated, the real attraction of AOL was its CEO, Tim Armstrong, the first head of advertising sales at Google and the man who oversaw its transition from Madison Avenue–style advertising to AdWords' breakthrough discoveries. He was president of Google's Americas sales division when, like Sheryl Sandberg before him, Armstrong left Google for AOL in 2009 with a profound grasp of AdWords' surveillance DNA and the determination to rescue AOL's balance sheet with surveillance capitalism gene therapy.

The FCC settlement with Verizon was another gloomy example of a public institution outmatched by the velocity and resources of a determined surveillance capitalist. In March 2016, long after the announcement of Verizon's tactical redirection, the FCC reached a $1.35 million settlement with Verizon over its hidden ID privacy violations. Although Verizon agreed to relaunch its cookie on an opt-in basis, the settlement did not extend to AOL's advertising network, which is where the action had moved. Verizon's burgeoning new supply routes would remain unchallenged.

In light of the high-stakes dispossession competition already under way among key ISPs, it is not surprising that the proposal quickly became a political lightning rod. ISPs, their lobbyists, policy advisors, and political allies lined up to kill the effort, stating that ISPs' competitive prospects would be unfairly impeded: "Telecom companies are against this proposal, arguing it puts them on an unequal footing with other internet companies that collect data on users, like Google.…" In October 27, 2016, FCC commissioners in a 3–2 vote delivered a landmark ruling in favor of, in this case, consumer protection on the internet. It was a historic day not only in the young life of surveillance capitalism but also in the venerable and long life of the FCC, an agency that had never before passed such online protections.

In a chilling epilogue to this chapter in the history of surveillance capitalism, on March 28, 2017, a newly elected Republican Congress voted in favor of a resolution to overturn the broadband privacy regulations over which the FCC had struggled just months earlier. The rules had required cable and phone companies to obtain meaningful consent before using personal information for ads and profiling. The companies understood, and they persuaded Republican senators, that the principle of consent would strike a serious blow to the foundational mechanisms of the new capitalism: the legitimacy of unilateral surplus dispossession, ownership rights to surplus, decision rights over surplus, and the right to lawless space for the prosecution of these activities. To this end the resolution also prevented the FCC from seeking to establish similar protections in the future.

The reversal meant that although federal laws protected the privacy of a telephone call, the same information transmitted by internet immediately enters the ISPs’ surplus supply chains. This roust finally signaled the end of the myth of “free.” The Faustian pact that had been sold to the world’s internet users posed surveillance as the bitter price of free services such as Google’s Search and Facebook’s social network. This obfuscation is no longer tenable, as every consumer who pays his or her monthly telecom bill now also purchases the privilege of a remote and abstract but nevertheless rapacious digital strip search.

As the Wall Street Journal reports, new startups such as Affirm, LendUp, and ZestFinance "use data from sources such as social media, online behavior and data brokers to determine the creditworthiness of tens of thousands of U.S. consumers who don't have access to loans," more evidence that decision rights and the privacy they enable have become luxuries that too many people cannot afford.

Such methods were originally developed for markets in Africa to help the "unbanked"—people without established credit—to qualify for loans. One lending group interviewed potential customers in low-income countries and concluded that it would be easy to exploit the already beleaguered poor: "Most said they had no problem sharing personal details in exchange for much-needed funds." But these app-based lending startups are typically developed and funded in Silicon Valley, and it is therefore not surprising that the same techniques have become part of a wider trend of exploiting American families that have been economically hollowed out by the financial crisis and neoliberalism's austerity medicine.

Another example of surveillance-as-a-service is a firm that sells deep vetting of potential employees and tenants to employers and landlords. For instance, a prospective tenant receives a demand from her potential landlord that requires her to grant full access to all social media profiles. The service then "scrapes your site activity," including entire conversation threads and private messages, runs it through natural language processing and other analytic software, and finally spits out a report that catalogues everything from your personality to your "financial stress level," including exposing protected status information such as pregnancy and age. There is no opportunity for affected individuals to view or contest information.

Another example is Safegraph, a company that partners with all those apps that are tracking your behavior to amass “high precision/low false positive” data collected “in background from large populations.” According to the Washington Post, the company collected 17 trillion location markers from 10 million smartphones in November 2016 alone, data that were sold to two university researchers, among others, for a detailed study of political influences on patterns of family behavior on Thanksgiving Day that year. Despite the widely employed euphemisms of “anonymization” and “deidentification,” Safegraph tracks individual devices and the movement of their owners throughout the day, producing data that are sufficiently granular to be able to identify individuals’ home locations.

I have suggested that the dangers of surveillance capitalism cannot be fully grasped through either the lens of privacy or of monopoly. In Chapter 6 I offer a new way of thinking about danger. The threats we face are even more fundamental as surveillance capitalists take command of the essential questions that define knowledge, authority, and power in our time: Who knows? Who decides? Who decides who decides?

On December 4, 1492, Columbus escaped the onshore winds that had prevented his departure from the island that we now call Cuba. Within a day he dropped anchor off the coast of a larger island known to its people as Quisqueya or Bohio, setting into motion what historians call the “conquest pattern.” It’s a design that unfolds in three phases: the invention of legalistic measures to provide the invasion with a gloss of justification, a declaration of territorial claims, and the founding of a town to legitimate and institutionalize the conquest. The sailors could not have imagined that their actions that day would write the first draft of a pattern whose muscle and genius would echo across space and time to a digital twenty-first century.

According to the philosopher of language John Searle, a declaration is a particular way of speaking and acting that establishes facts out of thin air, creating a new reality where there was nothing. Here is how it works: sometimes we speak to simply describe the world—“you have brown eyes”—or to change it—“Shut the door.” A declaration combines both, asserting a new reality by describing the world as if a desired change were already true: “All humans are created equal.” “They are yours to command.” As Searle writes, “We make something the case by representing it as being the case.”

The six declarations laid the foundation for the wider project of surveillance capitalism and its original sin of dispossession. They must be defended at any cost because each declaration builds on the one before it. If one falls, they all fall: • We claim human experience as raw material free for the taking.

Thus, the age of surveillance capitalism was inaugurated with six declarations that defined it as an age of conquest. Surveillance capitalism succeeded by way of aggressive declaration, and its success stands as a powerful illustration of the invasive character of declarative words and deeds, which aim to conquer by imposing a new reality. These twenty-first-century invaders do not ask permission; they forge ahead, papering the scorched earth with faux-legitimation practices. Instead of cynically conveyed monarchical edicts, they offer cynically conveyed terms-of-service agreements whose stipulations are just as obscured and incomprehensible. They build their fortifications, fiercely defending their claimed territories, while gathering strength for the next incursion. Eventually, they build their towns in intricate ecosystems of commerce, politics, and culture that declare the legitimacy and inevitability of all that they have accomplished.

Its declarative victories have been the means through which it amassed world-historic concentrations of knowledge and power. These are the bulwarks that enable its continued progress. Schmidt has occasionally revealed something like this point. When describing "modern technology platforms," he writes that "almost nothing, short of a biological virus, can scale as quickly, efficiently, or aggressively as these technology platforms, and this makes the people who build, control, and use them powerful too."

On the strength of its unprecedented concentrations of knowledge and power, surveillance capitalism achieves dominance over the division of learning in society—the axial principle of social order in an information civilization. This development is all the more dangerous because it is unprecedented. It cannot be reduced to known harms and therefore does not easily yield to known forms of combat.

Any consideration of the division of learning must resolve these dilemmas expressed in three essential questions. The first question is “Who knows?” This is a question about the distribution of knowledge and whether one is included or excluded from the opportunity to learn. The second question is “Who decides?” This is a question about authority: which people, institutions, or processes determine who is included in learning, what they are able to learn, and how they are able to act on their knowledge. What is the legitimate basis of that authority? The third question is “Who decides who decides?” This is a question about power. What is the source of power that undergirds the authority to share or withhold knowledge?

The young manager would ultimately find his answers, but they were not what either of us had hoped for. Even as the pulp mill workers struggled and often triumphed, Hayek’s worldview was taking hold at the highest policy levels and Jensen’s operational disciplines were finding an eager welcome on Wall Street, which quickly learned to impose them on every public company. The result was a cost-down business model oriented to its Wall Street audience, which insisted on automating and exporting jobs rather than investing in the digital skills and capabilities of the US worker. The answer to the question Who knows? was that the machine knows, along with an elite cadre able to wield the analytic tools to troubleshoot and extract value from information. The answer to Who decides? was a narrow market form and its business models that decide. Finally, in the absence of a meaningful double movement, the answer to Who decides who decides? defaults entirely to financial capital bound to the disciplines of shareholder-value maximization.

It is not surprising that nearly forty years later, a Brookings Institution report laments that millions of US workers are “shut out of decent middle-skill opportunities” in the face of “rapid digitalization.” The report exhorts companies to “invest urgently in IT upskilling strategies for incumbent workers, knowing that digital skills represent a key channel of productivity gains.” How different might our society be if US businesses had chosen to invest in people as well as in machines?

Most companies opted for the smart machine over smart people, producing a well-documented pattern that favors substituting machines and their algorithms for human contributors in a wide range of jobs. By now, these include many occupations far from the factory floor. This results in what economists call “job polarization,” which features some high-skill jobs and other low-skill jobs, with automation displacing most of the jobs that were once “in the middle.” And although some business leaders, economists, and technologists describe these developments as necessary and inevitable consequences of computer-based technologies, research shows that the division of learning in the economic domain reflects the strength of neoliberal ideology, politics, culture, and institutional patterns. For example, in continental and northern Europe, where key elements of the double movement have survived in some form, job polarization is moderated by substantial investments in workforce education that produce a more inclusive division of learning as well as high-quality innovative products and services.

A wholly new electronic text now extends far beyond the confines of the factory or office. Thanks to our computers, credit cards, and phones, and the cameras and sensors that proliferate in public and private spaces, just about everything we now do is mediated by computers that record and codify the details of our daily lives at a scale that would have been unimaginable only a few years ago. We have reached the point at which there is little that is omitted from the continuous accretion of this new electronic text. In later chapters we review many illustrations of the new electronic text as it spreads silently but relentlessly, like a colossal oil slick engulfing everything in its path: your breakfast conversation, the streets in your neighborhood, the dimensions of your living room, your run in the park.

The result is that both the world and our lives are pervasively rendered as information. Whether you are complaining about your acne or engaging in political debate on Facebook, searching for a recipe or sensitive health information on Google, ordering laundry soap or taking photos of your nine-year-old, smiling or thinking angry thoughts, watching TV or doing wheelies in the parking lot, all of it is raw material for this burgeoning text. Information scholar Martin Hilbert and his colleagues observe that even the foundational elements of civilization, including “language, cultural assets, traditions, institutions, rules, and laws… are currently being digitized, and for the first time, explicitly put into visible code,” then returned to society through the filter of “intelligent algorithms” deployed to govern a rapidly multiplying range of commercial, governmental, and social functions. The essential questions confront us at every turn: Who knows? Who decides? Who decides who decides?

In other words, the division of labor was summoned into society at the beginning of the twentieth century by the rapidly changing circumstances of the first modernity’s new individuals, discussed in Chapter 2. It was an essential response to their new “conditions of existence.” As people like my great-grandparents joined the migration to a modern world, the old sources of meaning that had bonded communities across space and time melted away. What would hold society together in the absence of the rules and rituals of clan and kin? Durkheim’s answer was the division of labor.

The most remarkable effect of the division of labor is not that it increases output of functions divided, but that it renders them solidary. Its role… is not simply to embellish or ameliorate existing societies, but to render societies possible which, without it, would not exist.… It passes far beyond purely economic interests, for it consists in the establishment of a social and moral order sui generis.

Durkheim’s vision was neither sterile nor naive. He recognized that things can take a dark turn and often do, resulting in what he called an “abnormal” (sometimes translated as “pathological”) division of labor that produces social distance, injustice, and discord in place of reciprocity and interdependency. In this context, Durkheim singled out the destructive effects of social inequality on the division of labor in society, especially what he viewed as the most dangerous form of inequality: extreme asymmetries of power that make “conflict itself impossible” by “refusing to admit the right of combat.” Such pathologies can be cured only by a politics that asserts the people’s right to contest, confront, and prevail in the face of unequal and illegitimate power over society. In the late nineteenth century and most of the twentieth century, that contest was led by labor and other social movements that asserted social equality through institutions such as collective bargaining and public education.

The transformation that we witness in our time echoes these historical observations as the division of learning follows the same migratory path from the economic to the social domain once traveled by the division of labor. Now the division of learning “passes far beyond purely economic interests,” for it establishes the basis for our social order and its moral content.

The division of learning is to us, members of the second modernity, what the division of labor was to our grandparents and great-grandparents, pioneers of the first modernity. In our time the division of learning emerges from the economic sphere as a new principle of social order and reflects the primacy of learning, information, and knowledge in today’s quest for effective life. And just as Durkheim warned his society a century ago, today our societies are threatened as the division of learning drifts into pathology and injustice at the hands of the unprecedented asymmetries of knowledge and power that surveillance capitalism has achieved.

Surveillance capitalism’s command of the division of learning in society begins with what I call the problem of the two texts. The specific mechanisms of surveillance capitalism compel the production of two “electronic texts,” not just one. When it comes to the first text, we are its authors and readers. This public-facing text is familiar and celebrated for the universe of information and connection it brings to our fingertips. Google Search codifies the informational content of the world wide web. Facebook’s News Feed binds the network. Much of this public-facing text is composed of what we inscribe on its pages: our posts, blogs, videos, photos, conversations, music, stories, observations, “likes,” tweets, and all the great massing hubbub of our lives captured and communicated.

As legal scholar Frank Pasquale describes it, “The decisions at the Googleplex are made behind closed doors… the power to include, exclude, and rank is the power to ensure which public impressions become permanent and which remain fleeting.… Despite their claims of objectivity and neutrality, they are constantly making value-laden, controversial decisions. They help create the world they claim to merely ‘show’ us.” When it comes to the shadow text, surveillance capitalism’s laws of motion compel both its secrecy and its continuous growth. We are the objects of its narratives, from whose lessons we are excluded. As the source from which all the treasure flows, this second text is about us, but it is not for us. Instead, it is created, maintained, and exploited outside our awareness for others’ benefit.

Surveillance capitalism’s ability to corrupt and control these texts produces unprecedented asymmetries of knowledge and power that operate precisely as Durkheim had feared: the relatively free rein accorded to this market form and the innately illegible character of its action have enabled it to impose substantial control over the division of learning outside of our awareness and without means of combat. When it comes to the essential questions, surveillance capital has gathered the power and asserted the authority to supply all the answers. However, even authority is not enough. Only surveillance capital commands the material infrastructure and expert brainpower to rule the division of learning in society.

Information is digital, but its volume exceeds our ability to discern its meaning. As the solution to this problem, information scholar Martin Hilbert counsels, “The only option we have left to make sense of all the data is to fight fire with fire,” using “artificially intelligent computers” to “sift through the vast amounts of information.… Facebook, Amazon, and Google have promised to… create value out of vast amounts of data through intelligent computational analysis.” The rise of surveillance capitalism necessarily turns Hilbert's advice into a dangerous proposition. Although he does not mean to, Hilbert merely confirms the privileged position of the surveillance capitalists and the asymmetrical power that enables them to bend the division of learning to their interests.

Google's asymmetrical power draws on all the social sources that we have considered: its declarations, its defensive fortifications, its exploitation of law, the legacy of surveillance exceptionalism, the burdens of second-modernity individuals, and so on. But its power would not be operational without the gargantuan material infrastructure that surveillance revenues have bought. Google is the pioneer of “hyperscale,” considered to be “the largest computer network on Earth.” Hyperscale operations are found in high-volume information businesses such as telecoms and global payments firms, where data centers require millions of “virtual servers” that exponentially increase computing capabilities without requiring substantial expansion of physical space, cooling, or electrical power demands. The machine intelligence at the heart of Google's formidable dominance is described as “80 percent infrastructure,” a system that comprises custom-built, warehouse-sized data centers spanning 15 locations and, in 2016, an estimated 2.5 million servers in four continents.

Google is known as a “full stack AI company” that uses its own data stores “to train its own algorithms running on its own chips deployed on its own cloud.” Its dominance is further strengthened by the fact that machine learning is only as intelligent as the amount of data it has to train on, and Google has the most data. By 2013, the company understood that its shift into the “neural networks” that define the current frontier of artificial intelligence would substantially increase computational demands and require a doubling of its data centers. As Urs Hölzle, Google’s senior vice president of technical infrastructure, put it, “The dirty secret behind [AI] is that they require an insane number of computations to just actually train the network.” If the company had tried to process the growing computational workload with traditional CPUs, he explained, “We would have had to double the entire footprint of Google—data centers and servers—just to do three minutes or two minutes of speech recognition per Android user per day.”

In 2016 it announced the development of a new chip for “deep learning inference” called the tensor processing unit (TPU). The TPU would dramatically expand Google’s machine intelligence capabilities, consume only a fraction of the power required by existing processors, and reduce both capital expenditure and the operational budget, all while learning more and faster.

The concentration of AI talent at Google reflects a larger trend. In 2017, US companies are estimated to have allocated more than $650 million to fuel the AI talent race, with more than 10,000 available positions at top employers across the country. The top five tech companies have the capital to crowd out competitors: startups, universities, municipalities, established corporations in other industries, and less wealthy countries. In Britain, university administrators are already talking about a “missing generation” of data scientists. The huge salaries of the tech firms have lured so many professionals that there is no one left to teach the next generation of students. As one scholar described it, “The real problem is these people are not dispersed through society. The intellect and expertise is concentrated in a small number of companies.”

In the absence of a robust double movement in which democratic institutions and civil society tether raw information capitalism to the people's interests—however imperfectly—we are thrown back on the market form of the surveillance capitalist companies in this most decisive of contests over the division of learning in society.

Building on Simitis's work, Berkeley's Paul M. Schwartz warned in 1989 that computerization would transform the delicate balance of rights and obligations upon which privacy law depends: “Today the enormous amounts of personal data available in computers threaten the individual in a way that renders obsolete much of the previous legal protection.” Most important, Schwartz foresaw that the scale of the still-emerging crisis would impose risks that exceed the scope of privacy law: “The danger that the computer poses is to human autonomy. The more that is known about a person, the easier it is to control him. Insuring the liberty that nourishes democracy requires a structuring of societal use of information and even permitting some concealment of information."

Both Simitis and Schwartz sensed the ascent of the division of learning as the axial principle of a new computational societal milieu, but they could not have anticipated the rise of surveillance capitalism and its consequences. Although the explosive growth of the information continent shifts a crucial axis of the social order from a twentieth-century division of labor to a twenty-first-century division of learning, it is surveillance capitalists who command the field and unilaterally lay claim to a disproportionate share of the decision rights that shape the division of learning in society.

The “invasion of privacy” is now a predictable dimension of social inequality, but it does not stand alone. It is the systematic result of a “pathological” division of learning in society in which surveillance capitalism knows, decides, and decides who decides. Demanding privacy from surveillance capitalists or lobbying for an end to commercial surveillance on the internet is like asking Henry Ford to make each Model T by hand or asking a giraffe to shorten its neck. Such demands are existential threats. They violate the basic mechanisms and laws of motion that produce this market leviathan's concentrations of knowledge, power, and wealth.

This unprecedented concentration of knowledge produces an equally unprecedented concentration of power: asymmetries that must be understood as the unauthorized privatization of the division of learning in society. This means that powerful private interests are in control of the definitive principle of social ordering in our time, just as Durkheim warned of the subversion of the division of labor by the powerful forces of industrial capital a century ago. As things currently stand, it is the surveillance capitalist corporations that know. It is the market form that decides. It is the competitive struggle among surveillance capitalists that decides who decides.

The titanic power struggles of the twentieth century were between industrial capital and labor, but the twenty-first century finds surveillance capital pitted against the entirety of our societies, right down to each individual member. The competition for surveillance revenues bears down on our bodies, our homes, and our cities in a battle for power and profit as violent as any the world has seen. Surveillance capitalism cannot be imagined as something “out there” in factories and offices. Its aims and effects are here… are us.

Ours is not simply a case of being ambushed and outgunned. We were caught off guard because there was no way that we could have imagined these acts of invasion and dispossession, any more than the first unsuspecting Taíno cacique could have foreseen the rivers of blood that would flow from his inaugural gesture of hospitality toward the hairy, grunting, sweating men, the adelantados who appeared out of thin air waving the banner of the Spanish monarchs and their pope as they trudged across the beach. Why have we been slow to recognize the “original sin of simple robbery” at the heart of this new capitalism? Like the Tainos, we faced something altogether new to our story: the unprecedented. And, like them, we risk catastrophe when we assess new threats through the lens of old experience.

On the “supply side,” surveillance capitalists deftly employed the entire arsenal of the declaration to assert their authority and legitimacy in a new and undefended digital world. They used declarations to take without asking. They camouflaged their purpose with illegible machine operations, moved at extreme velocities, sheltered secretive corporate practices, mastered rhetorical misdirection, taught helplessness, purposefully misappropriated cultural signs and symbols associated with the themes of the second modernity—empowerment, participation, voice, individualization, collaboration—and baldly appealed to the frustrations of second-modernity individuals thwarted in the collision between psychological yearning and institutional indifference.

Now the story of surveillance capitalism moves in this new direction. In Part II, I invite you to rekindle your sense of astonishment as we follow the trail of this second economic imperative defined by the prediction of human behavior. The prediction imperative enlarges the complexity of surplus operations as economies of scale are joined by economies of scope and economies of action. These new disciplines drive surveillance capitalism far into the intimate reaches of our daily lives and deep into our personalities and our emotions. Ultimately, they compel the development of highly inventive but resolutely secret new means to interrupt and modify our behavior for the sake of surveillance revenues. These operations challenge our elemental right to the future tense, which is the right to act free of the influence of illegitimate forces that operate outside our awareness to influence, modify, and condition our behavior. We grow numb to these incursions and the ways in which they deform our lives. We succumb to the drumbeat of inevitability, but nothing here is inevitable. Astonishment is lost but can be found again.

In 2015, during a session at the winter playground for neoliberals—and increasingly surveillance capitalists—Schmidt was asked for his thoughts about the future of the internet. Sitting alongside his former Google colleagues Sheryl Sandberg and Marissa Mayer, he did not hesitate to share his belief that “The internet will disappear. There will be so many IP addresses… so many devices, sensors, things that you are wearing, things that you are interacting with, that you won’t even sense it. It will be part of your presence all the time. Imagine you walk into a room and the room is dynamic.” The audience gasped in astonishment, and shortly thereafter, headlines around the world exploded in shock at the former Google CEO’s pronouncement that the end of the internet was at hand.

Schmidt was, in fact, merely paraphrasing computer scientist Mark Weiser’s seminal 1991 article, “The Computer for the 21st Century,” which has framed Silicon Valley’s technology objectives for nearly three decades. Weiser introduced what he called “ubiquitous computing” with two legendary sentences: “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” He described a new way of thinking “that allows the computers themselves to vanish into the background.… Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as taking a walk in the woods.”

Weiser understood that the virtual world could never be more than a shadow land no matter how much data it absorbs: “Virtual reality is only a map, not a territory. It excludes desks, offices, other people… weather, trees, walks, chance encounters and, in general, the infinite richness of the universe.” He wrote that virtual reality “simulates” the world rather than “invisibly enhancing the world that already exists.” In contrast, ubiquitous computing would infuse that real world with a universally networked apparatus of silent, “calm,” and voracious computing. Weiser refers to this apparatus as the new “computing environment” and delights in the possibilities of its limitless knowledge, such as knowing “the suit you looked at for a long time last week because it knows both of your locations, and it can retroactively find the designer’s name even though that information did not interest you at the time.”

For surveillance capitalists, this transition is not a choice. Surveillance profits awakened intense competition over the revenues that flow from new markets for future behavior. Even the most sophisticated process of converting behavioral surplus into products that accurately forecast the future is only as good as the raw material available for processing. Surveillance capitalists therefore must ask this: what forms of surplus enable the fabrication of prediction products that most reliably foretell the future? This question marks a critical turning point in the trial-and-error elaboration of surveillance capitalism. It crystallizes a second economic imperative—the prediction imperative—and reveals the intense pressure that it exerts on surveillance capitalist revenues.

The first wave of prediction products enabled targeted online advertising. These products depended upon surplus derived at scale from the internet. I have summarized the competitive forces that drive the need for surplus at scale as the “extraction imperative.” Competition for surveillance revenues eventually reached a point at which the volume of surplus became a necessary but insufficient condition for success. The next threshold was defined by the quality of prediction products. In the race for higher degrees of certainty, it became clear that the best predictions would have to approximate observation. The prediction imperative is the expression of these competitive forces.

The shift toward economies of scope defines a new set of aims: behavioral surplus must be vast, but it must also be varied. These variations are developed along two dimensions. The first is the extension of extraction operations from the virtual world into the “real” world, where we actually live our actual lives. Surveillance capitalists understood that their future wealth would depend upon new supply routes that extend to real life on the roads, among the trees, throughout the cities. Extension wants your bloodstream and your bed, your breakfast conversation, your commute, your run, your refrigerator, your parking space, your living room.

Economies of scope also proceed along a second dimension: depth. The drive for economies of scope in the depth dimension is even more audacious. The idea here is that highly predictive, and therefore highly lucrative, behavioral surplus would be plumbed from intimate patterns of the self. These supply operations are aimed at your personality, moods, and emotions, your lies and vulnerabilities. Every level of intimacy would have to be automatically captured and flattened into a tidal flow of data points for the factory conveyor belts that proceed toward manufactured certainty.

These interventions are designed to enhance certainty by doing things: they nudge, tune, herd, manipulate, and modify behavior in specific directions by executing actions as subtle as inserting a specific phrase into your Facebook news feed, timing the appearance of a BUY button on your phone, or shutting down your car engine when an insurance payment is late.

As the prediction imperative gathers force, it gradually becomes clear that extraction was the first phase of a far-more-ambitious project. Economies of action mean that real-world machine architectures must be able to know as well as to do. Extraction is not enough; now it must be twinned with execution. The extraction architecture is combined with a new execution architecture, through which hidden economic objectives are imposed upon the vast and varied field of behavior.

Gradually, as surveillance capitalism’s imperatives and the material infrastructures that perform extraction-and-execution operations begin to function as a coherent whole, they produce a twenty-first-century “means of behavioral modification.” The aim of this undertaking is not to impose behavioral norms, such as conformity or obedience, but rather to produce behavior that reliably, definitively, and certainly leads to desired commercial results. The research director of Gartner, the well-respected business advisory and research firm, makes the point unambiguously when he observes that mastery of the “internet of things” will serve as “a key enabler in the transformation of business models from ‘guaranteed levels of performance’ to ‘guaranteed outcomes.’”

Finally, I want to underscore that although it may be possible to imagine something like the “internet of things” without surveillance capitalism, it is impossible to imagine surveillance capitalism without something like the “internet of things.” Every command arising from the prediction imperative requires this pervasive real-world material “knowing and doing” presence. The new apparatus is the material expression of the prediction imperative, and it represents a new kind of power animated by the economic compulsion toward certainty. Two vectors converge in this fact: the early ideals of ubiquitous computing and the economic imperatives of surveillance capitalism.

Futuristic as this may sound, the vision of individuals and groups as so many objects to be continuously tracked, wholly known, and shunted this way or that for some purpose of which they are unaware has a history. It was coaxed to life nearly sixty years ago under the warm equatorial sun of the Galapagos Islands, when a giant tortoise stirred from her torpor to swallow a succulent chunk of cactus into which a dedicated scientist had wedged a small machine.

It was a time when scientists reckoned with the obstinacy of free-roaming animals and concluded that surveillance was the necessary price of knowledge. Locking these creatures in a zoo would only eliminate the very behavior that scientists wanted to study, but how were they to be surveilled? The solutions once concocted by scholars of elk herds, sea turtles, and geese have been refurbished by surveillance capitalists and presented as an inevitable feature of twenty-first-century life on Earth. All that has changed is that now we are the animals.

MacKay stressed that the transmission and monitoring of sensor data were only part of the story. The route was not enough; it had to be the routing. He argued for a “reverse process” of telestimulation that would not only monitor behavior but also reveal how it could be modified and optimized, providing what he regarded as “a remote dialogue between the subject and the experimenter.”

Paradiso’s brilliant group of data scientists, engineers, musicians, and artists reconceives the world through the lens of Google Search by applying the same disciplines that mastered the web—datafication, indexing, browsing, and searching—to master reality itself. Without “ubiquitous sensate environments,” Paradiso writes, “the cognitive engines of this everywhere-enabled world are deaf, dumb, and blind, and can’t respond relevantly to the real-world events that they aim to augment.” In other words, ubiquitous computing is meaningless without the ubiquitous sensing that conveys the experience for computation.

Paradiso and his colleagues wrestled with the paradox of, on the one hand, proliferating sensor data in nearly every environment—from smartphones to home devices to streets to cameras to cars—and, on the other hand, the difficulties involved in integrating sensor-generated data flows and producing meaningful analyses. Their answer was “DoppelLab,” a digital platform for combining and visually representing sensor data. The idea is to transform any physical space, from the interior of an office building to an entire city, into a “browse-able environment” where you can see and hear everything going on in that space as it flows from thousands or billions or trillions of sensors. Just as browsers like Netscape first “gave us access to the mass of data contained on the internet, so will software browsers enable us to make sense of the flood of sensor data that is on the way.”

According to Paradiso and his coauthor, Gershon Dublon, the next great technological challenge is "context aggregation," which means the ability to assemble rapidly expanding sensor information into new "applications." The idea is that every physical space and every trace of behavior within that space-bees buzzing, your smile, the temperature fluctuations in my closet, their breakfast conversation, the swoosh of the trees-will be "informated" (translated into information). Spaces can be aggregated into a seamless flow of searchable information, sights, and sounds in much the same way that Google once aggregated web pages for indexing and searching: "This shift will create a seamless nervous system that covers the planet-and one of the main challenges for the computing community now is how to merge the rapidly evolving 'omniscient' electronic sensoria onto human perception."

For all their brilliance, these creative scientists appear to be unaware of the restless economic order eager to commandeer their achievements under the flag of surveillance revenues. Paradiso does not reckon with the translation of his paradise of omniscience into the realpolitik of surveillance capitalism as the prediction imperative insists on surplus culled from these new flows and surveillance capitalists fill the front seats of the classroom of digital omniscience.

Waning levels of government leadership and funding for “ubiquitous computing” leave the technology companies to lead in basic research and applications, each vying to become “the Google” of the new apparatus and its architectures of extraction and execution. Despite the radical prospects of the ubiquitous connected sensate computational apparatus and the often-repeated claim “It will change everything,” technology firms in the US have, thus far, continued their run of relative lawlessness, unimpeded by any comprehensive social or regulatory vision.

In place of “policy” or a “social contract,” it is capitalism, and increasingly surveillance capitalism, that shapes the action. New behavioral futures markets and “targeted applications” are eagerly awaited. As Microsoft’s director of the machine intelligence platform for integrating and analyzing data from the “internet of things” says, “The part that’s equally cool and creepy is what happens after everybody and their competitor gets on board with smart devices: a big secondary market for data… a secondary revenue source.” These markets, he explains, are “just like Google or Facebook’s” markets for targeted advertising." An IBM report concurs: “Thanks to the internet of things, physical assets are turning into participants in real-time global digital markets. The countless types of assets around us will become as easily indexed, searched and traded as any online commodity.… We call this the ‘liquification of the physical world.’”

Unstructured data cannot merge and flow in the new circuits of liquefied assets bought and sold. They are friction. Green fixes on the declarative term that simultaneously names the problem and justifies its solution: dark data. The message we saw honed in the online world—“If you're not in the system, you don't exist”—is refined for this new phase of dispossession. Because the apparatus of connected things is intended to be everything, any behavior of human or thing absent from this push for universal inclusion is dark: menacing, untamed, rebellious, rogue, out of control. The stubborn expanse of dark data is framed as the enemy of IBM's and its customers' ambitions. … The tension is that no thing counts until it is rendered as behavior, translated into electronic data flows, and channeled into the light as observable data. Everything must be illuminated for counting and herding.

In this way the notion of “dark data” handily becomes the “data exhaust” of ubiquitous computing. It provides the moral, technical, commercial, and legal rationale for powerful systems of machine intelligence that can capture and analyze behaviors and conditions never intended for a public life. For those who seek surveillance revenues, dark data represent lucrative and necessary territories in the dynamic universal jigsaw constituted by surveillance capitalism's urge toward scale, scope, and action. Thus, the technology community casts dark data as the intolerable “unknown unknown” that threatens the financial promise of the “internet of things.”

It is therefore understandable that Green portrays machine intelligence—and specifically IBM's anthropomorphized artificial intelligence system called “Watson”—as the authoritative savior of an apparatus threatened by waste and incomprehensibility. Machine intelligence is referred to as “cognitive computing” at IBM, presumably to avoid the uneasy connotations of inscrutable power associated with words like machine and artificial.

As the prediction imperative pulls supply operations into the real world, product or service providers in established sectors far from Silicon Valley are enthralled by the prospects of surveillance revenues. For example, the CEO of Allstate Insurance wants to be like Google: “There are lots of people who are monetizing data today. You get on Google, and it seems like it's free. It's not free. You're giving them information; they sell your information. Could we, should we, sell this information we get from people driving around to various people and capture some additional profit source…? It's a long-term game.”

Auto insurers are besieged by consultants and would-be technology partners who proffer surveillance capitalistic strategies that promise a new chapter of commercial success. “Uncertainty will be strongly reduced,” intones a McKinsey report on the future of the insurance industry. “This leads to demutualization and a focus on predicting and managing individual risks rather than communities.” A report by Deloitte’s Center for Financial Services counsels “risk minimization”—a euphemism for guaranteed outcomes—through monitoring and enforcing policyholder behavior in real time, an approach called “behavioral underwriting.” “Insurers can monitor policyholder behavior directly,” Deloitte advises, by “recording the times, locations, and road conditions when they drive, whether they rapidly accelerate or drive at high or even excessive speeds, how hard they brake, as well as how rapidly they make turns and whether they use their turn signals.” Telematics produce continuous data flows, so real-time behavioral surplus can replace the traditional “proxy factors,” such as demographic information, that had previously been used to calculate risk. This means that surplus must be both plentiful (economies of scale) and varied (economies of scope) in both range and depth.

As certainty replaces uncertainty, premiums that once reflected the necessary unknowns of everyday life can now rise and fall from millisecond to millisecond, informed by the precise knowledge of how fast you drive to work after an unexpectedly hectic early morning caring for a sick child or if you perform wheelies in the parking lot behind the supermarket. “We know that 16-year old drivers have a whole lot of accidents… but not every 16-year old is a lousy driver,” observes one insurance industry telematics expert. Rates based on actual behavior are “a big advantage in being able to price appropriately.” This kind of certainty means that insurance contracts designed to mitigate risk now give way to machine processes that respond “almost immediately” to nuanced infractions of prescribed behavioral parameters and thus substantially decrease risk or eliminate it entirely.

Telematics are not intended merely to know but also to do (economies of action). They are hammers; they are muscular; they enforce. Behavioral underwriting promises to reduce risk through machine processes designed to modify behavior in the direction of maximum profitability. Behavioral surplus is used to trigger punishments, such as real-time rate hikes, financial penalties, curfews, and engine lockdowns, or rewards, such as rate discounts, coupons, and gold stars to redeem for future benefits. The consultancy firm AT Kearney anticipates “IoT enriched relationships” to connect “more holistically” with customers “to influence their behaviors.”

Varian's blithe statement that "it's a lot easier" to instruct a vehicular monitoring system to shut off a car when a payment is late is not hyperbole. For example, Spireon, which describes itself as "the largest aftermarket vehicle telematics company" and specializes in tracking and monitoring vehicles and drivers for a variety of customers such as lenders, insurers, and fleet owners, offers a system akin to Varian's ideal. Its "LoanPlus Collateral Management System" pushes alerts to drivers when they have fallen behind in their payments, remotely disables the vehicle when delinquency exceeds a predetermined period, and locates the vehicle for the repo man to recover.

The consultancy firms are aligned in advising all their insurance clients to get into the surveillance game. AT Kearney acknowledges that the "connected car" is a proving ground for what is to come: "Ultimately, IoT's true value depends on customers adjusting their behaviors and risk profiles based on feedback from their 'things.'" Health insurers are another target: "Wearable accelerometers" could "improve traceability of their compliance" with prescribed exercise regimes, and "digestible sensors" could track compliance with dietary and medication schedules, "providing higher truth and better granularity than a monthly refill."

Behavioral underwriting offers auto insurers cost savings and efficiencies, but it is not the endgame for a revitalized insurance industry. The analytics that produce targeted advertising in the online world are repurposed for the real world, laying the foundation for new behavioral futures markets that trade in predictions of customer behavior. This is where the real drive for surveillance revenues is focused. For example, an executive of cloud services provider Covisint advises clients aiming to “cash in” on automotive telematics to move beyond targeted ads to “targeted applications.” These are not ads on a screen but real-life experiences shaped by the same capabilities as targeted ads and designed to lure you into real places for the sake of others’ profit.

Behavioral surplus is understood as the raw material for products that establish “co-marketing” with other services such as “towing, auto repair, car washes, restaurants, retail outlets.…” The consultants at McKinsey make a similar recommendation, advising insurers that the “internet of things” enables their expansion into “completely new areas” such as “data market-places.” Health surplus can be “monetized,” says Deloitte, by providing “relevant referrals.”

The data companies were once regarded as mere “suppliers,” but it is more likely that the auto companies will become suppliers to the data behemoths. “Google tries to accompany people throughout their day, to generate data and then use that data for economic gain,” acknowledges Daimler’s CEO. “It’s at that point where a conflict with Google seems preprogrammed.” Google and Amazon are already locked in competition for the dashboard of your car, where their systems will control all communication and applications. From there it is a short step to telemetry and related data.

Drivers are persuaded, induced, incentivized, or coerced into a quid pro quo that links pricing to the expansion of a real-world extraction/execution architecture aimed at new behavioral surplus flows (economies of scope). Behavioral data drawn from their experience are processed, and the results flow in two directions. First, they return to the drivers, executing procedures to interrupt and shape behavior in order to enhance the certainty, and therefore profitability, of predictions (economies of action). Second, prediction products that rank and sort driver behavior flow into newly convened behavioral futures markets in which third parties lay bets on what drivers will do now, soon, and later: Will he maintain a high safety rating? Will she act in compliance with our rules? Will he drive like a girl? These bets translate into pricing, incentive structures, and monitoring and compliance regimes. In both operations, surplus drawn from the driver's experience is repurposed as the means to shape and compel the driver's experience for the sake of guaranteed outcomes. Most of this occurs, as MacKay advised, outside the driver's awareness while she still thinks that she is free.

As Varian writes, "Because transactions are now computer-mediated we can observe behavior that was previously unobservable and write contracts on it. This enables transactions that were simply not feasible before." Varian's "we" refers to those with privileged access to the shadow text into which behavioral data flow. Our behavior, once unobservable, is declared as free for the taking, theirs to own, and theirs to decide how to use and how to profit from. This includes the production of "new contractual forms" that compel us in ways that would not have been possible but for surveillance capitalism's original declarations of dispossession.

Varian recognized that the subregions of automotive telematics exemplify this new economic frontier when he wrote, "Nowadays it's a lot easier just to instruct the vehicular monitoring system not to allow the car to be started and to signal the location where it can be picked up." Yawn. But wait. "A lot easier" for whom? He means, of course, a lot easier for the "we" that now observes what was, until surveillance capitalism, unobservable and executes actions that were, until surveillance capitalism, not feasible. Varian's laid-back, simple prose is a kind a lullaby that makes his observations seem banal, so ordinary as to barely warrant comment. But in Varian's scenario, what happens to the driver? What if there is a child in the car? Or a blizzard? Or a train to catch? Or a day-care center drop-off on the way to work? A mother on life support in the hospital still miles away? A son waiting to be picked up at school?

It was not long ago that Varian's prosaic proclamations were regarded as the stuff of nightmares. In his 1967 book The Year 2000 the hyper-rational wunderkind futurist Herman Kahn anticipated many of the capabilities that Varian now assigns to the new extraction/execution architecture. Kahn was no shrinking violet. He was rumored to be director Stanley Kubrick's model for the title character in Dr. Strangelove, and he was well-known for arguing that nuclear war is both “winnable” and “survivable.” Yet it was Kahn who foresaw innovations such as Varian's vehicular monitoring system and characterized them as “a twenty-first century nightmare.” Among his many technology-oriented insights, Kahn foresaw automated computer systems that track all vehicular movements and also listen to and record conversations with all the capability available for high-speed scan and search. He imagined computer systems able to detect and respond to individual behavior—a raised voice, a threatening tone: “Such computers may also be able to apply a great deal of inferential logic on their own—they may become a sort of transistorized Sherlock Holmes making hypotheses and investigating leads in a more or less autonomous or self-motivated manner.…” Anyone who wields this kind of knowledge, he concluded, is, like Faust, “less immoral than amoral… indifferent to the fate of those who stand in his way rather than brutal.”

Contemporary reviewers of Kahn's book invariably seized upon the dark “nightmare scenarios” of the computerized surveillance theme, the science-fiction–like forms of control that, they assumed, “will be actively feared and resented by many.” Despite the wide range of scenarios that Kahn presented in his book on the distant year 2000, Kahn's voyage into the “unthinkable” was viewed by the public as a way to prepare for “the worst possible outcome” in a terrifying “nightmare of social controls.” Yet now that same nightmare is rendered as an enthusiastic progress report on surveillance capitalism's latest triumphs. Varian's update is delivered without self-consciousness or a hint of controversy, rather than the astonishment and revulsion that were predicted just decades ago. How has the nightmare become banal? Where is our sense of astonishment and outrage?

So let us establish our bearings. What Varian celebrates here is not a new form of contract but rather a final solution to the enduring uncertainty that is the raison d'être of “contract” as a means of “private ordering.” In fact, the use of the word contract in Varian's formulation is a perfect example of the horseless-carriage syndrome. Varian's invention is unprecedented and cannot be understood as simply another kind of contract. It is, in fact, the annihilation of contract; this invention is better understood as the uncontract.

The uncontract is a feature of the larger complex that is the means of behavioral modification, and it is therefore an essential modality of surveillance capitalism. It contributes to economies of action by leveraging proprietary behavioral surplus to preempt and foreclose action alternatives, thus replacing the indeterminacy of social processes with the determinism of programmed machine processes. This is not the automation of society, as some might think, but rather the replacement of society with machine action dictated by economic imperatives.

The uncontract is not a space of contractual relations but rather a unilateral execution that makes those relations unnecessary. The uncontract desocializes the contract, manufacturing certainty through the substitution of automated procedures for promises, dialogue, shared meaning, problem solving, dispute resolution, and trust: the expressions of solidarity and human agency that have been gradually institutionalized in the notion of "contract" over the course of millennia. The uncontract bypasses all that social work in favor of compulsion, and it does so for the sake of more-lucrative prediction products that approximate observation and therefore guarantee outcomes.

This substitution of machine work for social work is possible thanks to the success of Google's declarations and the road that Google paved for surveillance capitalists' dominance of the division of learning. Sitting in the catbird seat, Google can observe what was previously unobservable and know what was previously unknowable. As a result, the company can do what was previously undoable: bypass social relations in favor of automated machine processes that compel the behaviors that advance commercial objectives.

The sense of incontestable certainty that infuses Paradiso's vision has long been recognized as a key feature of utopianism. In their definitive history of utopian thought, Frank and Fritzie Manuel wrote that "since the end of the eighteenth century the predictive utopia has become a major form of imaginative thought and has preempted certain scientific techniques of forecasting… the contemporary utopia… binds past, present, and future together as though fated. The state they depict appears virtually ordained either by god or by history; there is a carry-over of millenarian certainty.…"

Now the proselytizers of ubiquitous computing join Marx and other modern utopianists in postulating a new phase of history, like Paradiso’s “revolutionary phase shift,” in which all of society is reassembled in a novel and superior pattern. Despite the fact that inevitability is the opposite of politics and history, apostles of the apparatus routinely hijack historical metaphors that lend a veneer of gravitas to their claims. The rise of the apparatus is alternatively cast as the inauguration of a new “age,” “era,” “wave,” “phase,” or “stage.” This kind of historical framing conveys the futility of opposition to the categorical inevitability of the march toward ubiquity.

Silicon Valley is the axis mundi of inevitabilism. Among high-tech leaders, within the specialist literature, and among expert professionals there appears to be universal agreement on the idea that everything will be connected, knowable, and actionable in the near future: ubiquity and its consequences in total information are an article of faith.

The relentless drumbeat of inevitabilist messages presents the new apparatus of ubiquity as the product of technological forces that operate beyond human agency and the choices of communities, an implacable movement that originates outside history and exerts a momentum that in some vague way drives toward the perfection of the species and the planet. The image of technology as an autonomous force with unavoidable actions and consequences has been employed across the centuries to erase the fingerprints of power and absolve it of responsibility. The monster did it, not Victor Frankenstein. However, the ankle bracelet does not monitor the prisoner; the criminal justice system does that.

Every doctrine of inevitability carries a weaponized virus of moral nihilism programmed to target human agency and delete resistance and creativity from the text of human possibility. Inevitability rhetoric is a cunning fraud designed to render us helpless and passive in the face of implacable forces that are and must always be indifferent to the merely human. This is the world of the robotized interface, where technologies work their will, resolutely protecting power from challenge.

Surveillance capitalist leaders assume that we will succumb to the naturalistic fallacy as Steinbeck's farmers were meant to do. Because Google is successful—because surveillance capitalism is successful—its rules must obviously be right and good. Like the bank agents, Google wants us to accept that its rules simply reflect the requirements of autonomous processes, something that people cannot control. However, our grasp of the inner logic of surveillance capitalism suggests otherwise. Men and women made it, and they can control it. They merely choose not to do so.

Inevitabilism enshrines the apparatus of ubiquity as progress but conceals the realpolitik of surveillance capitalism at work behind the scenes. We know that there can be alternative paths to a robust information capitalism that produces genuine solutions for a third modernity. We have seen that surveillance capitalism was discovered and honed in history, handcrafted by men and women to serve the interests of impatient capital. It is this same logic that now demands ubiquity, ready to colonize technical developments for the sake of its imperatives and growth. Inevitabilism operates in the service of these imperatives as it distracts attention from the ambitions of a rising economic order and the competitive anxieties that drive the surveillance project toward certainty, thus necessitating its ever-more-voracious claims on our behavior.

Inevitabilism precludes choice and voluntary participation. It leaves no room for human will as the author of the future. This raises questions: At what point does inevitabilism's claim to ubiquitous extraction and execution shade into abuse? Will inevitabilism's utopian declarations summon new forms of coercion designed to quiet restless populations unable to quell their hankering for a future of their choice?

Cisco has 120 "smart cities" globally, some of which have embraced Cisco Kinetic, which as Jahangir Mohammed, the company's vice president and general manager of IoT, explains in a blog post, "is a cloud-based platform that helps customers extract, compute, and move data from connected things to IoT applications to deliver better outcomes.… Cisco Kinetic gets the right data to the right applications at the right time… while executing policies to enforce data ownership, privacy, security and even data sovereignty laws." But, as is so often the case, the most audacious effort to transform the urban commons into the surveillance capitalist's equivalent of Paradiso's 250-acre marsh comes from Google, which has introduced and legitimated the concept of the "for-profit city." Just as MacKay had counseled and Weiser proselytized, the computer would be operational everywhere and detectable nowhere, always beyond the edge of individual awareness.

Sidewalk Labs' first public undertaking was the installation of several hundred free internet-enabled kiosks in New York City, ostensibly to combat the problem of "digital inequality." As we saw with Google Street View, the company can siphon a lot of valuable information about people from a Wi-Fi network, even if they don't use the kiosks. Doctoroff has characterized the Sidewalk Labs' kiosks as "fountains of data" that will be equipped with environmental sensors and also collect "other data, all of which can create very hyperlocal information about conditions in the city."

Sidewalk refers to its high-tech services as “new superpowers to extend access and mobility.” Algorithms designed to maintain critical behaviors within a prescribed zone of action would manage these data flows: “In a world in which we can monitor things like noise or vibrations, why do we need to have these very prescriptive building codes?” As an alternative, Doctoroff suggests “performance-based zoning” administered by the ubiquitous apparatus through the medium of algorithms. These processes, like Varian’s vehicular monitoring systems, are indifferent to why you behave as long as they can monitor and control the behavior you produce.

In April 2016 a “curated group of leaders” in tech, media, and finance met at the Yale Club in Manhattan to hear Sidewalk CEO Dan Doctoroff’s talk: “Google City: How the Tech Juggernaut Is Reimagining Cities—Faster Than You Realize.” His remarks provide a candid assessment of the “Google city” as a market operation shaped by the prediction imperative. He could not have been more direct in articulating Sidewalk Labs’ approach as a translation of Google’s online world to the reality of city life: … In effect, what we’re doing is replicating the digital experience in physical space.… So ubiquitous connectivity; incredible computing power including artificial intelligence and machine learning; the ability to display data; sensing, including cameras and location data as well as other kinds of specialized sensors.… We fund it all… through a very novel advertising model.… We can actually then target ads to people in proximity, and then obviously over time track them through things like beacons and location services as well as their browsing activity.

The vast and varied ground campaign already underway turns the prediction imperative into concrete activity. In pursuit of economies of scope, a wave of novel machine processes are honed for extraction, rendering people and things as behavioral data. For the sake of economies of action, the apparatus learns to interrupt the flow of personal experience in order to influence, modify, and direct our behavior, guided by the plans and interests of self-authorizing commercial actors and the buzzing market cosmos in which they participate. In nearly every case the agents of institutionalization present their novel practices as if they are one thing, when they are, in fact, something altogether different. The realpolitik of commercial surveillance operations is concealed offstage while the chorus of actors singing and dancing under the spotlights holds our attention and sometimes even our enthusiasm. They sweat under the stage lights for the sake of one aim: that we fail to notice the answers or, better yet, forget to ask the questions: Who knows? Who decides? Who decides who decides?

In October 2017, Doctoroff appeared with Alphabet Executive Chairman Eric Schmidt and Canadian Prime Minister Justin Trudeau to reveal that Toronto would be the site of Sidewalk’s planned development. Its intent is to develop the right mix of technology that it can then license to cities around the world. “The genesis of the thinking for Sidewalk Labs came from Google’s founders getting excited thinking of ‘all the things you could do if someone would just give us a city and put us in charge,’” Toronto’s Globe and Mail reported Schmidt as saying, noting that “he joked he knew there were good reasons that doesn’t happen.” Then, just as quickly, the paper related Schmidt’s reaction when he first learned that Sidewalk, and by extension Alphabet, had secured this opportunity in Toronto: “Oh my God! We’ve been selected. Now, it’s our turn.”

We worry about companies that amass our personal data, and we wonder why they should profit. “Who owns the data?” we ask. But every discussion of data protection or data ownership omits the most important question of all: why is our experience rendered as behavioral data in the first place? It has been far too easy to overlook this important step in the chain of events that produces behavioral surplus. This chapter and the next draw our attention to the gap between experience and data, as well as to the specific operations that target this gap on a mission to transform the one into the other. I call these operations rendition. We have seen that the dispossession of human experience is the original sin of surveillance capitalism, but this dispossession is not mere abstraction. Rendition describes the concrete operational practices through which dispossession is accomplished, as human experience is claimed as raw material for datafication and all that follows, from manufacturing to sales. A focus on these intermediate practices illustrates that the apparatus of ubiquity is not a passive one-way mirror. Rather, it actively creates its own stores of knowledge through rendition.

Surveillance capitalism must work both sides of the equation. On one side, its technologies are designed to render our experience into data, as in rendering oil from fat. This typically occurs outside of our awareness, let alone our consent. On the other side of the equation, every time we encounter a digital interface we make our experience available to “datafication,” thus “rendering unto surveillance capitalism” its continuous tithe of raw-material supplies.

As we saw in Chapter 1, the Aware Home project developed at Georgia Tech just a year before the invention of surveillance capitalism employed different practices that embodied very different assumptions: (1) that it must be the individual alone who decides what experience is rendered as data, (2) that the purpose of the data is to enrich the individual’s life, and (3) that the individual is the sole arbiter of how the data are shared or put to use. Nearly two decades later, the Aware Home is barely more than an archeological fragment reminding us of the road not taken toward an empowering digital future and a more-just division of learning in society. Down that road, it is the individual who knows, decides, and decides who decides: an end in herself, not a means to others’ ends. The lesson of the Aware Home is that there can be rendition without surveillance capitalism. However, the lesson of this chapter and the next is that there can be no surveillance capitalism without rendition.

Nothing is exempt, as products and services from every sector join devices like the Nest thermostat in the competition for surveillance revenues. For example, in July 2017 iRobot’s autonomous vacuum cleaner, Roomba, made headlines when the company’s CEO, Colin Angle, told Reuters about its data-based business strategy for the smart home, starting with a new revenue stream derived from selling floor plans of customers’ homes scraped from the machine’s new mapping capabilities. Angle indicated that iRobot could reach a deal to sell its maps to Google, Amazon, or Apple within the next two years. In preparation for this entry into surveillance competition, a camera, new sensors, and software had already been added to Roomba’s premier line, enabling new functions, including the ability to build a map while tracking its own location. The market had rewarded iRobot’s growth vision, sending the company’s stock price to $102 in June 2017 from just $35 a year earlier, translating into a market capitalization of $2.5 billion on revenues of $660 million.

Angle's confidence-enhancing strategy goes to the heart of the larger rendition project, for which surveillance capitalist purveyors of "smart" home products have developed a singular approach. On the one hand, they stress that customers can opt in to data sharing. On the other hand, customers who refuse to opt in face limited product functionality and data security. In these Requirimiento-style relationships, instead of the adelantados' message, "Bend the knee or we destroy you," the message here is "Bend the knee or we degrade your purchase."

A dense, twelve-page privacy policy accompanies the bed. Customers are advised that providing information is an affirmation of consent to use that information in-line with the policy, which employs the usual onerous terms: third-party sharing, Google analytics, targeted advertising, and much more. In addition, if customers create a user profile to maximize the effectiveness of the app, the company also collects "biometric and sleep-related data about how You, a Child, and any person that uses the Bed slept, such as that person's movement, positions, respiration, and heart rate while sleeping." It also collects all the audio signals in your bedroom. As with most such policies, customers are advised that the company can "share" or "exploit" personal information even "after You deactivate or cancel the Services and/or your Sleep Number account or User Profiles(s)."

This same coercive Requirimiento twist can be found in the lengthy, dense legal compacts associated with Alphabet-owned Nest thermostats. The terms-of-service and end-user licensing agreements reveal oppressive privacy and security consequences in which sensitive information is shared with other devices, unnamed personnel, and third parties for the purposes of analysis and ultimately for trading in behavioral futures markets, action that ricochets back to the owner in the form of targeted ads and messages designed to push more products and services. Despite this, courts have generally upheld companies' claims that they bear no liability without a clear demonstration of economic harm to the consumer.

Each is reimagined as a gateway to the new apparatus, praised for being “smart” while traditional alternatives are reviled for remaining “dumb.” It is important to acknowledge that in this context, “smart” is a euphemism for rendition: intelligence that is designed to render some tiny corner of lived experience as behavioral data. Each smart object is a kind of marionette; for all its “smartness,” it remains a hapless puppet dancing to the puppet master’s hidden economic imperatives.

Examples of products determined to render, monitor, record, and communicate behavioral data proliferate, from smart vodka bottles to internet-enabled rectal thermometers, and quite literally everything in between. The business developer for a spirits company thus cites his plan for a “connected bottle”: “The more we learn about consumers and their behaviors, the better services we can connect to them.” Many brands are determined “to give packaging a speaking role in an increasingly interactive marketplace.” Global spirits distributor Diageo promises “smart sensor-equipped bottles” that can track purchases and sales data, and, most importantly, “communicate with consumers’ devices and switch gears—recipes versus sales promos—once the bottle is opened.” A producer of bar equipment states the case plainly enough: “It really is all about… allowing these [bar] owners to see stuff that they couldn’t see before and maximize their profits.”

Today our homes are in surveillance capitalism’s crosshairs, as competitors chased a $14.7 billion market for smart-home devices in 2017, up from $6.8 billion just a year earlier and expected to reach more than $101 billion by 2021. … Even the baby’s nursery is reconceived as a source of fresh behavioral surplus.

Here is at least one endgame: the “smart home” and its “internet of things” are the canvas upon which the new markets in future behavior inscribe their presence and assert their demands in our most-intimate spaces. Key to the story is that all of this action is prosecuted in support of a larger market process that bets on the future of our behavior and over which we have no knowledge or control. Each node in the network—the vacuum cleaner, the mattress, the thermostat—must play its part, beginning with the frictionless rendition of behavior, as the whole team of seething insistent “smart” things joins the migration to surveillance revenues. As we are shorn of alternatives, we are forced to purchase products that we can never own while our payments fund our own surveillance and coercion. Adding insult to injury, data rendered by this wave of things are notoriously insecure and easily subject to breaches. Moreover, manufacturers have no legal responsibility to notify device owners when data are stolen or hacked.

Companies such as Qualcomm, Intel, and ARM are developing tiny, always-on, low-power computer vision modules that can be added to any device, such as your phone or refrigerator, or any surface. A Qualcomm executive says that appliances and toys can know what’s going on around them: “A doll could detect when a child’s face turns toward it.”

Consider “smart skin,” developed by brilliant university scientists and now poised for commercial elaboration. Initially valued for its ability to monitor and diagnosis health conditions from Parkinson’s disease to sleep disorders, smart skin is now hailed for its promise of ultra-unobtrusive ubiquity. Researchers at Georgia Tech developed a version of “smart skin” that sucks energy from radio waves and other energy sources, eliminating the need for batteries. Smart skin, described as “the ultimate sensing tool that could potentially allow for the mass implementation of perpetual wireless networks,” can cognize, sense, analyze, wirelessly communicate, and “modify parameters” using simple radio frequency (RFID) technology. As in the case of Paradiso’s “sensor tape,” the researchers stress that it can also “be applied everywhere” to “monitor, sense, and interact with the world around us in a perpetual way, thus significantly enhancing ambient intelligence,” all of it as inconspicuous as a “decal sticker.” They suggest, for example, the shelves of grocery stores, where revenue opportunities are plentiful.

Rendition has become a surveillance capitalist project shaped by its imperatives and directed toward its objectives. In the composition of the shadow text, rendition is Step One: the concrete operationalization of the “original sin of simple robbery” that defined this market project from the start. Google rendered the Earth, its streets, and its dwelling places, bypassing our consent and defying our protests. Facebook rendered the social network and its limitless details for the sake of the company’s behavioral futures markets. Now the ubiquitous apparatus is the means to the ubiquitous rendition of human experience. We have seen the urgency with which surveillance capitalists pursue the elimination of “friction” as a critical success factor in supply operations. The prediction imperative makes boundaries and borders intolerable, and surveillance capitalists will do almost anything to eliminate them. This pursuit transforms “connection” into a commercial imperative and transforms individual autonomy into a threat to surveillance revenues.

Surveillance capitalism’s rendition practices overwhelm any sensible discussion of “opt in” and “opt out.” There are no more fig leaves. The euphemisms of consent can no longer divert attention from the bare facts: under surveillance capitalism, rendition is typically unauthorized, unilateral, gluttonous, secret, and brazen. These characteristics summarize the asymmetries of power that put the “surveillance” in surveillance capitalism. They also highlight a harsh truth: it is difficult to be where rendition is not. As industries far beyond the technology sector are lured by surveillance profits, the ferocity of the race to find and render experience as data has turned rendition into a global project of surveillance capital.

This chapter and the next survey a range of rendition activities in the pursuit of economies of scope. The remainder of this chapter concentrates on extension, the first dimension of scope, as rendition operations move into the real world, seizing fresh unexpected chunks of human experience. Extension wants every corner and crevice, every utterance and gesture on the path to dispossession. All that is moist and alive must hand over its facts. There can be no shadow, no darkness. The unknown is intolerable. The solitary is forbidden.

In this pursuit, we are necessarily involved in citing specific actors, products, and techniques, knowing that the details of persons and companies are constantly churning. Firms will be bought and sold, fail or succeed; people will come and go. Specific technologies, products, and techniques will be abandoned, refined, and surpassed. When they fall, new ones will take their place, as long as surveillance capitalism is allowed to flourish. Velocity and churn have been critical to surveillance capitalism’s success, and we cannot allow the constant movement to inhibit our determination to grasp the “laws of motion” that command this roiling landscape. It is the pattern and its purpose that we want to grasp.

The rendition of your body begins quite simply with your phone. Even when your city is not “smart” or owned and operated by Google, market players with an interest in your behavior know how to find your body. For all of the elaborate ways in which surveillance capitalists labor to render reality as behavior for surplus, the simplest and most profound is their ability to know exactly where you are all the time. Your body is reimagined as a behaving object to be tracked and calculated for indexing and search. Most smartphone apps demand access to your location even when it’s not necessary for the service they provide, simply because the answer to this question is so lucrative.

Another mobile marketing firm recommends “life pattern marketing” based on techniques derived from military intelligence known as “patterns of life analysis.” These involve gathering location and other data from phones, satellites, vehicles, and sensors to assemble intelligence on the daily behavioral patterns of a “person of interest” in order to predict future behavior. Marketers are exhorted to “map the daily patterns” of a “target audience” in order to “intercept people in their daily routines with brand and promotional messages.” As the firm emphasizes, “The psychological power of the perception of ubiquity is profound. Life Pattern Marketing forms a powerful psychological imprint on consumers.”

Unsurprisingly, Google represents the vanguard of location-based tracking. A 2016 affidavit from law-enforcement officials seeking a search warrant for a California bank robber made plain why Google location data are unparalleled: “Google collects and retains location data from Android-enabled mobile devices. Google collects this data whenever one of their services is activated and/or whenever there is an event on the mobile device such as a phone call, text messages, internet access, or email access.” The officials on the case requested location information from Google because it offers far more detail than even the phone companies can provide. The location systems in Android combine cell-tower data with GPS, Wi-Fi networks, and other information culled from photos, videos, and other sources: “That lets Android pinpoint users to a single building, rather than a city block.”

In November 2017 Quartz investigative reporters discovered that since early 2017, Android phones had been collecting location information by triangulating the nearest cell towers, even when location services were disabled, no apps were running, and no carrier SIM card was installed in the phone. The information was used to manage Google’s “push” notifications and messages sent to users on their Android phones, enabling the company to track “whether an individual with an Android phone or running Google apps has set foot in a specific store, and use that to target the advertising a user subsequently sees.”

Google’s location history system is a product of the corporation’s global mapping operations. Though active for over a decade, it was unveiled to the public only in 2015 as “Your Timeline,” a feature that “allows you to visualize your real-world routines.” The corporation calculated that any negative reaction to the volume and persistence of tracking revealed by Timeline would be mitigated by the value of users’ active contributions to their own stocks of behavioral surplus as they fine-tune the information, add relevant photos, insert comments, and so forth. This was represented as an individual’s investment in personalized services such as Google Now so that it might more effectively comb your e-mail and apps in order to push relevant traffic and weather updates, notifications, suggestions, and reminders. Location data is the quid pro quo for these services.

However, with as little as three bits of data easily culled from the public record—birth date, zip code, and sex—reidentification science has demonstrated its ability to de-anonymize meta-data with “disturbing ease.” In a summary of this research, legal scholar Paul Ohm writes that “re-identification makes all our secrets fundamentally easier to discover and reveal. Our enemies will find it easier to connect us to facts that they can use to blackmail, harass, defame, frame or discriminate against us.… This mistake pervades nearly every information privacy law.” Regarding the massive caches of supposedly anonymous behavior surplus, Ohm calls them “databases of ruin.”

In 2013 a group of MIT and Harvard computer scientists demonstrated that because individuals tend to have idiosyncratic mobility signatures, any analyst with the right tools can easily extract the mobility pattern of a specific individual within a large anonymized data set of location meta-data. Another research team demonstrated that data collected by seemingly “innocuous” smartphone-embedded sensors, such as accelerometers, gyroscopes, and magnetometers, can be used to infer “an ever-growing range of human activities and even moods.”

Princeton computer scientists Arvind Narayanan and Edward Felten summed it up this way: “There is no known effective method to anonymize location data, and no evidence that it’s meaningfully achievable.”

Google has developed internet-enabled fabrics, claiming that it aims to bring inductive yarns to every garment and fabric on Earth. "If you can weave the sensor into the textile, as a material," explains project leader Ivan Poupyrev, "you're moving away from the electronics. You're making the basic materials of the world around us interactive."

There is a numbing repetition of MacKay's themes throughout the literature on wearables. Just as he insisted that telemetric devices must operate "outside the awareness" of "unrestrained animals," today's developers stress that wearables must be "unobtrusive" to avoid raising alarm. They are to be "continuous," "pervasive," and, crucially, "low cost" in order to achieve economies of scope. The digital marketing firm Ovum forecasts 650 million wearables by 2020, nearly double the number used in 2016, and its research suggests that growth is largely driven by the lure of surveillance revenues.

Before the birth and spread of surveillance capitalism, it was possible to imagine digital representations of your body as an enrichment of the intimate relationships between a patient and a trusted doctor, a mother and her child, elderly parents and their adult children. As surveillance capitalism overwhelms the digital milieu, that vision has been made ridiculous. Both the Aware Home and the telemedicine design assume that all behavioral data are reinvested in service to the human being who is the subject of these arrangements, providing serenity, trust, and dignity: a chance for real knowledge and empowerment.

In the US, most health and fitness applications are not subject to health privacy laws, and the laws that do exist do not adequately take into account either new digital capabilities or the ferocity of surveillance capitalist operations. Companies are expected to self-regulate by following guidelines suggested by the Federal Trade Commission (FTC) and other government agencies.

The agencies' well-meaning guidelines overlook the inconvenient truth that transparency and privacy represent friction for surveillance capitalists in much the same way that improving working conditions, rejecting child labor, or shortening the working day represented friction for the early industrial capitalists. It took targeted laws to change working conditions back then, not suggestions. Then as now, the problems to which these pleas for self-restraint are addressed cannot be understood as excesses, mistakes, oversights, or lapses of judgment. They are necessitated by the reigning logic of accumulation and its relentless economic imperatives.

The team also examined the trackers’ transmission of the Bluetooth Media Access Controller or “MAC” address that is unique to each phone. When this address is publicly discoverable, any third party with an interest in your movements—retailers who want to know your mall activity, insurers concerned about your compliance with an exercise regime—can “persistently” track your phone. Multiple data sets logged over time can be combined to form a fine-grained picture of your movements, enabling targeted applications and heightening the probability of guaranteed outcomes. The only real protection is when an app randomly but regularly generates a new MAC address for your phone, but of the nine trackers, only Apple’s performed this operation.

The report also identifies a general pattern of careless security as well as the ability to generate false data. The researchers observed that consumers are likely to be misled and confused, overestimating the extent of security measures and underestimating “the breadth of personal data collected by fitness tracking companies.” As they concluded, “We discovered severe security vulnerabilities, incredibly sensitive geolocation transmissions that serve no apparent benefit to the end user, and… policies leaving the door open for the sale of users’ fitness data to third parties without express consent of the users.”

The researchers note that although the FDA approved the prescription of a range of apps that transmit sensitive health data, the behind-the-scenes practices of these apps are “understudied.” They examined 211 diabetes apps and randomly sampled 65 of them for close analysis of data-transmission practices.

Among these apps, merely downloading the software automatically “authorized collection and modification of sensitive information.” The researchers identified a great deal of backstage action, including apps that modify or delete your information (64 percent), read your phone status and identity (31 percent), gather location data (27 percent), view your Wi-Fi connections (12 percent), and activate your camera in order to access your photos and videos (11 percent). Between 4 percent and 6 percent of the apps went even further: reading your contact lists, calling phone numbers found in your device, modifying your contacts, reading your call log, and activating your microphone to record your speech.

Finally, the research team unearthed an even darker secret: privacy policies do not matter. Of the 211 apps in the group, 81 percent did not have privacy policies, but for those that did, “not all of the provisions actually protected privacy.” Of those apps without privacy policies, 76 percent shared sensitive information with third parties, and of those with privacy policies, 79 percent shared data while only about half admitted doing so in their published disclosures. In other words, privacy policies are more aptly referred to as surveillance policies, and that is what I suggest we call them.

There are many new territories of body rendition: organs, blood, eyes, brain waves, faces, gait, posture. Each of these expresses the same patterns and purpose that we have seen here. The surveillance capitalists relentlessly fight any attempts to constrain rendition. The ferocity with which they claim their “right to rendition” out of thin air is ample evidence of its foundational importance in the pursuit of surveillance revenues.

Between 2009 and 2017, the company increased its lobbying spend by a factor of fifty, building “a massive lobbying entourage of Washington power brokers.” Facebook’s $4.6 million in donations during the 2016 election cycle complemented its lobbying budget of $11.5 million in 2017.

If rendition is interrupted, surveillance capitalism cannot stand, for the whole enterprise rests on this original sin. This fact is amply displayed in the public drama surrounding the ill-fated 2015 attempt to produce public guidelines on the creation and use of biometric information through a voluntary “privacy multi-stakeholder” process convened by the National Telecommunications and Information Association (NTIA) under the auspices of the US Department of Commerce. After weeks of negotiations, consumer advocates walked out in protest over the hard-line position of the tech companies and their lobbyists on the single most pivotal issue: consent.

The talks continued without the advocates, and in 2016 the NTIA issued its “Privacy Best Practice Recommendations for Commercial Facial Recognition Use.” The guidelines should be understood as the “best” for the surveillance capitalists but as the “worst” for everyone else. In the language of these guidelines, the tech companies, retailers, and others determined to chase surveillance revenues are simply “encouraged” to make their policies on facial recognition “available to consumers, in a reasonable manner.…” Where companies impose facial recognition on a physical location, they are “encouraged” to provide “notice” to consumers. Rendition operations are tacitly accorded legitimacy, not only in the lack of contest, but because they stand as the immovable facts draped in the cheap garlands of toothless “best practices.”

Under the regime of surveillance capitalism, individuals do not render their experience out of choice or obligation but rather out of ignorance and the dictatorship of no alternatives. The ubiquitous apparatus operates through coercion and stealth. Our advance into life necessarily takes us through the digital, where involuntary rendition has become an inescapable fact. We are left with few rights to know, or to decide who knows, or to decide who decides.

This is a new frontier of behavioral surplus where the dark data continent of your inner life—your intentions and motives, meanings and needs, preferences and desires, moods and emotions, personality and disposition, truth telling or deceit—is summoned into the light for others' profit. The point is not to cure but to render all of it as immeasurably tiny bits of behavior available for calculation so that each can take its place on the assembly line that moves from raw materials to product development, manufacturing, and sales.

The machine invasion of human depth is prosecuted under the banner of "personalization," a slogan that betrays the zest and cynicism brought to the grimy challenge of exploiting second-modernity needs and insecurities for outsize gain. From the point of view of the prediction imperative, personalization is a means of "individualizing" supply operations in order to secure a continuous flow of behavioral surplus from the depths. This process can be accomplished successfully only in the presence of our unrelenting hunger for recognition, appreciation, and most of all, support.

In fact, Varian's notion of personalization is the precise opposite of the relationships with trusted professionals to which he refers. Doctors, accountants, and attorneys are held to account by mutual dependencies and reciprocities dictated by the extensive institutionalization of professional education, codes of conduct, and procedures for evaluation and review. Violation of these rules risks punishment in the form of professional sanction and public law. Google and its brethren in surveillance capitalism bear no such risks.

Varian reasons that inequality offers an opportunity to raise the ante on Google's quid pro quo for effective life. He counsels that the way to predict the future is to observe what rich people have because that's also what the middle class and the poor will want. “What do rich people have now?” he asks rhetorically. “Personal assistants.”

That the luxuries of one generation or class become the necessities of the next has been fundamental to the evolution of capitalism during the last five hundred years. Historians describe the “consumer boom” that ignited the first industrial revolution in late-eighteenth-century Britain, when, thanks to visionaries like Josiah Wedgewood and the innovations of the early modern factory, families new to the middle class began to buy the china, furniture, and textiles that only the rich had previously enjoyed. This new “propensity to consume” is considered “unprecedented in the depth to which it penetrated the lower reaches of society.…” In 1767 the political economist Nathaniel Forster worried that “fashionable luxury” was spreading “like a contagion,” and he complained of the “perpetual restless ambition in each of the inferior ranks to raise themselves to the level of those immediately above them.” Adam Smith wrote insightfully on this social process, noting that upper-class luxuries can in time be recast as “necessaries.” This occurs as “the established rules of decency” change to reflect new customs introduced by elites, triggering lower-cost production methods that transform what was once unattainable into newly affordable goods and services. Ford's Model T is the outstanding twentieth-century example of this progression.

Varian casts personalization as a twenty-first-century equivalent of these historical dynamics, the new “necessaries” for the harried masses bent under the weight of stagnant wages, dual-career obligations, indifferent corporations, and austerity's hollowed-out public institutions. Varian's bet is that the digital assistant will be so vital a resource in the struggle for effective life that ordinary people will accede to its substantial forfeitures. “There is no putting the genie back in the bottle,” Varian the inevitabilist insists. “Everyone will expect to be tracked and monitored, since the advantages, in terms of convenience, safety, and services, will be so great… continuous monitoring will be the norm.” Everyone, that is, except those wealthy or stubborn enough to achieve effective life without Google's assistance and thus escape the worst excesses of rendition. As decision rights and self-determination become privileges of the wealthy, what will Varian offer to answer those who clamor for the same?

Historically, breakthroughs in lower-cost goods and services unleashed expansions of production and employment, higher wages, and an improved standard of living for many. Varian has no such reciprocities in mind. Instead, he pokes his finger into the open wound of second-modernity insecurities and bends our pain to the objectives of the surveillance project. With Varian, the hunger for new necessities is construed as an opportunity to dispossess, even as it conveniently provides the justification for that dispossession right down to the depths.

Google's breakthrough crawler enabled the lightning-fast indexing of the world wide web, the apparatus of ubiquity then enabled new operations to crawl reality, and now in this third phase, distinct supply operations are required to crawl our lives. In Google Now one sees an initial foray into this new space, in which the web crawler's ability to find information combines with new life-crawling operations intended to render, anticipate, and, as we shall see, ultimately modify your behavior. Online and offline behavioral surplus—your e-mail content, where you went this afternoon, what you said, what you did, how you felt—are combined into prediction products that can serve an emerging marketplace in which every aspect of your daily reality is up for bid.

Google joined other tech companies determined to establish “conversation” as the medium through which humans engage with the apparatus. In time, the obsession with voice may be surpassed or joined by others so that merely thinking a thought or waving a finger can translate into and initiate action. For now, there are compelling reasons for the race to the spoken word. The first is obvious: reliable voice recognition can translate a sprawling landscape of service interactions into low-cost automated processes of theoretically unlimited scale and scope, a fact that has been noted by labor economists for some time. The competitive race between a new crop of “personal digital assistants” is best understood from this point of view. The voice that rises to dominance, the One Voice, will be the colossus of behavioral surplus pipelines with a potentially insurmountable competitive advantage in its ability to corner and kidnap the dominant share of human experience.

“Conversation” stands alone in its promise to dominate raw-material supply, and the rewards to the One Voice would be astronomical. Casual talk helps to blur the boundaries between “it”—the apparatus saturated with commercial agents—and us. In conversation we imagine friendship. The more we fancy the apparatus as our confidante, nanny, governess, and support system—a disembodied, pervasive “Mrs. Doubtfire” for each person—the more experience we allow it to render, and the richer its supply operations grow. Communication is the first human joy, and a conversational interface is prized for the frictionless ease in which a mere utterance can trigger action, especially market action: “Let there be light.” “Let there be new running shoes.” What could be dreamier than to speak and have it be so?

Spontaneous and fluid, universally burbling “conversation” turns the new personal digital assistant into a voice that sits between your life and the new markets for your life, between your experience and the auctioning of your experience: “a runtime, a new interface” that creates the sensation of mastery while, in fact, giving it away.

In this commercial dreamscape, words that were once conceived of as “behind closed doors” are eagerly rendered as surplus. These new supply operations convert your talk into behavior for surplus in two ways. The first derives from what you say, the second from how you say it. Smart-home devices such as Amazon’s Echo or Google Home render rivers of casual talk from which sophisticated content analyses produce enhanced predictions that “anticipate” your needs.

Google’s Assistant arrived already integrated into Google’s new messaging app, Allo, where it can search for information, execute tasks, or even compose routine messages on your behalf. Most importantly, Assistant animates the firm’s home device, Google Home. The idea is that in time, the device (or its successor) will claim for rendition a theoretically limitless scope of animate and inanimate domestic activities: conversations, lightbulbs, queries, schedules, movement, travel planning, heating systems, purchases, home security, health concerns, music, communication functions, and more.

There was a time when you searched Google, but now Google searches you. Advertisements for Google Home feature loving families leading busy, intricate lives but visibly relieved to return home and fall into the arms of this omniscient, efficient caretaker. This second-modernity dream come true extracts an unusually high tax for its promise of a more effective life. For each user to have his or her own individual Google, as Pichai envisions, Google must have each individual.

Pieces of your talk are regularly farmed out in bulk to third-party firms that conduct "audio review processes" in which virtual scorers, tasked to evaluate the degree of match between the machine's text and the original chunk of human speech, review audio recordings retained from smartphones, messaging apps, and digital assistants. Companies such as Amazon, Google, and Microsoft use these audio analyses to improve the algorithms of their voice systems. The tech companies insist that such recordings are anonymous, nothing more than voices without identities. "Partners do not have the ability to tie the voice samples back to specific people," one Microsoft executive asserted. But one journalist who signed on to a virtual job as an audio recording analyst concluded just the opposite, as she listened to recordings full of pathos, intimacy, and easily identifiable personal information: Within the recordings themselves, users willingly surrender personal information-information that is especially valuable in these review processes because they are so specific.

Within the recordings themselves, users willingly surrender personal information—information that is especially valuable in these review processes because they are so specific. Uncommon names, difficult-to-pronounce cities and towns, hyperlocal oddities.… I heard people share their full names to initiate a call or offer up location-sensitive information while scheduling a doctor’s appointment… the recordings capture people saying things they’d never want heard, regardless of anonymity.… There isn’t much to keep people who are listening to these recordings from sharing them.

The TV’s “surveillance policy”—yes, even a TV has a surveillance policy now—reveals the layers of surveillance effort and commercial interest that operate outside of awareness in our homes. Samsung acknowledges that the voice commands aimed at triggering the TV’s voice-recognition capabilities are sent to a third party and adds, “Please be aware that if your spoken words include personal or other sensitive information, that information will be among the data captured and transmitted to a third party through your use of Voice Recognition.” Samsung disclaims responsibility for the policies of third-party firms, as nearly all surveillance policies do, including the one that actually collects and translates the talk of its unsuspecting customers. Samsung advises that “You should exercise caution and review the privacy statements applicable to the third-party websites and services you use.” The intrepid consumer determined to study these documents will find no succor in Nuance’s privacy policy, only the same catechism found in Samsung’s and that of nearly every company. It also encourages you to read the privacy policies of the companies to whom it’s selling your conversations, and so it goes: a forced march toward madness or surrender.

In 2017 the FTC reached a $2.2 million settlement of a complaint initiated by the Office of the New Jersey Attorney General against Vizio, one of the world’s largest manufacturers and distributors of internet-enabled smart televisions. Vizio’s supply operations appear to be even more aggressive than Samsung’s. Investigators discovered that “on a second-by-second basis, Vizio collected a selection of pixels on the screen that it matched to a database of TV, movie, and commercial content.” The company then identified additional viewing data “from cable or broadband service providers, set-top boxes, streaming devices, DVD players, and over-the-air broadcasts.” All of this amounted to as much as 100 billion data points each day just from the 11 million TVs identified in the settlement. Vizio disguised its supply operations behind a setting called “Smart Interactivity,” described to consumers as a feature that “enables program offers and suggestions” without any indication of its actual functions.

Vizio then turned that mountain of data into cash by selling consumers’ viewing histories to advertisers and others. And let’s be clear: We’re not talking about summary information about national viewing trends. According to the complaint, Vizio got personal. The company provided consumers’ IP addresses to data aggregators, who then matched the address with an individual consumer or household. Vizio’s contracts with third parties prohibited the re-identification of consumers and households by name, but allowed a host of other personal details—for example, sex, age, income, marital status, household size, education, and home ownership.

A concurring statement from FTC Acting Chairwoman Maureen K. Ohlhausen emphasized that the settlement broke new ground in its allegation that “individualized television viewing activity falls within the definition of sensitive information” that merits protection by the FTC. This finger in the dike would not hold back the tidal wave of similar incursions as the prediction imperative cracks the whip to drive the hunt for unexplored pieces of talk from daily life. Rendition takes command of even the most benign supply sources, such as toys, which have now become “toys that spy.” A new breed of interactive dolls and toy robots, including a girl doll called “My Friend Cayla,” turn out to be supply hubs for underage behavioral surplus, subjecting young children and their parents’ smartphones “to ongoing surveillance… without any meaningful data protection standards.”

The popular playthings, marketed by Genesis Toys, are bundled with a mobile application that, once downloaded to a smartphone, “provides the data processing” to enable the toy’s ability to capture and understand whatever the child says. Along the way, the app accesses most of the phone’s functions, including many that are irrelevant to the toy’s operations such as contact lists and the camera. The app establishes a Bluetooth connection that links the toy to the internet, and it records and uploads conversations as the toy actively engages the child in discussion. One doll targeted in the complaint systematically prompts children to submit a range of personal information, including where they live.

In this future, children learn the principles of the One Voice—a run time, a new interface. It is available everywhere to execute their commands, anticipate their desires, and shape their possibilities. The omnipresence of the One Voice, with its fractious, eager marketplace-of-you concealed under its skirts, changes many things. Intimacy as we have known it is compromised, if not eliminated. Solitude is deleted. The children will learn first that there are no boundaries between self and market. Later they will wonder how it could ever have been different.

When Mattel hired a new CEO in January 2017, it's no surprise that the company looked to Google, selecting its president for the Americas with responsibility for Google's commercial and advertising sales operations. Most analysts agreed that the appointment heralded Mattel's commitment to its innovations in internet-enabled toys and virtual reality, but the appointment underscores the shift in focus from making great products for you to collecting great data about you.

In 2017 Germany's Federal Network Agency banned the Cayla doll as an illegal surveillance device and urged parents to destroy any such dolls in their possession. In the US the FTC had yet to take any action against the doll or Genesis Toys. Meanwhile, the connected dollhouse prepares our children and families for the connected room (a project that Mattel announced in January 2017 and then nine months later shelved amid an uproar from parents and privacy advocates), which paves the way for the connected home, whose purveyors hope will numb us to the connected world as we travel the path of ubiquity's manifest destiny and its promise of surveillance revenues.

In pursuit of the what and the how of voice surplus, the logic of competition is to corner as much supply as possible. The urge toward totality generates competitive pressures to become the run time and the new interface: the dominant, if not exclusive, medium through which we access and engage the apparatus as it engages us. It's a race to corner all the talk as a prerequisite for achieving the privileged status of the One Voice, which will bestow upon the winner the ability to anticipate and monetize all the moments of all the people during all the days.

Amazon aggressively opened Alexa to third-party developers in order to expand the assistant’s range of “skills,” such as reading a recipe or ordering a pizza. It also opened its platform to smart-home device makers from lighting systems to dishwashers, turning Alexa into a single voice for controlling your home systems and appliances. In 2015 Amazon announced that Alexa would be sold as a service, known as “Amazon Lex,” enabling any company to integrate Alexa’s brain into its products. Amazon Lex is described as “a service for building conversational interfaces into any application using voice and text.… Lex enables you to define entirely new categories of products.” As Alexa’s senior vice president explained, “Our goal is to try to create a kind of open, neutral ecosystem for Alexa… and make it as pervasive as we possibly can.”

By 2018, Amazon had inked deals with home builders, installing its Dot speakers directly into ceilings throughout the house as well as Echo devices and Alexa-powered door locks, light switches, security systems, door bells, and thermostats. As one report put it, “Amazon can acquire more comprehensive data on people’s living habits.…” The company wants to sell real-world services such as house cleaning, plumbing, and restaurant delivery, but according to some insiders, the vision is more far-reaching: an omniscient voice that knows all experience and anticipates all action. Already, forward-looking Amazon patents include the development of a “voice-sniffer algorithm” integrated into any device and able to respond to hot words such as “bought,” “dislike,” or “love” with product and service offers.

Amazon is on the hunt for behavioral surplus. This explains why the company joined Apple and Google in the contest for your car’s dashboard, forging alliances with Ford and BMW. “Shopping from the steering wheel” means hosting behavioral futures markets in your front seat. Alexa is ready with restaurant recommendations or advice on where to get your tires checked. “As pervasive as possible” explains why Amazon wants its Echo/Alexa device to also function as a home phone, able to make and receive calls; why it inked an agreement to install Echo in the nearly 5,000 rooms of the Wynn resort in Las Vegas; and why it is selling Alexa to call centers to automate the process of responding to live questions from customers by phone and text. Each expansion of Alexa’s territory increases the volume of voice surplus accrued to Amazon’s servers and fed to Alexa.

Google also wants its “personal assistant,” Google Home, to double as your home phone. Samsung reappears as another contender with its acquisition of “Viv,” a powerful new voice system designed by the original developers of Apple’s personal assistant Siri, who were frustrated with the constraints of Apple’s approach. Viv’s lead developer explained that “you can get things done by talking to things… a marketplace that will become the next big area.…”

If life is a wild horse, then the digital assistant is one more means by which that horse is to be broken by rendition. Unruly life is brought to heel, rendered as behavioral data and reimagined as a territory for browsing, searching, knowing, and modifying. Just as surveillance capitalism transformed the web into a market onslaught fueled by the capture and analysis of behavioral surplus, so everyday life is set to become a mere canvas for the explosion of a new always-on market cosmos dedicated to our behavior and from which there is no escape.

“We are used to face-to-face interaction where words disappear.… I assumed that keyboard communication was like a letter or a phone call, but now I understand that it doesn’t disappear. The myth is that electronic communication is invisible.…” He was a brilliant research scientist at a large pharmaceuticals company that I called “Drug Corp” in my 1988 book, In the Age of the Smart Machine. I had spent several years visiting the research group there as they shifted much of their daily communications from face-to-face meetings to DIALOG, one of the world’s first “computer conferencing” systems. DIALOG was a precursor to a technology that we now call “social media.” The DIALOG platform created a new social space in which the scientific community at Drug Corp elaborated and extended “their networks of relationships, access to information, thoughtful dialogue, and social banter,” as I described it then. They embraced DIALOG with great enthusiasm, but it did not end well. “In time,” I wrote, “it became clear that they had also unwittingly exposed once evanescent and intangible aspects of their social exchange to an unprecedented degree of hierarchical scrutiny.” The interviews that stretched across those years documented the scientists’ gradual awakening to new dangers as dimensions of personal experience that were implicit and private suddenly became explicit and public in ways that they did not anticipate and with consequences that they lamented deeply.

Thanks to the new computer-mediated milieu, the scientists’ social and professional conversations now appeared as an electronic text: visible, knowable, shareable. It enriched their work in many ways, but it also created unexpected vulnerabilities as dispositions, values, attitudes, and social interactions were recast as objects of scrutiny. In a series of conflicts that unfolded over the years, I watched as the managers and executives at Drug Corp were simply unable to quell their inclination to use the new social text as a means to evaluate, critique, and punish. On more than one occasion I learned of managers who had printed out pages of DIALOG conversations in order to spread them out on the floor and analyze opinions on a particular subject, sometimes actually taking scissors to the pages and organizing the entries by theme or person.

The enduring witness of the text was adapted as a medium through which managers attempted “to control and channel what had always been the most ephemeral aspects of subordinates’ behavior.” All the original excitement and promise melted into cynicism and anxiety as the scientists gradually withdrew from DIALOG, opting instead for a routine e-mail application and a preference for more-perfunctory, impersonal messages.

“Personalization” is once again the euphemism that spearheads this generation of prediction products manufactured from the raw materials of the self. These innovations extend the logic of earlier iterations of dispossession: from web crawling to reality crawling to life crawling to self crawling. As has been the case in each iteration, insights and techniques once intended to illuminate and enrich quickly disappear into the magnetic field of the commercial surveillance project, only to reappear later as ever more cunning methods of supply, manufacture, and sales.

In a 2012 paper Kosinski and Stillwell concluded that “user personality can be easily and effectively predicted from public data” and warned that social media users are dangerously unaware of the vulnerabilities that follow their innocent but voluminous personal disclosures. Their discussion specifically cited Facebook CEO Mark Zuckerberg’s unilateral upending of established privacy norms in 2010, when he famously announced that Facebook users no longer have an expectation of privacy. Zuckerberg had described the corporation’s decision to unilaterally release users’ personal information, declaring, “We decided that these would be the social norms now, and we just went for it.”

In 2013 another provocative study by Kosinski, Stillwell, and Microsoft’s Thore Graepel revealed that Facebook “likes” could “automatically and accurately estimate a wide range of personal attributes that people would typically assume to be private,” including sexual orientation, ethnicity, religious and political views, personality traits, intelligence, happiness, use of addictive substances, parental separation, age, and gender.

The authors appeared increasingly ambivalent about the social implications of their work. On the one hand, they announced that these new predictive capabilities could be used to “improve numerous products and services.” They concluded that online businesses can adjust their behavior to match each user’s personality, with marketing and product recommendations psychologically tailored to each individual. But the authors also warned that automated prediction engines run by companies, governments, or Facebook itself can compute millions of profiles without individual consent or awareness, discovering facts “that an individual may not have intended to share.” The researchers cautioned that “one can imagine situations in which such predictions, even if incorrect, could pose a threat to an individual’s well-being, freedom, or even life.”

Despite these ethical quandaries, by 2015 Kosinski had moved to Stanford University (first to the Computer Science Department and then to the Graduate School of Business), where his research quickly attracted funding from the likes of Microsoft, Boeing, Google, the National Science Foundation, and the Defense Advanced Research Projects Agency (DARPA).

A paper published in 2015 broke fresh ground again by announcing that the accuracy of the team’s computer predictions had equaled or outpaced that of human judges, both in the use of Facebook “likes” to assess personality traits based on the five-factor model and to predict “life outcomes” such as “life satisfaction,” “substance use,” or “depression.” The study made clear that the real breakthrough of the Facebook prediction research was the achievement of economies in the exploitation of these most-intimate behavioral depths with “automated, accurate, and cheap personality assessment tools” that effectively target a new class of “objects” once known as your “personality.”

Personality analysis for commercial advantage is built on behavioral surplus—the so-called meta-data or mid-level metrics—honed and tested by researchers and destined to foil anyone who thinks that she is in control of the “amount” of personal information that she reveals in social media. In the name of, for example, affordable car insurance, we must be coded as conscientious, agreeable, and open. This is not easily faked because the surplus retrieved for analysis is necessarily opaque to us. We are not scrutinized for substance but for form. The price you are offered does not derive from what you write about but how you write it. It is not what is in your sentences but in their length and complexity, not what you list but that you list, not the picture but the choice of filter and degree of saturation, not what you disclose but how you share or fail to, not where you make plans to see your friends but how you do so: a casual “later” or a precise time and place? Exclamation marks and adverb choices operate as revelatory and potentially damaging signals of your self.

That the “personality” insights themselves are banal should not distract us from the fact that the volume and depth of the new surplus supplies enabled by these extraction operations are unprecedented; nothing like this has ever been conceivable. As Kosinski told an interviewer in 2015, few people understand that companies such as “Facebook, Snapchat, Microsoft, Google and others have access to data that scientists would not ever be able to collect.” Data scientists have successfully predicted traits on the five-factor personality model with surplus culled from Twitter profile pictures (color, composition, image type, demographic information, facial presentation, and expression…), selfies (color, photographic styles, visual texture…), and Instagram photos (hue, brightness, saturation…). Others have tested alternative algorithmic models and personality constructs. Another research team demonstrated the ability to predict “satisfaction with life” from Facebook messages. This new world has no manager on his or her hands and knees scissoring pages of computer conferencing messages into thematically organized piles of paper scraps. It’s not the office floor that is crawled. It is you.

Kosinski observed that “all of our interactions are being mediated through digital products and services which basically means that everything is being recorded.” He even characterized his own work as “pretty creepy”: “I actually want to stress that I think that many of the things that… one can do should certainly not be done by corporations or governments without users’ consent.” Recognizing the woefully asymmetric division of learning, he lamented the refusals of Facebook and the other internet firms to share their data with the “general public,” concluding that “it’s not because they’re evil, but because the general public is bloody stupid… as a society we lost the ability to convince large companies that have enormous budgets and enormous access to data to share this goodness with us.… We should basically grow up finally and stop it.”

By early 2015, IBM announced that its Watson Personality Service was open for business. The corporation’s machine intelligence tools are even more complex and invasive than those used in most academic studies. In addition to the five-factor personality model, IBM assesses each individual across twelve categories of “needs,” including “Excitement, Harmony, Curiosity, Ideal, Closeness, Self-expression, Liberty, Love, Practicality, Stability, Challenge, and Structure.” It then identifies “values,” defined as “motivating factors which influence a person’s decision-making across five dimensions: Self-transcendence/Helping others, Conservation/Tradition, Hedonism/Taking pleasure in life, Self-enhancement/Achieving success, and Open to change/Excitement.”

IBM promises “limitless” applications of its new surplus supplies and “deeper portraits of individual customers.” As we would expect, these operations are tested among captive employees who, once habituated, can become docile members of a behaviorally purified society.

Thanks to rendition, a handful of now-measurable personal characteristics, including the “need for love,” predict the likelihood of “liking a brand.” In a Twitter targeted-ad experiment, IBM found that it could significantly increase click-through rates and “follow” rates by targeting individuals with high “openness” and low “neuroticism” scores on the five-factor personality analysis. In another study, IBM rendered behavioral data from 2,000 Twitter users to establish metrics such as response rates, activity levels, and elapsed time between tweets, in addition to psycholinguistic analyses of tweets and five-factor personality analysis. IBM “trained” its predictive model by asking the 2,000 users either location-related or product-related questions. The findings showed that personality information predicted the likelihood of responses. People whom the machines rated as moral, trusting, friendly, extroverted, and agreeable tended to respond, compared to low response rates from people rated as cautious and anxious. Many of the characteristics that we try to teach our children and model in our own behavior are simply repurposed as dispossession opportunities for hidden machine processes of rendition. In this new world, paranoia and anxiety function as sources of protection from machine invasion for profit. Must we teach our children to be anxious and suspicious?

Among this new cohort of mercenaries was Cambridge Analytica, the UK consulting firm owned by the reclusive billionaire and Donald Trump backer Robert Mercer. The firm’s CEO, Alexander Nix, boasted of its application of personality-based “micro-behavioral targeting” in support of the “Leave” and the Trump campaigns during the ramp-up to the 2016 Brexit vote and the US presidential election. Nix claimed to have data resolved “to an individual level where we have somewhere close to four or five thousand data points on every adult in the United States.” While scholars and journalists tried to determine the truth of these assertions and the role that these techniques might have played in both 2016 election upsets, the firm’s new chief revenue officer quietly announced the firm’s less glamorous but more lucrative postelection strategy: “After this election, it’ll be full-tilt into the commercial business.” Writing in a magazine for car dealers just after the US election, he tells them that his new analytic methods reveal “how a customer wants to be sold to, what their personality type is, and which methods of persuasion are most effective.… What it does is change people’s behavior through carefully crafted messaging that resonates with them.… It only takes small improvements in conversion rates for a dealership to see a dramatic shift in revenue.”

Kogan was well-known to Facebook. He had collaborated with its data scientists on a 2013 project in which the company provided data on 57 billion “friendships.” This time, he paid approximately 270,000 people to take a personality quiz. Unknown to these participants, Kogan’s app enabled him to access their Facebook profiles and, on average, the profiles of about 160 of each of the test takers’ friends, “none of whom would have known or had reason to suspect” this invasion. It was a massive rendition operation from which Kogan successfully produced psychological profiles of somewhere between 50 and 87 million Facebook users, data that he then sold to Cambridge Analytica.

“We exploited Facebook to harvest millions of people’s profiles,” Wylie admitted, “and built models to exploit what we knew about them and target their inner demons.” His summary of Cambridge Analytica’s accomplishments is a précis of the surveillance capitalist project and a rationale for its determination to render from the depths. These are the very capabilities that have gathered force over the nearly two decades of surveillance capitalism’s incubation in lawless space. These practices produced outrage around the world, when in fact they are routine elements in the daily elaboration of surveillance capitalism’s methods and goals, both at Facebook and within other surveillance capitalist companies. Cambridge Analytica merely reoriented the surveillance capitalist machinery from commercial markets in behavioral futures toward guaranteed outcomes in the political sphere.

Its operations were designed to produce ignorance through secrecy and the careful evasion of individual awareness. Wylie calls this "information warfare," correctly acknowledging the asymmetries of knowledge and power that are essential to the means of behavioral modification: I think it's worse than bullying, because people don't necessarily know it's being done to them.

Irrespective of Cambridge Analytica’s actual competence and its ultimate political impact, the plotting and planning behind its ambitions are testament to the pivotal role of rendition from the depths in the prediction and modification of behavior, always in pursuit of certainty. Billionaires such as Zuckerberg and Mercer have discovered that they can muscle their way to dominance of the division of learning in society by setting their sights on these rendition operations and the fortunes they tell. They aim to be unchallenged in their power to know, to decide who knows, and to decide who decides. The rendition of “personality” was an important milestone in this quest: a frontier, yes, but not the final frontier.

In 2015 an eight-year-old startup named Realeyes won a 3.6 million euro grant from the European Commission for a project code-named “SEWA: Automatic Sentiment Analysis in the Wild.” The aim was “to develop automated technology that will be able to read a person’s emotion when they view content and then establish how this relates to how much they liked the content.” The director of video at AOL International called the project “a huge leap forward in video ad tech” and “the Holy Grail of video marketing.” Just a year later, Realeyes won the commission’s Horizon 2020 innovation prize thanks to its “machine learning-based tools that help market researchers analyze the impact of their advertising and make it more relevant.”

Emotion analytics products such as SEWA use specialized software to scour faces, voices, gestures, bodies, and brains, all of it captured by “biometric” and “depth” sensors, often in combination with imperceptibly small, “unobtrusive” cameras. This complex of machine intelligence is trained to isolate, capture, and render the most subtle and intimate behaviors, from an inadvertent blink to a jaw that slackens in surprise for a fraction of a second. Combinations of sensors and software can recognize and identify faces; estimate age, ethnicity, and gender; analyze gaze direction and blinks; and track distinct facial points to interpret “micro-expressions,” eye movements, emotions, moods, stress, deceit, boredom, confusion, intentions, and more: all at the speed of life.

These behaviors also elude the conscious mind. The machines capture the nanosecond of disgust that precedes a rapid-fire sequence of anger, comprehension, and finally joy on the face of a young woman watching a few frames of film, when all she can think to say is “I liked it!”

Clients are advised to “play your audience emotions to stay on top of the game.” The company’s website offers a brief review of the history of research on human emotions, concluding that “the more people feel, the more they spend.… Intangible ‘emotions’ translate into concrete social activity, brand awareness, and profit.”

This is not the first time that the unconscious mind has been targeted as an instrument of others' aims. Propaganda and advertising have always been designed to appeal to unacknowledged fears and yearnings. These have relied more on art than science, using gross data or professional intuition for the purpose of mass communication. Those operations cannot be compared to the scientific application of today's historic computational power to the micro-measured, continuous rendition of your more-or-less actual feelings. The new toolmakers do not intend to rob you of your inner life, only to surveil and exploit it. All they ask is to know more about you than you know about yourself.

The program of emotional rendition began innocently enough with MIT Media Lab professor Rosalind Picard and the new field of computer science that she called “affective computing.” She was among the first to recognize the opportunity for a computational system to automate the analysis of Ekman’s facial configurations and correlate micro-expressions with their emotional causality. She aimed to combine facial expression with the computation of vocal intonation and other physiological signals of emotion that could be measured as behavior. In 1997 she published Affective Computing, which posited a practical solution to the idea that some emotions are available to the conscious mind and can be expressed “cognitively” (“I feel scared”), whereas others may elude consciousness but nevertheless be expressed physically in beads of sweat, a widening of the eyes, or a nearly imperceptible tightening of the jaw.

The key to affective computing, Picard argued, was to render both conscious and unconscious emotion as observable behavior for coding and calculation. A computer, she reasoned, would be able to render your emotions as behavioral information. Affect recognition, as she put it, is “a pattern recognition problem,” and “affect expression” is pattern synthesis. The proposition was that “computers can be given the ability to recognize emotions as well as a third-person human observer.”

Picard imagined her affective insights being put to use in ways that were often good-hearted or, at the very least, benign. Most of the applications she describes conform to the logic of the Aware Home: any knowledge produced would belong to the subject to enhance his or her reflexive learning. For example, she envisioned a “computer-interviewing agent” that could function as an “affective mirror” coaching a student in preparation for a job interview or a date and an automatic agent that could alert you to hostile tones in your own prose before you press “send.” Picard anticipated other tools combining software and sensors that she believed could enhance daily life in a range of situations, such as helping autistic children develop emotional skills, providing software designers with feedback on users’ frustration levels, assigning points to video game players to reward courage or stress reduction, producing learning modules that stimulate curiosity and minimize anxiety, and analyzing emotional dynamics in a classroom. She imagined software agents that learn your preferences and find you the kinds of news stories, clothing, art, or music that make you smile. Whatever one’s reaction to these ideas might be, they share one key pattern: unlike SEWA’s model, Picard’s data were intended to be for you, not merely about you.

Back in 1997, Picard acknowledged the need for privacy “so that you remain in control over who gets access to this information.” Importantly for our analysis, in the final pages of her book she expressed some concerns, writing that “there are good reasons not to broadcast your affective patterns to the world… you might flaunt your good mood in front of friends… you probably do not want it picked up by an army of sales people who are eager to exploit mood-based buying habits, or by advertisers eager to convince you that you’d feel better if you tried their new soft drink right now.” She noted the possibility of intrusive workplace monitoring, and she voiced reservations about the possibility of a dystopian future in which “malevolent” governmental forces use affective computing to manipulate and control the emotions of populations.

In a pattern that is by now all too familiar, safeguards lagged as surveillance capitalism flourished. By early 2014, Facebook had already applied for an “emotion detection” patent designed to implement each of Picard’s fears. The idea was “one or more software modules capable of detecting emotions, expressions or other characteristic of a user from image information.” As always, the company was ambitious. Its list of detectable emotions “by way of example and not limitation” included expressions such as “a smile, joy, humor, amazement, excitement, surprise, a frown, sadness, disappointment, confusion, jealousy, indifference, boredom, anger, depression, or pain.”

Picard would eventually become part of this new dispossession industry with a company called Affectiva that was cofounded with Rana el Kaliouby, an MIT Media Lab postdoctoral research scientist and Picard protégé. The company’s transformation from doing good to doing surveillance capitalism is a metaphor for the fate of the larger undertaking of emotion analysis as it is rapidly drawn into the competitive maelstrom for surveillance revenues.

Both Picard and Kaliouby had shared a vision of applying their research in medical and therapeutic settings. The challenges of autistic children seemed a perfect fit for their discoveries, so they trained a machine system called MindReader to recognize emotions using paid actors to mimic specific emotional responses and facial gestures. Early on, MIT Media Lab corporate sponsors Pepsi, Microsoft, Bank of America, Nokia, Toyota, Procter and Gamble, Gillette, Unilever, and others had overwhelmed the pair with queries about using their system to measure customers' emotional responses. Kaliouby describes the women's hesitation and their determination to focus on "do-good" applications. According to her account, the Media Lab encouraged the two women to "spin off" their work into the startup they called Affectiva, imagined as a "baby IBM for emotionally intelligent machines."

It wasn't long before the new company found itself fielding significant interest from ad agencies and marketing firms itching for automated rendition and analysis from the depths. Describing that time, as Picard told one journalist, "Our CEO was absolutely not comfortable with the medical space." As a result, Picard was "pushed out" of the firm three years after its founding. As an Affectiva researcher recounted, "We began with a powerful set of products that could assist people who have a very difficult time with perceiving affect.… Then they started to emphasize only the face, to focus on advertisements, and on predicting whether someone likes a product, and just went totally off the original mission."

This is the commercial context in which Kaliouby came to feel that it is perfectly reasonable to assert that an “emotion chip” will become the base operational unit of a new “emotion economy.” She speaks to her audiences of a chip embedded in all things everywhere, running constantly in the background, producing an “emotion pulse” each time you check your phone: “I think in the future we’ll assume that every device just knows how to read your emotions.” At least one company, Emoshape, has taken her proposition seriously. The firm, whose tagline is “Life Is the Value,” produces a microchip that it calls “the industry’s first emotion synthesis engine,” delivering “high performance machine emotion awareness.” The company writes that its chip can classify twelve emotions with up to 98 percent accuracy, enabling its “artificial intelligence or robot to experience 64 trillion possible distinct emotional states.”

Kaliouby imagines that pervasive “emotion scanning” will come to be as taken for granted as a “cookie” planted in your computer to track your online browsing. After all, those cookies once stirred outrage, and now they inundate every online move. … As a start, Affectiva pioneered the notion of “emotion as a service,” offering its analytics on demand: “Just record people expressing emotion and then send those videos or images to us to get powerful emotion metrics back.”

The possibilities in the depth dimension seem endless, and perhaps they will be if Affectiva, its clients, and fellow travelers are free to plunder our selves at will. … “Happiness as a service” seems to be within reach. “I do believe that if we have information about your emotional experiences we can help you be in a positive mood,” Kaliouby says. She imagines emotion-recognition systems issuing reward points for happiness because, after all, happy customers are more “engaged.”

Rendition is by now a global project of surveillance capital, and in the depth dimension we see it at its most pernicious. Intimate territories of the self, like personality and emotion, are claimed as observable behavior and coveted for their rich deposits of predictive surplus. Now the personal boundaries that shelter inner life are officially designated as bad for business by a new breed of mercenaries of the self determined to parse and package inner life for the sake of surveillance revenues. Their expertise disrupts the very notion of the autonomous individual by rewarding “boundarylessness” with whatever means are available—offers of elite status, bonuses, happiness points, discounts, “buy” buttons pushed to your device at the precise moment predicted for maximum success—so that we might strip and surrender to the pawing and prying of the machines that serve the new market cosmos.

I want to deliberately sidestep a more detailed discussion of what is “personality” or “emotion,” “conscious” or “unconscious,” in favor of what I hope is a less fractious truth thrown into relief by this latest phase of incursion. Experience is not what is given to me but rather what I make of it. The same experience that I deride may invite your enthusiasm. The self is the inward space of lived experience from which such meanings are created. In that creation I stand on the foundation of personal freedom: the “foundation” because I cannot live without making sense of my experience.

No matter how much is taken from me, this inward freedom to create meaning remains my ultimate sanctuary. Jean-Paul Sartre writes that “freedom is nothing but the existence of our will,” and he elaborates: “Actually it is not enough to will; it is necessary to will to will.” This rising up of the will to will is the inner act that secures us as autonomous beings who project choice into the world and exercise the qualities of self-determining moral judgment that are civilization’s necessary and final bulwark. This is the sense behind another of Sartre’s insights: “Without bearings, stirred by a nameless anguish, the words labor.… The voice is born of a risk: either to lose oneself or win the right to speak in the first person.”

As the prediction imperative drives deeper into the self, the value of its surplus becomes irresistible, and cornering operations escalate. What happens to the right to speak in the first person from and as my self when the swelling frenzy of institutionalization set into motion by the prediction imperative is trained on cornering my sighs, blinks, and utterances on the way to my very thoughts as a means to others’ ends? It is no longer a matter of surveillance capital wringing surplus from what I search, buy, and browse. Surveillance capital wants more than my body’s coordinates in time and space. Now it violates the inner sanctum as machines and their algorithms decide the meaning of my breath and my eyes, my jaw muscles, the hitch in my voice, and the exclamation points that I offered in innocence and hope.

Picard did not foresee the market forces that would transform the rendition of emotion into for-profit surplus: means to others' ends. That her vision is made manifest in thousands of activities should be a triumph, but it is diminished by the fact that so many of those activities are now bound to the commercial surveillance project. Each failure to establish bearings contributes to habituation, normalization, and ultimately legitimation. Subordinated to the larger aims of surveillance capitalism, the thrust of the affective project changed as if distorted in a fun-house mirror.

The new power is action," a senior software engineer told me. "The intelligence of the internet of things means that sensors can also be actuators." The director of software engineering for a company that is an important player in the "internet of things" added, "It's no longer simply about ubiquitous computing. Now the real aim is ubiquitous intervention, action, and control. The real power is that now you can modify real-time actions in the real world. Connected smart sensors can register and analyze any kind of behavior and then actually figure out how to change it. Real-time analytics translate into real-time action." The scientists and engineers I interviewed call this new capability "actuation," and they describe it as the critical though largely undiscussed turning point in the evolution of the apparatus of ubiquity.

This actuation capability defines a new phase of the prediction imperative that emphasizes economies of action. This phase represents the completion of the new means of behavior modification, a decisive and necessary evolution of the surveillance capitalist "means of production" toward a more complex, iterative, and muscular operational system. It is a critical achievement in the race to guaranteed outcomes. Under surveillance capitalism the objectives and operations of automated behavioral modification are designed and controlled by companies to meet their own revenue and growth objectives.

Sensors are used to modify people's behavior just as easily as they modify device behavior. There are many great things we can do with the internet of things, like lowering the heat in all the houses on your street so that the transformer is not overloaded, or optimizing an entire industrial operation. But at the individual level, it also means the power to take actions that can override what you are doing or even put you on a path you did not choose.

Surveillance capitalists adapted many of the highly contestable assumptions of behavioral economists as one cover story with which to legitimate their practical commitment to a unilateral commercial program of behavior modification. The twist here is that nudges are intended to encourage choices that accrue to the architect, not to the individual. The result is data scientists trained on economies of action who regard it as perfectly normal to master the art and science of the “digital nudge” for the sake of their company’s commercial interests. For example, the chief data scientist for a national drugstore chain described how his company designs automatic digital nudges that subtly push people toward the specific behaviors favored by the company: “You can make people do things with this technology. Even if it’s just 5% of people, you’ve made 5% of people do an action they otherwise wouldn’t have done, so to some extent there is an element of the user’s loss of self-control.”

"Herding" is a second approach that relies on controlling key elements in a person's immediate context. The uncontract is an example of a herding technique. Shutting down a car's engine irreversibly changes the driver's immediate context, herding her out the car door. Herding enables remote orchestration of the human situation, foreclosing action alternatives and thus moving behavior along a path of heightened probability that approximates certainty. "We are learning how to write the music, and then we let the music make them dance," an "internet of things" software developer explains, adding, We can engineer the context around a particular behavior and force change that way.

We can engineer the context around a particular behavior and force change that way. Context-aware data allow us to tie together your emotions, your cognitive functions, your vital signs, etcetera. We can know if you shouldn’t be driving, and we can just shut your car down. We can tell the fridge, “Hey, lock up because he shouldn’t be eating,” or we tell the TV to shut off and make you get some sleep, or the chair to start shaking because you shouldn’t be sitting so long, or the faucet to turn on because you need to drink more water.

“Conditioning” is a well-known approach to inducing behavior change, primarily associated with the famous Harvard behaviorist B. F. Skinner. He argued that behavior modification should mimic the evolutionary process, in which naturally occurring behaviors are “selected” for success by environmental conditions. Instead of the earlier, more simplistic model of stimulus/response, associated with behaviorists such as Watson and Pavlov, Skinner interpolated a third variable: “reinforcement.” In his laboratory work with mice and pigeons, Skinner learned how to observe a range of naturally occurring behaviors in the experimental animal and then reinforce the specific action, or “operant,” that he wanted the animal to reproduce. Ultimately, he mastered intricate designs or “schedules” of reinforcement that could reliably shape precise behavioral routines.

The goal of everything we do is to change people’s actual behavior at scale. We want to figure out the construction of changing a person’s behavior, and then we want to change how lots of people are making their day-to-day decisions. When people use our app, we can capture their behaviors and identify good and bad [ones]. Then we develop “treatments” or “data pellets” that select good behaviors. We can test how actionable our cues are for them and how profitable certain behaviors are for us.

Although it is still possible to imagine automated behavioral modification without surveillance capitalism, it is not possible to imagine surveillance capitalism without the marriage of behavior modification and the technological means to automate its application. This marriage is essential to economies of action. For example, one can imagine a fitness tracker, a car, or a refrigerator whose data and operational controls are accessible exclusively to their owners for the purposes of helping them to exercise more often, drive safely, and eat healthily. But as we have already seen in so many domains, the rise of surveillance capitalism has obliterated the idea of the simple feedback loop characteristic of the behavioral value reinvestment cycle. In the end, it’s not the devices; it’s Max Weber’s “economic orientation,” now determined by surveillance capitalism.

The allure of surveillance revenues drives the continuous accumulation of more and more predictive forms of behavioral surplus. The most predictive source of all is behavior that has already been modified to orient toward guaranteed outcomes.

Remember that Google economist Hal Varian extolled the “new uses” of big data that proceed from ubiquitous computer-mediated transactions. Among these he included the opportunity for “continuous experimentation.” Varian noted that Google has its engineering and data science teams consistently running thousands of “A/B” experiments that rely on randomization and controls to test user reactions to hundreds of variations in page characteristics from layout to buttons to fonts. … Varian endorsed and celebrated this self-authorizing experimental role, warning that all the data in the world “can only measure correlation, not causality.” Data tell what happened but not why it happened. In the absence of causal knowledge, even the best predictions are only extrapolations from the past.

The result of this conundrum is that the last crucial element in the construction of high-quality prediction products—i.e., those that approximate guaranteed outcomes—depends upon causal knowledge. As Varian says, “If you really want to understand causality, you have to run experiments. And if you run experiments continuously, you can continuously improve your system.”

Varian awards surveillance capitalists the privilege of the experimenter’s role, and this is presented as another casual fait accompli. In fact, it reflects the final critical step in surveillance capitalists’ radical self-dealing of new rights. In this phase of the prediction imperative, surveillance capitalists declare their right to modify others’ behavior for profit according to methods that bypass human awareness, individual decision rights, and the entire complex of self-regulatory processes that we summarize with terms such as autonomy and self-determination.

What follows now are two distinct narratives of surveillance capitalists as “experimenters” who leverage their asymmetries of knowledge to impose their will on the unsuspecting human subjects who are their users. The experimental insights accumulated through their one-way mirrors are critical to constructing, fine-tuning, and exploring the capabilities of each firm’s for-profit means of behavioral modification. In Facebook’s user experiments and in the augmented-reality game Pokémon Go (imagined and incubated at Google), we see the commercial means of behavioral modification evolving before our eyes. Both combine the components of economies of action and the techniques of tuning, herding, and conditioning in startling new ways that expose the Greeks secreted deep in the belly of the Trojan horse: the economic orientation obscured behind the veil of the digital.

In 2012 Facebook researchers startled the public with an article provocatively titled “A 61-Million-Person Experiment in Social Influence and Political Mobilization,” published in the scientific journal Nature. In this controlled, randomized study conducted during the run-up to the 2010 US Congressional midterm elections, the researchers experimentally manipulated the social and informational content of voting-related messages in the news feeds of nearly 61 million Facebook users while also establishing a control group.

The results showed that users who received the social message were about 2 percent more likely to click on the “I Voted” button than did those who received the information alone and 0.26 percent more likely to click on polling place information. The Facebook experimenters determined that social messaging was an effective means of tuning behavior at scale because it “directly influenced political self-expression, information seeking and real-world voting behavior of millions of people,” and they concluded that “showing familiar faces to users can dramatically improve the effectiveness of a mobilization message.”

The team calculated that the manipulated social messages sent 60,000 additional voters to the polls in the 2010 midterm elections, as well as another 280,000 who cast votes as a result of a “social contagion” effect, for a total of 340,000 additional votes. In their concluding remarks, the researchers asserted that “we show the importance of social influence for effecting behavior change… the results suggest that online messages might influence a variety of offline behaviors, and this has implications for our understanding of the role of online social media in society.…”

The experiment succeeded by producing social cues that “suggested” or “primed” users in ways that tuned their real-world behavior toward a specific set of actions determined by the “experimenters.” In this process of experimentation, economies of action are discovered, honed, and ultimately institutionalized in software programs and their algorithms that function automatically, continuously, ubiquitously, and pervasively to achieve economies of action. Facebook’s surplus is aimed at solving one problem: how and when to intervene in the state of play that is your daily life in order to modify your behavior and thus sharply increase the predictability of your actions now, soon, and later.

Experiments are run on every user at some point in their tenure on the site. Whether that is seeing different size ad copy, or different marketing messages, or different call-to-action buttons, or having their feeds generated by different ranking algorithms.… The fundamental purpose of most people at Facebook working on data is to influence and alter people’s moods and behavior. They are doing it all the time to make you like stories more, to click on more ads, to spend more time on the site. This is just how a website works, everyone does this and everyone knows that everyone does this.

The Facebook study’s publication evoked fierce debate as experts and the wider public finally began to reckon with Facebook’s—and the other internet companies’—unprecedented power to persuade, influence, and ultimately manufacture behavior. Harvard’s Jonathan Zittrain, a specialist in internet law, acknowledged that it was now possible to imagine Facebook quietly engineering an election, using means that its users could neither detect nor control.

Public concern failed to destabilize Facebook’s self-authorizing practice of behavior modification at scale. Even as the social influence experiment was being debated in 2012, a Facebook data scientist was already collaborating with academic researchers on a new study, “Experimental Evidence of Massive-Scale Emotional Contagion Through Social Networks,” submitted to the prestigious Proceedings of the National Academy of Sciences in 2013, where it was edited by a well-known Princeton social psychologist, Susan Fiske, and published in June 2014.

This time the experimenters “manipulated the extent to which people (N = 689,003) were exposed to emotional expressions in their News Feed.” The experiment was structured like one of those allegedly benign A/B tests. In this case one group was exposed to mostly positive messages in their news feed and the other to predominantly negative messages. The idea was to test whether even subliminal exposure to specific emotional content would cause people to change their own posting behavior to reflect that content. It did. Whether or not users felt happier or sadder, the tone of their expression changed to reflect their news feed.

The experimental results left no doubt that once again Facebook’s carefully designed, undetectable, and uncontestable subliminal cues reached beyond the screen into the daily lives of hundreds of thousands of naive users, predictably actuating specific qualities of emotional expression through processes that operate outside the awareness of their human targets, just as Stuart MacKay had originally prescribed for Galapagos turtles and Canadian elk (see Chapter 7). … Online messages influence our experience of emotions, which may affect a variety of offline behaviors. … The team celebrated its work as "some of the first experimental evidence to support the controversial claims that emotions can spread throughout a network," and they reflected on the fact that even their relatively minimal manipulation had a measurable effect, albeit a small one.

What Facebook researchers failed to acknowledge in either experiment is that a person’s susceptibility to subliminal cues and his or her vulnerability to a "contagion" effect is largely dependent upon empathy: the ability to understand and share in the mental and emotional state of another person, including feeling another’s feelings and being able to take another’s point of view—sometimes characterized as "affective" or "cognitive" empathy. Psychologists have found that the more a person can project himself or herself into the feelings of another and take the other’s perspective, the more likely he or she is to be influenced by subliminal cues, including hypnosis.

Empathy is considered essential to social bonding and emotional attachment, but it can also trigger "vicarious anxiety" for victims or others who are genuinely distressed. Some psychologists have called empathy a "risky strength" because it predisposes us to experience others’ happiness but also their pain. The successful tuning evident in both Facebook experiments is the result of the effective exploitation of the natural empathy present in its population of users.

Fiske acknowledged that she had been persuaded by Facebook's argument that the experimental manipulation was an unremarkable extension of the corporation's standard practice of manipulating people's news feeds. As Fiske recounted, "They said… that Facebook apparently manipulates people's News Feeds all the time.… Who knows what other research they're doing." In other words, Fiske recognized that the experiment was merely an extension of Facebook's standard practices of behavioral modification, which already flourish without sanction.

The Wall Street Journal reported that the Facebook data science group had run more than 1,000 experiments since its inception in 2007 and operated with "few limits" and no internal review board. Writing in the Guardian, psychology professor Chris Chambers summarized that "the Facebook study paints a dystopian future in which academic researchers escape ethical restriction by teaming up with private companies to test increasingly dangerous or harmful interventions."

Grimmelmann imagined "Internal Review Board laundering," in which academics could "circumvent research ethics regulations whenever they work just closely enough with industry partners. The exception would swallow the Common Rule."

Despite his conviction on this point, Grimmelmann acknowledged in the final pages of his analysis that even the most rigorous imposition of the Common Rule would do little to curb the immense power of a company such as Facebook that routinely manipulates user behavior at scale, using means that are indecipherable and therefore uncontestable. Like Fiske, Grimmelmann sensed the larger project of economies of action just beyond the reach of established law and social norms.

Facebook owns an unprecedented means of behavior modification that operates covertly, at scale, and in the absence of social or legal mechanisms of agreement, contest, and control. Even the most stringent application of the “Common Rule” would be unlikely to change these facts.

Three years after the publication of the contagion study, the Australian broke the story on a confidential twenty-three-page Facebook document written by two Facebook executives in 2017 and aimed at the company's Australian and New Zealand advertisers. The report depicted the corporation's systems for gathering “psychological insights” on 6.4 million high school and tertiary students as well as young Australians and New Zealanders already in the workforce. The Facebook document detailed the many ways in which the corporation uses its stores of behavioral surplus to pinpoint the exact moment at which a young person needs a “confidence boost” and is therefore most vulnerable to a specific configuration of advertising cues and nudges: “By monitoring posts, pictures, interactions, and Internet activity, Facebook can work out when young people feel 'stressed,' 'defeated,' 'overwhelmed,' 'anxious,' 'nervous,' 'stupid,' 'silly,' 'useless,' and a 'failure.'"

Facebook publicly denied these practices, but Antonio Garcia-Martinez, a former Facebook product manager and author of a useful account of Silicon Valley titled Chaos Monkeys, described in the Guardian the routine application of such practices and accused the corporation of “lying through their teeth.” He concluded, “The hard reality is that Facebook will never try to limit such use of their data unless the public uproar reaches such a crescendo as to be un-mutable.” Certainly the public challenge to Facebook’s insertion of itself into the emotional lives of its users, as expressed in the contagion study, and its pledge to self-regulate did not quell its commercial interest in users’ emotions or the corporation’s compulsion to systematically exploit that knowledge on behalf of and in collaboration with its customers. It did not, because it cannot, not as long as the company’s revenues are bound to economies of action under the authority of the prediction imperative.

Facebook’s persistence warns us again of the dispossession cycle’s stubborn march. Facebook had publicly acknowledged and apologized for its overt experimental incursions into behavior modification and emotional manipulation, and it promised adaptations to curb or mitigate these practices. Meanwhile, a new threshold of intimate life had been breached. Facebook’s potential mastery of emotional manipulation became discussable and even taken for granted as habituation set in. From Princeton’s Fiske to critic Grimmelmann and supporter Meyer, the experts believed that if Facebook’s activities were to be forced into a new regulatory regime, the corporation would merely continue in secret. The Australian documents opened one door on these covert practices, suggesting the completion of the cycle with the redirection of action into clandestine zones protected by opacity and indecipherability, just as these scholars had anticipated.

The evasion of individual and group awareness was critical to Facebook's behavior-modification success, just as MacKay had stipulated. The first paragraph of the research article on emotional contagion celebrates this evasion: “Emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness.”

This evasion is neither accidental nor incidental, but actually essential to the structure of the whole surveillance capitalist project. Individual awareness is the enemy of telestimulation because it is the necessary condition for the mobilization of cognitive and existential resources. There is no autonomous judgment without awareness. Agreement and disagreement, participation and withdrawal, resistance or collaboration: none of these self-regulating choices can exist without awareness.

A rich and flourishing research literature illuminates the antecedents, conditions, consequences, and challenges of human self-regulation as a universal need. The capacity for self-determination is understood as an essential foundation for many of the behaviors that we associate with critical capabilities such as empathy, volition, reflection, personal development, authenticity, integrity, learning, goal accomplishment, impulse control, creativity, and the sustenance of intimate enduring relationships. “Implicit in this process is a self that sets goals and standards, is aware of its own thoughts and behaviors, and has the capacity to change them,” write Ohio State University professor Dylan Wagner and Dartmouth professor Todd Heatherton in an essay about the centrality of self-awareness to self-determination: “Indeed, some theorists have suggested that the primary purpose of self awareness is to enable self-regulation.” Every threat to human autonomy begins with an assault on awareness, “tearing down our capacity to regulate our thoughts, emotions, and desires.”

The salience of self-awareness as a bulwark against self-regulatory failure is also underscored in the work of two Cambridge University researchers who developed a scale to measure a person's “susceptibility to persuasion.” They found that the single most important determinant of one's ability to resist persuasion is what they call “the ability to premeditate.” This means that people who harness self-awareness to think through the consequences of their actions are more disposed to chart their own course and are significantly less vulnerable to persuasion techniques. Self-awareness also figures in the second-highest-ranking factor on their scale: commitment. People who are consciously committed to a course of action or set of principles are less likely to be persuaded to do something that violates that commitment.

We have seen already that democracy threatens surveillance revenues. Facebook's practices suggest an equally disturbing conclusion: human consciousness itself is a threat to surveillance revenues, as awareness endangers the larger project of behavior modification. Philosophers recognize “self-regulation,” “self-determination,” and “autonomy” as “freedom of will.” The word autonomy derives from the Greek and literally means “regulation by the self.” It stands in contrast to heteronomy, which means “regulation by others.” The competitive necessity of economies of action means that surveillance capitalists must use all means available to supplant autonomous action with heteronomous action.

However, it would be dangerous to nurse the notion that today's surveillance capitalists simply represent more of the same. This structural requirement of economies of action turns the means of behavioral modification into an engine of growth. At no other time in history have private corporations of unprecedented wealth and power enjoyed the free exercise of economies of action supported by a pervasive global architecture of ubiquitous computational knowledge and control constructed and maintained by all the advanced scientific know-how that money can buy.

Most pointedly, Facebook's declaration of experimental authority claims surveillance capitalists' prerogatives over the future course of others' behavior. In declaring the right to modify human action secretly and for profit, surveillance capitalism effectively exiles us from our own behavior, shifting the locus of control over the future tense from “I will” to “You will.” Each one of us may follow a distinct path, but economies of action ensure that the path is already shaped by surveillance capitalism's economic imperatives. The struggle for power and control in society is no longer associated with the hidden facts of class and its relationship to production but rather by the hidden facts of automated engineered behavior modification.

The doorbell rang just as the warm water hit his aching back muscles. Had she forgotten her key? Shower interrupted, he threw on a tee and shorts and ran downstairs, opening the front door to a couple of teenagers waving their cell phones in his face. “Hey, you’ve got a Pokémon in your backyard. It’s ours! Okay if we go back there and catch it?”

Back in 2009, as we saw in Chapter 5, Google Maps product vice president and Street View boss John Hanke ignored the Broughton protestors, insisting that only he and Google knew what was best, not just for Broughton but for all people. Now here was Hanke again at surveillance capitalism’s next frontier, this time as the founder of the company behind Pokémon Go, Niantic Labs. Hanke, you may recall, nursed an abiding determination to own the world by mapping it. He had founded Keyhole, the satellite mapping startup funded by the CIA and later acquired by Google and rechristened as Google Earth. At Google, he was a vice president for Google Maps and a principal in its controversial push to commandeer public and private space through its Street View project.

Hanke recounts that Pokémon Go was born out of Google Maps, which also supplied most of the game’s original development team. Indeed, Street View’s mystery engineer, Marius Milner, had joined Hanke in this new phase of incursion. By 2010, Hanke had set up his own launch pad, Niantic Labs, inside the Google mother ship. His aim was the development of “parallel reality” games that would track and herd people through the very territories that Street View had so audaciously claimed for its maps. In 2015, following the establishment of the Alphabet corporate structure and well after the development of Pokémon Go, Niantic Labs was formally established as an independent company with $30 million in funding from Google, Nintendo (the Japanese company that originally hosted Pokémon on its “Game Boy” devices in the late 1990s), and the Pokémon Company.

It is worth noting that Hanke chose to name his group after a nineteenth-century merchant sailing vessel undone by greed. The Niantic had been sold and repurposed for the more lucrative whaling trade when it set sail for San Francisco and the northern Pacific whaling grounds in 1849. The ship’s captain made an unplanned stop in Panama to board hundreds of pilgrims bound for the California Gold Rush, all of them eager to pay top dollar for cramped, smelly quarters on the whaler. The captain’s avarice proved fatal to the ship’s prospects when those passengers infected the ship’s crew with gold fever. The sailors abandoned captain and vessel upon docking in San Francisco, heading instead for gold country. Unable to continue the journey, the captain was forced to sell the ship for a pittance, leaving it wedged deep in the sandy shallows at the foot of Clay and Montgomery streets. In 2016 Hanke took up the quest of that rebellious crew. His Niantic was bound for a new century’s gold rush at the frontier of the prediction imperative’s next wave of conquest: economies of action.

At its zenith in the summer of 2016, Pokémon Go was a surveillance capitalist’s dream come true, fusing scale, scope, and actuation; yielding continuous sources of behavioral surplus; and providing fresh data to elaborate the mapping of interior, exterior, public, and private spaces. Most important, it provided a living laboratory for telestimulation at scale as the game’s owners learned how to automatically condition and herd collective behavior, directing it toward real-time constellations of behavioral futures markets, with all of this accomplished just beyond the rim of individual awareness. In Hanke’s approach, economies of action would be achieved through the dynamics of a game.

Hanke reflected on what he and his team had learned from Ingress. Most important was the Niantic team's “surprise” as they observed how much “the behavior of the players changes.” Hanke grasped that the seeds of behavior modification were planted in the game's rules and social dynamic: “If you want to turn the world into your game board, the places you want people to interact with have to have certain characteristics.… There should be a reason for the player to go there.… The game is enabling them and nudging you to have those interactions.”

All games circumscribe behavior with rules, rewarding some forms of action and punishing others, and Niantic is not the first to employ the structure of a game as a means of effecting behavior change in its players. Indeed, "gamification" as an approach to behavioral engineering is a subject of intense interest that has produced a robust academic and popular literature. According to Wharton professor Kevin Werbach, games include three tiers of action. At the highest level are the "dynamics" that drive the motivational energy of the game. These can be emotions aroused by competition or frustration, a compelling narrative, a structure of progression that creates the experience of development toward a higher goal, or relationships that produce feelings such as team spirit or aggression. Next are the "mechanics." These are the procedural building blocks that drive the action and also build engagement. For example, a game may be structured as a competition or a solo challenge, as turn taking and cooperation, as transactions and winner take all, as team sport or individual conquest. Finally, there are the game "components" that operationalize the mechanics. These are the most-visible aspects of a game: points to represent progress, quests laid out as predefined challenges, "badges" to represent achievements, "leader boards" to visually display all players' progress, "boss fights" to mark the culmination of a level, and so forth.

Most research on games concludes that these structures can be effective at motivating action, and researchers generally predict that games will increasingly be used as the methodology of choice to change individual behavior. In practice, this has meant that the power of games to change behavior is shamelessly instrumentalized as gamification spreads to thousands of situations in which a company merely wants to tune, herd, and condition the behavior of its customers or employees toward its own objectives. Typically, this involves importing a few components, such as reward points and levels of advancement, in order to engineer behaviors that serve the company's immediate interests, with programs such as customer loyalty schemes or internal sales competitions. One analyst compiled a survey of more than ninety such "gamification cases," complete with return-on-investment statistics. Ian Bogost, a professor of interactive computing at Georgia Tech and a digital culture observer, insists that these systems should be called "exploitationware" rather than games because their sole aim is behavior manipulation and modification.

Pokémon Go takes these capabilities in a wholly new direction, running game players through the real world, but not for the sake of the game they think they are playing. Hanke's unique genius is to point the game's behavior-modification efforts toward a target that occupies an unexplored zone beyond the boundaries of players' awareness. It aims to shape behavior in an even larger game of surveillance capitalism.

The unprecedented pattern was faintly discernible within days of the game's launch. A Virginia bar offered a discount to a Pokémon Go team; a tea shop in San Francisco offered a "buy one get one free" to the game's players. The owner of a pizza bar in Queens, New York, paid about $10 for "Lure Modules," a bit of virtual game paraphernalia intended to attract Pokémon to a specific location, successfully producing virtual creatures on bar stools and in bathroom stalls. During the first weekend of game play, the bar's food and drink sales shot up by 30 percent and later were reported to be 70 percent above average. Bloomberg reporters gushed that the game had achieved the retailers' elusive dream of using location tracking to drive foot traffic: "It's easy to imagine a developer selling ads within the game world to local merchants, or even auctioning off the promise to turn specific shops and restaurants into destinations for players."

By July 12, the Financial Times exulted that "speculation has surged over the game's future power as a cash cow to retailers and other cravers of footfall." Nintendo shares were up 52 percent, adding $10.2 billion to its market capitalization.

The notion of "sponsored locations" is a euphemism for Niantic's behavioral futures markets, ground zero in Hanke's new gold rush. The elements and dynamics of the game, combined with its novel augmented-reality technology, operate to herd populations of game players through the real-world monetization checkpoints constituted by the game's actual customers: the entities who pay to play on the real-world game board, lured by the promise of guaranteed outcomes.

The game had demonstrated that it was possible to achieve economies of action on a global scale while simultaneously directing specific individual actions toward precise local market opportunities where high bidders enjoy an ever-closer approximation of guaranteed outcomes.

These requirements suggest that Niantic would conduct its operations in ways that establish substantial surplus supply chains aimed at scale and scope. Indeed, the company's “surveillance policy” signals its demand for behavioral data in excess of what is reasonable for effective game operations. Just six days after the game's release in July 2016, BuzzFeed reporter Joseph Bernstein advised Pokémon users to check how much data the app was collecting from their phones. According to his analysis, “Like most apps that work with the GPS in your smartphone, Pokémon Go can tell a lot of things about you based on your movement as you play: where you go, when you went there, how you got there, how long you stayed, and who else was there. And, like many developers who build those apps, Niantic keeps that information.” Whereas other location-based apps might collect similar data, Bernstein concluded that “Pokémon Go's incredibly granular, block-by-block map data, combined with its surging popularity, may soon make it one of, if not the most, detailed location-based social graphs ever compiled.”

The genius of Pokémon Go was to transform the game you see into a higher-order game of surveillance capitalism, a game about a game. The players who took the city as their board, roaming its parks and pizzerias, unwittingly constituted a wholly different kind of human game board for this second and more consequential game. The players in this other real game could not be found in the clot of enthusiasts waving their phones at the edge of David's lawn. In the real game, prediction products take the form of protocols that impose forms of telestimulation intended to prod and herd people across real-world terrains to spend their real-world money in the real-world commercial establishments of Niantic's flesh-and-blood behavioral futures markets.

In the end we recognize that the probe was designed to explore the next frontier: the means of behavioral modification. The game about the game is, in fact, an experimental facsimile of surveillance capitalism's design for our future. It follows the prediction imperative to its logical conclusion, in which data about us in scale and scope combine with actuation mechanisms that align our behavior with a new market cosmos. All the flows of surplus from all the spaces, all the things, all the bodies, all the laughter, and all the tears are finally aimed at this triumph of certain outcomes and the revenue that it can unleash.

The new global means of behavioral modification that we see under construction at Facebook and Niantic represent a new regressive age of autonomous capital and heteronomous individuals, when the very possibilities of democratic flourishing and human fulfillment depend upon the reverse.

What has been forgotten here is that the Common Rule was the product of a similar challenge to principles of individual autonomy and democratic fidelity. It was one result of a deeply contested struggle in which democratically minded public officials joined with social activists, scholars, and litigators to resist the design, development, and deployment of behavioral modification as a mode of governmental power. It was not long ago that US society mobilized to resist, regulate, and control the means of behavioral modification, and it is this history that we can now draw upon to rediscover our bearings and rouse our awareness.

The Senate investigation was triggered by a growing sense of public alarm at the spread of psychological techniques for behavior control. There were many points of origin, but most salient was the influence of the cold war and the range of psychological techniques and programs for behavior modification that it had bred. The Korean War had publicized communist "brainwashing" techniques that, according to then newly appointed CIA Director Allen Dulles, had reduced US prisoners of war to a state of robotic passivity, in which the victim's brain "becomes a phonograph playing a disc put on its spindle by an outside genius over which it has no control." America's enemies appeared to be on the verge of mastering the art and science of "mind control" with psychological and pharmacological methods unknown to the US military. There were reports of Chinese and Soviet achievements in the remote alteration of a subject's mental capacities and the elimination of his "free will." Dulles committed the agency to rapid research in and development of "mind control" capabilities, from "de-patterning" and "rewiring" an individual to shaping an entire country's attitudes and actions.

Thus began a morbidly fascinating and often bizarre chapter in the history of American spy craft. Much of the new work was conducted in the context of the CIA’s highly classified MKUltra project, which was tasked with “research and development of chemical, biological, and radiological materials capable for employment in clandestine operations to control human behavior.” According to testimony in the 1975 Senate investigation of covert CIA Foreign and Military Intelligence operations, a 1963 Inspector General’s report on MKUltra noted several reasons for the program’s secrecy, but chief among them was the fact that behavior modification was seen as illegitimate. “Research in the manipulation of human behavior is considered by many authorities in medicine and related fields to be professionally unethical, therefore the reputation of professional participants in the MKUltra program are on occasion in jeopardy,” the report began. It also noted that many of the program’s activities were illegal, violated the rights and interests of US citizens, and would alienate public opinion.

Their research concluded that “mind control” was better understood as a complex system of conditioning based on unpredictable schedules of reinforcement, consistent with B. F. Skinner’s important discoveries on operant conditioning. According to Harvard historian Rebecca Lemov, the “mind control” researchers had a powerful effect on the CIA and other branches of the military. The notion that “human material was changeable”—that one’s personality, identity, awareness, and capacity for self-determining behavior could be crushed, eliminated, and replaced by external control—incited a new sense of panic and vulnerability: “If indeed the world was rife with threats to the inner as much as the outer man, then experts in these realms were needed more than ever. Many good and well-meaning professors—self-described or de facto human engineers—participated in the CIA’s programs to bring about slow or rapid change in the minds and behavior of people.”

Another factor was the 1971 publication of B. F. Skinner’s incendiary social meditation Beyond Freedom & Dignity. Skinner prescribed a future based on behavioral control, rejecting the very idea of freedom (as well as every tenet of a liberal society) and cast the notion of human dignity as an accident of self-serving narcissism. Skinner imagined a pervasive “technology of behavior” that would one day enable the application of behavior-modification methods across entire human populations.

The ensuing storm of controversy made Beyond Freedom & Dignity an international best seller. “Skinner’s science of human behavior, being quite vacuous, is as congenial to the libertarian as to the fascist,” wrote Noam Chomsky in a widely read review of the book. “It would be not absurd but grotesque to argue that since circumstances can be arranged under which behavior is quite predictable—as in a prison, for example, or… concentration camp.… therefore there need be no concern for the freedom and dignity of ‘autonomous man.’” (In the mid-1970s graduate department at Harvard where I studied and Skinner professed, many students referred to the book as Toward Slavery and Humiliation.)

From the first lines of the preface of the subcommittee's 1974 report, authored by Senator Ervin, it should be evident to any twenty-first-century captive of surveillance capitalism that US society has undergone a social discontinuity more profound than the mere passage of decades suggests. It is worth reading Ervin's own words, to grasp the passion with which he located the subcommittee's work at the heart of the Enlightenment project, pledging to defend the liberal ideals of freedom and dignity: When the founding fathers established our constitutional system of government, they based it on their fundamental belief in the sanctity of the individual...

The report’s critique of behavior modification has unique relevance for our time. It begins by asking a question that we must also ask: “How did they get away with it?” Their answer invokes the “exceptionalism” of that era. Just as surveillance capitalism was initially able to root and flourish under the protection of a so-called “war against terror” and the compulsion for certainty that it stirred, in the middle of the twentieth century the means of behavioral modification migrated from the lab to the world at large primarily under the cover of cold-war anxieties. Later, the behavior-change professionals of the 1960s and 1970s were summoned into civilian practice by a society turned fearful after years of urban riots, political protests, and rising levels of crime and “delinquency.” The senators reasoned that calls for “law and order” had motivated the search for “immediate and efficient means to control violence and other forms of anti-social behavior. The interest in controlling violence replaced more time-consuming attempts to understand its sources.”

With so many behavior-modification programs aimed at involuntary populations in state prisons and mental institutions, the senators recognized that the means of behavioral modification had to be reckoned with as a form of state power and questioned the government’s constitutional right to “control” citizens’ behavior and mentation.

The senators reserved their most vivid rebukes for what they considered as the two most extreme and pernicious behavior-modification techniques. The first was psychosurgery. The second was “electrophysiology,” defined as “the use of mechanical devices to control various aspects of human behavior.” The report notes with special horror the example of “devices” designed to be worn by a subject “constantly to monitor and control his behavior through a computer” and to “prevent a suspected behavior from occurring.”

The First Amendment, the subcommittee argued, “must equally protect the individual’s right to generate ideas,” and the right to privacy should protect citizens from intrusions into their thoughts, behavior, personality, and identity lest these concepts “become meaningless.” It was in this context that Skinnerian behavioral engineering was singled out for critical examination: “A major segment of the emerging behavior control technology is concerned with conditioning, through which various forms of persuasion are used to stimulate certain types of behaviors while suppressing others.”

In anticipation of future gamification techniques as means of behavioral modification, the subcommittee report also noted with apprehension more “benign” approaches that relied on “positive reinforcement,” from “gold-star incentives” to elaborate reward systems, in order “to restructure personality through artificially applied techniques.” … The subcommittee’s work had enduring consequences. Not only did prisoners’ and patients’ rights groups gain momentum in their efforts to end the behavioral oppression suffered in public institutions, but psychologists also began to discuss the need to professionalize their discipline with clear ethical standards, accreditation procedures, training programs, and career ladders. The National Research Act, passed in 1974, stipulated the creation of institutional review boards and laid the foundation for the evolution and institutionalization of the Common Rule for the ethical treatment of human subjects, from which Facebook famously held itself exempt.

According to Skinner’s biographer, “the battle had climaxed.” The public had made Beyond Freedom & Dignity a bestseller “but had just as surely rejected Skinner’s argument that there were cultural matters more important than preserving and extending individual freedom.”

A Federal Bureau of Prisons official recommended that program leaders avoid using “the term ‘behavioral modification’ but to talk about positive reward and reinforcements for the type of behavior we are attempting to instill.” Another said, “We’re doing what we always did… but to call it ‘behavior modification’ just makes things more difficult.” Skinner’s 1976 “primer,” titled About Behaviorism, motivated by what he believed were public misconceptions stirred by the wave of harsh reaction to Beyond Freedom & Dignity, failed to capture much public attention.

Most fascinating is that throughout these years of anxiety and debate, it was impossible to imagine the means of behavioral modification as anything other than owned and operated by the government: a privileged modality of state power. A 1966 Harvard Law Review article addressed issues of electronic tracking, surveillance, and behavioral control, reasoning that it would “consider governmental attempts to change conduct, since these seem more likely than private attempts.” The democratic impulse of US society, repelled by the excesses of its intelligence agencies, their support of criminal activities undertaken by the Nixon administration, and the migration of behavior modification as a means of disciplinary control in state institutions, led to the rejection of behavioral modification as an extension of governmental power.

Unknown to the senators, scholars, rights activists, litigators, and many other citizens who stood against the antidemocratic incursions of the behavioral engineering vision, these methods had not died. The project would resurface in a wholly unexpected incarnation as a creature of the market, its unprecedented digital capabilities, scale, and scope now flourishing under the flag of surveillance capitalism. During the same years that US democratic forces combined to resist behavior modification as a form of state power, the energies of capitalist counterinsurgency were already at work within society. The corporation was to enjoy the rights of personhood but be free of democratic obligations, legal constraints, moral calculations, and social considerations. Certainly in the US case, a weakened state in which elected officials depend upon corporate wealth in every election cycle has shown little appetite to contest behavior modification as a market project, let alone to stand for the moral imperatives of the autonomous individual.

In its latest incarnation, behavioral modification comes to life as a global digital market architecture unfettered by geography, independent of constitutional constraints, and formally indifferent to the risks it poses to freedom, dignity, or the sustenance of the liberal order that Ervin’s subcommittee was determined to defend. This contrast is even more distressing in light of the fact that in the mid-twentieth century, the means of behavior modification were aimed at individuals and groups who were construed as “them”: military enemies, prisoners, and other captives of walled disciplinary regimes.

Although it is obviously the case that we are excluded because the knowledge is not for us, these chapters have revealed a deeper structural basis for our exclusion. Now we know that surveillance capitalists' ability to evade our awareness is an essential condition for knowledge production. We are excluded because we are friction that impedes the elaboration of the shadow text and with it surveillance capitalism's knowledge dominance.

As to who decides, this division of learning has been decided by the declarations and incursions of the owners of private surveillance capital as another essential condition of accumulation, enabled by the reluctance of the state to assert democratic oversight in this secret realm. Finally, there is the question of who decides who decides. So far, it is the asymmetrical power of surveillance capital unencumbered by law that decides who decides.

The commodification of behavior under the conditions of surveillance capitalism pivots us toward a societal future in which an exclusive division of learning is protected by secrecy, indecipherability, and expertise. Even when knowledge derived from your behavior is fed back to you in the first text as a quid pro quo for participation, the parallel secret operations of the shadow text capture surplus for crafting into prediction products destined for other marketplaces that are about you rather than for you. These markets do not depend upon you except first as a source of raw material from which surplus is derived, and then as a target for guaranteed outcomes. We have no formal control because we are not essential to the market action. In this future we are exiles from our own behavior, denied access to or control over knowledge derived from our experience. Knowledge, authority, and power rest with surveillance capital, for which we are merely "human natural resources."

I made a promise to complete this work. This promise is my flag planted in the future tense. It represents my commitment to construct a future that cannot come into being should I abandon my promise. This future will not exist without my capacity first to imagine its facts and then to will them into being. I am an inchworm moving with determination and purpose across the distance between now and later. Each tiny increment of territory that I traverse is annexed to the known world, as my effort transforms uncertainty into fact. … My promise, though, is an anchor that girds me against the vagaries of my moods and temptations. It is the product of my will to will and a compass that steers my course toward a desired future that is not yet real. Events may originate in energy sources outside my will and abruptly alter my course in ways that I can neither predict nor control. Indeed, they have already done so. Despite this certain knowledge of uncertainty, I have no doubt that I am free. I can promise to create a future, and I can keep my promise. If the book that I have imagined is to exist in the future, it must be because I will to will it so. I live in an expansive landscape that already includes a future that only I can imagine and intend. In my world, this book I write already exists. In fulfilling my promise, I make it manifest. This act of will is my claim to the future tense.

The freedom of will is the existential bone structure that carries the moral flesh of every promise, and my insistence on its integrity is not an indulgence in nostalgia or a random privileging of the pre-digital human story as somehow more truly human. This is the only kind of freedom that we can guarantee ourselves, no matter the weight of entropy or inertia, and irrespective of the forces and fears that attempt to collapse time into an eternity of shadowboxing now, and now, and now. These bones are the necessary condition for the possibility of civilization as a “moral milieu” that favors the dignity of the individual and respects the distinctly human capacities for dialogue and problem solving. Any person, idea, or practice that breaks these bones and tears this flesh robs us of a self-authored and we-authored future.

I suggest that we now face the moment in history when the elemental right to the future tense is endangered by a panvasive digital architecture of behavior modification owned and operated by surveillance capital, necessitated by its economic imperatives, and driven by its laws of motion, all for the sake of its guaranteed outcomes.

Most simply put, there is no freedom without uncertainty; it is the medium in which human will is expressed in promises. Of course, we do not only make promises to ourselves; we also make promises to one another. When we join our wills and our promises, we create the possibility of collective action toward a shared future, linked in determination to make our vision real in the world. This is the origin of the institution we call “contract,” beginning with the ancient Romans.

Recall Varian's assertion that if someone stops making monthly car payments, "Nowadays it's a lot easier just to instruct the vehicular monitoring system not to allow the car to be started and to signal the location where it can be picked up." Varian calls this new capability a "new contract form," when in reality the uncontract abandons the human world of legally binding promises and substitutes instead the positivist calculations of automated machine processes. Without so much as a tip of the cap and a "fare-thee-well," Varian's uncontract disposes of several millennia of societal evolution during which Western civilization institutionalized the contract as a grand achievement of shared will.

Like builders, people in contractual agreements undertake this kind of collaboration. It’s not simply finding the way through a maze to an already agreed-upon end point, but rather the continuous refinement and clarification of ends and means in the face of unanticipated obstacles. This sociality of contract may entail conflict, frustration, oppression, or anger, but it can also produce trust, cooperation, cohesion, and adaptation as the means through which human beings confront an unknowable future.

Were “contract utopia” to exist, Williamson says, it would best be described as a “plan” that, like other “utopian modes,” requires “deep commitment to collective purposes” and “personal subordination.” Subordination to what? To the plan. Contract in this context of perfect rationality is what Williamson describes as “a world of planning.” Such planning was the basic institution of socialist economics, where the “new man” was idealized as possessing “a high level of cognitive competence” and therefore, it was espoused, could design highly effective plans. Varian deftly swaps out socialism’s “new man” and installs instead a market defined by surveillance capitalism’s economic imperatives, expressed through a ubiquitous computational architecture, the machine intelligence capabilities to which data are continuously supplied, the analytics that discern patterns, and the algorithms that convert them into rules. This is the essence of the uncontract, which transforms the human, legal, and economic risks of contracts into plans constructed, monitored, and maintained by private firms for the sake of guaranteed outcomes: less contract utopia than uncontract dystopia.

Most interesting is that when a local paper picked up the story, it quickly went viral across the web and traditional media. Millions of people read and responded to this drama, presumably because it stirred memories of something precious and necessary but now threatened with extinction. Jim Ford reminded us of the most cherished requirements of a civilized life: our shared assertion of rights to the future tense and its expression in the joining of wills in mutual commitment to dialogue, problem solving, and empathy. He was eloquent on this point: “Just be nice to people. It’s not that hard. The fact that this has gone so crazy is kind of sad. This should be a daily thing, a normal thing.”

In the dystopia of the uncontract, surveillance capitalism’s drive toward certainty fills the space once occupied by all the human work of building and replenishing social trust, which is now reinterpreted as unnecessary friction in the march toward guaranteed outcomes. The deletion of uncertainty is celebrated as a victory over human nature: our cunning and our opportunism. All that’s left to matter are the rules that translate reasons into action, the objective measures of behavior, and the degree of conformance between the two. Social trust eventually withers, a kind of vestigial oddity like a third nipple or wisdom teeth: traces of an evolutionary past that no longer appear in operational form because their context and therefore their purpose have vanished.

The uncontract and the for-profit circuits of behavior modification in which it executes its objectives construe society as an acrid wasteland in which mistrust is taken for granted. By positing our lives together as already failed, it justifies coercive intervention for the sake of certainty. Against this background of the gradual normalization of the automated plan and its planners, the human response of one repo man bears simple witness to precisely what surveillance capitalism must extinguish.

Human replenishment from the failures and triumphs of choosing the future in the face of uncertainty gives way to the blankness of perpetual compliance. The word trust lingers, but its referent in human experience dissolves into reminiscence, an archaic footnote to a barely remembered dream of a dream that has long since faded for the sake of a new dictatorship of market reasons. As the dream dies, so too does our sense of astonishment and protest. We grow numb, and our numbness paves the way for more compliance. A pathological division of learning forged by unprecedented asymmetries of knowledge and power fixes us in a new inequality marked by the tuners and the tuned, the herders and herded, the raw material and its miners, the experimenters and their unwitting subjects, those who will the future and those who are shunted toward others' guaranteed outcomes.

So let us establish our bearings. Uncertainty is not chaos but rather the necessary habitat of the present tense. We choose the fallibility of shared promises and problem solving over the certain tyranny imposed by a dominant power or plan because this is the price we pay for the freedom to will, which founds our right to the future tense. In the absence of this freedom, the future collapses into an infinite present of mere behavior, in which there can be no subjects and no projects: only objects.

In the future that the surveillance capitalism prepares for us, my will and yours threaten the flow of surveillance revenues. Its aim is not to destroy us but simply to author us and to profit from that authorship. Such means have been imagined in the past, but only now are they feasible. Such means have been rejected in the past, but only now have they been allowed to root. We are ensnared without awareness, shorn of meaningful alternatives for withdrawal, resistance, or protection.

The promise of the promise and the will to will run deeper than these deformities. They remind us of that place again where we humans heal the breach between the known and the unknowable, navigating the seas of uncertainty in our vessels of shared promises. In the real world of human endeavor, there is no perfect information and no perfect rationality. Life inclines us to take action and to make commitments even when the future is unknown. Anyone who has brought a child into the world or has otherwise given his or her heart in love knows this to be true.

In the course of the last ten chapters I have argued that surveillance capitalism represents an unprecedented logic of accumulation defined by new economic imperatives whose mechanisms and effects cannot be grasped with existing models and assumptions. This is not to say that the old imperatives—a compulsion toward profit maximization along with the intensification of the means of production, growth, and competition—have vanished. However, these must now operate through the novel aims and mechanisms of surveillance capitalism.

Surveillance capitalism’s new story begins with behavioral surplus discovered more or less ready-made in the online environment, when it was realized that the “data exhaust” clogging Google’s servers could be combined with its powerful analytic capabilities to produce predictions of user behavior. Those prediction products became the basis for a preternaturally lucrative sales process that ignited new markets in future behavior.

This dynamic established the extraction imperative, which expresses the necessity of economies of scale in surplus accumulation and depends upon automated systems that relentlessly track, hunt, and induce more behavioral surplus. Google imposed the logic of conquest, defining human experience as free for the taking, available to be rendered as data and claimed as surveillance assets.

In this new phase of competitive intensity, surveillance capitalists are forced from the virtual world into the real one. This migration necessitates new machine processes for the rendition of all aspects of human experience into behavioral data. Competition now occurs in the context of a rapidly evolving global architecture of ubiquitous computation and therefore ubiquitous supply opportunities, as prediction products are increasingly expected to approximate certainty and therefore to guarantee behavioral outcomes.

In a third phase of competitive intensity, surveillance capitalists discovered the necessity of economies of action based on new methods that go beyond tracking, capturing, analyzing, and predicting behavior in order to intervene in the state of play and actively shape behavior at the source. The result is that the means of production are subordinated to an elaborate new means of behavioral modification, which relies upon a variety of machine processes, techniques, and tactics (tuning, herding, conditioning) to shape individual, group, and population behavior in ways that continuously improve their approximation to guaranteed outcomes. Just as industrial capitalism was driven to the continuous intensification of the means of production, so surveillance capitalists are now locked in a cycle of continuous intensification of the means of behavioral modification.

Surveillance capitalists’ interests have shifted from using automated machine processes to know about your behavior to using machine processes to shape your behavior according to their interests. In other words, this decade-and-a-half trajectory has taken us from automating information flows about you to automating you.

In forty-six of the most prominent forty-eight surveys administered between 2008 and 2017, substantial majorities support measures for enhanced privacy and user control over personal data. (Only two early surveys were somewhat less conclusive, because so many participants indicated that they did not understand how or what personal information was being gathered.) Indeed, by 2008 it was well established that the more knowledge one has about “internet privacy practices,” the more one is likely to be very concerned about privacy.

Surveillance capitalist firms have tended to dismiss these survey results, pointing instead to the spectacular growth of users and revenue. This discrepancy has confounded research and public policy. With so many people rejecting the practices of surveillance capitalism, even considering how little most of us actually know about these practices, how is it that this market form has been able to succeed? The reasons are plentiful: 1.

2. Declaration as invasion: The lack of precedence left us disarmed and charmed. Meanwhile, Google learned the art of invasion by declaration, taking what it wanted and calling it theirs. The corporation asserted its rights to bypass our awareness, to take our experience and transform it into data, to claim ownership of and decisions over the uses of those data, to produce strategies and tactics that keep us ignorant of its practices, and to insist on the conditions of lawlessness required for these operations. These declarations institutionalized surveillance capitalism as a market form.

3. Historical context: Surveillance capitalism found shelter in the neoliberal zeitgeist that equated government regulation of business with tyranny. This “paranoid style” favored self-management regimes that imposed few limits on corporate practices. In a parallel development, the “war on terror” shifted the government’s attention from privacy legislation to an urgent interest in the rapidly developing skills and technologies of Google and other rising surveillance capitalists. These “elective affinities” produced a trend toward surveillance exceptionalism, which further sheltered the new market form from scrutiny and nurtured its development.

5. The dispossession cycle: First at Google and later at Facebook and other firms, surveillance capitalist leaders mastered the rhythms and stages of dispossession. Audacious incursions are pursued until resistance is met, followed by a range of tactics from elaborate public relations gambits to legal combat, all designed to buy time for gradual habituation to once-outrageous facts. A third stage features public demonstrations of adaptability and even retreat, while in the final stage resources are redirected to achieve the same objectives camouflaged by new rhetoric and tactics.

6. Dependency: The free services of Google, Facebook, and others appealed to the latent needs of second-modernity individuals seeking resources for effective life in an increasingly hostile institutional environment. Once bitten, the apple was irresistible. As surveillance capitalism spread across the internet, the means of social participation become coextensive with the means of behavioral modification.

7. Self-Interest: New markets in future behavior give rise to networks of fellow travelers, partners, collaborators, and customers whose revenues depend on the prediction imperative. Institutional facts proliferate. The pizzeria owner on the Pokémon Go map, the merchant who saturates his shop with beacons, and the insurance companies vying for behavioral data unite in the race toward guaranteed outcomes and surveillance revenues.

8. Inclusion: Many people feel that if you are not on Facebook, you do not exist. People all over the world raced to participate in Pokémon Go. With so much energy, success, and capital flowing into the surveillance capitalist domain, standing outside of it, let alone against it, can feel like a lonely and risky prospect.

10. Authority: Many also regard these corporations and their leaders as authorities on the future: geniuses who can see farther than the rest of us. It is easy to fall prey to the naturalistic fallacy, which suggests that because the companies are successful, they must also be right.

11. Social persuasion: As we have seen repeatedly, there is an endless cascade of beguiling rhetoric aimed at persuading people of the wonders associated with surveillance capitalist innovations: targeted advertising, personalization, and digital assistants. Beyond that, economies of action are intentionally designed to persuade us to follow one another along prescribed courses of action.

Surveillance capitalists dominate an abnormal division of learning in which they know things that we cannot know while compelled to conceal their intentions and practices in secret backstage action. It is impossible to understand something that has been crafted in secrecy and designed as fundamentally illegible. These systems are intended to ensnare us, preying on our vulnerabilities bred by an asymmetrical division of learning and amplified by our scarcity of time, resources, and support.

Surveillance capitalism rose from invention to domination in record time. This reflects its ability to attract capital and its laws of motion, but it also reflects a specific strategy in which velocity is consciously deployed to paralyze awareness and freeze resistance while distracting us with immediate gratifications. Surveillance capitalism's velocities outrun democracy even as they outrun our ability to understand what is happening and consider the consequences. This strategy is borrowed from a long legacy of political and military approaches to the production of speed as a form of violence, most recently known as "shock and awe."

The withdrawal of agreement takes two broad forms, a distinction that will be useful as we move into Part III. The first is what I call the counter-declaration. These are defensive measures such as encryption and other privacy tools, or arguments for “data ownership.” Such measures may be effective in discrete situations, but they leave the opposing facts intact, acknowledging their persistence and thus paradoxically contributing to their legitimacy. For example, if I “opt out” of tracking, I opt out for me, but my action does not challenge or alter the offending practice. The second form of disagreement is what I call the synthetic declaration. If the declaration is “check,” the counter-declaration is “checkmate,” and the synthetic declaration changes the game. It asserts an alternative framework that transforms the opposing facts. We bide our time with counter-declarations and make life more tolerable, but only a synthetic alternative vision will transform raw surveillance capitalism in favor of a digital future that we can call home.

I turn to the history of the Berlin Wall as an illustration of these two forms of disagreement. From 1961 through the early 1980s, courageous East Berliners carved seventy-one tunnels through the sandy soil beneath the city, affording several hundred people a means of escape to West Berlin. The tunnels are testament to the necessity of counter-declarations, but they did not bring down the wall or the power that sustained it. … The synthetic declaration gathered force over decades, but its full expression would have to wait until near midnight on November 9, 1989, when Harald Jäger, the senior officer on duty that night at the Bornholmer Street passage, gave the order to open the gates, and twenty thousand people surged across the wall into West Berlin. As one historian describes that event, “By the night of November 9, when the people appeared at the Berlin Wall and demanded to know of the border officials, Will you let us pass?, those people had become so certain of themselves, and the officials so unsure of themselves, that the answer was We will.”

Industrial capitalism followed its own logic of shock and awe, taking aim at nature to conquer “it” in the interests of capital; now surveillance capitalism has human nature in its sights. We have only gradually come to understand that the specific methods of domination employed by industrial capitalism for more than two centuries have fundamentally disoriented the conditions that support life on Earth, violating the most basic precepts of civilization. Despite the many benefits and immense accomplishments of industrial capitalism, it has left us perilously close to repeating the fate of the Easter Islanders, who wrecked the ground that gave them life, then fashioned statues to scan the horizon for the aid and succor that would never come. If industrial capitalism dangerously disrupted nature, what havoc might surveillance capitalism wreak on human nature?

If we are to rediscover our sense of astonishment, then let it be here: if industrial civilization flourished at the expense of nature and now threatens to cost us the Earth, an information civilization shaped by surveillance capitalism will thrive at the expense of human nature and threatens to cost us our humanity. Polanyi’s prophecy requires us to ask if we may yet avert this fate with our own synthetic declarations.

However, it would be wrong to suppose that surveillance capitalism can be grasped solely through the prism of its economic action or that the challenges we face are restricted to discerning, containing, and transforming its foundational mechanisms. The consequences of this new logic of accumulation have already leaked and continue to leak beyond commercial practices into the fabric of our social relations, transforming our relationships to ourselves and to one another. These transformations provide the soil in which surveillance capitalism has flourished: an invasive species that creates its own food supply. In transforming us, it produces nourishment for its own march forward.

This was the crucible of mass society, its hierarchical authority, and its centralized bureaucratic forms of public and private power, all of it haunted by the specters of conformity, obedience, and human standardization. Lives were defined by institutions that mirrored industrial organization: schools, hospitals, and even aspects of family and domestic life, in which ages and stages were understood as functions of the industrial system, from training to retirement.

Under surveillance capitalism, the “means of production” serves the “means of behavioral modification.” Machine processes replace human relationships so that certainty can replace trust. This new assembly relies upon a vast digital apparatus, world-historic concentrations of advanced computational knowledge and skill, and immense wealth. The arc of behavioral modification at scale integrates the many operations that we have examined: ubiquitous extraction and rendition, actuation (tuning, herding, conditioning), behavioral surplus supply chains, machine-intelligence–based manufacturing processes, fabrication of prediction products, dynamic behavioral futures markets, and “targeting,” which leads to fresh rounds of tuning, herding, conditioning, and the coercions of the uncontract, thus renewing the cycle.

As to this species of power, I name it instrumentarianism, defined as the instrumentation and instrumentalization of behavior for the purposes of modification, prediction, monetization, and control. In this formulation, “instrumentation” refers to the puppet: the ubiquitous connected material architecture of sensate computation that renders, interprets, and actuates human experience. “Instrumentalization” denotes the social relations that orient the puppet masters to human experience as surveillance capital wields the machines to transform us into means to others’ market ends.

When scholars, civil society leaders, journalists, public figures, and, indeed, most of us speak out courageously against this new power, invariably we look to Orwell’s Big Brother and more generally the specter of totalitarianism as the lens through which to interpret today’s threats. Google, Facebook, and the larger field of commercial surveillance are frequently depicted as “digital totalitarianism.” I admire those who have stood against the incursions of commercial surveillance, but I also suggest that the equation of instrumentarian power with totalitarianism impedes our understanding as well as our ability to resist, neutralize, and ultimately vanquish its potency. There is no historical precedent for instrumentarianism, but there is vivid precedent for this kind of encounter with an unprecedented new species of power.

Surveillance capitalists have no interest in murder or the reformation of our souls. Although their aims are in many ways just as ambitious as those of totalitarian leaders, they are also utterly distinct. The work of naming a strange form of power unprecedented in the human experience must begin anew for the sake of effective resistance and the creative power to insist on a future of our own making.

Most important is the fact that, like instrumentarianism, totalitarian power was also unprecedented. It literally defied human comprehension. There is much that we can learn from the struggles and missteps of scholars, journalists, and citizens as they found themselves overwhelmed by a force that they could neither fathom nor resist.

That year, the refashioning of the soul as the hallmark of the totalitarian impulse was immortalized by Stalin on a glittering, champagne-soaked Moscow evening. The setting was an auspicious literary gathering hosted by a compliant Maxim Gorky in the sprawling mansion that Stalin had presented to the revered author upon his return to Russia from a self-imposed Italian exile. Stalin took the floor for a toast as the room fell silent. "Our tanks are worthless if the souls who must steer them are made of clay. This is why I say: The production of souls is more important than that of tanks.… Man is reshaped by life itself, and those of you here must assist in reshaping his soul. That is what is important, the production of human souls. And that is why I raise my glass to you, writers, to the engineers of the soul." The authors assembled around Stalin that evening raised their glasses to his toast, persuaded perhaps by memories of less adaptive colleagues already exiled or executed, including the 1929 torture and murder of artists and writers in the Solovetsky Islands' aptly named Church of the Beheading.

Nazism also shifted the doctrine in an important way, asserting the "movement," not the "state," as the spiritual center of German totalitarianism, a relationship summarized during Hitler's first years as chancellor in National Socialism's popular slogan "The movement gives orders to the state."

That totalitarianism was a new species of power had confounded its analysis from the start, as both its Russian and German variants swept through those societies, challenging the foundations of Western civilization. Although these totalitarian regimes began to take root years before World War II—first in Russia in 1929 with Stalin’s ascension to power and then in Germany in 1933 with Hitler’s installation as chancellor—they eluded systematic study until the end of the war. Analysis was impeded in part by the sheer mystery and perpetual movement of the whole enterprise: the secret plans executed by secret police, the silent complicities and hidden atrocities, the ceaseless transformation of who or what was up or down, the intentional torsion of facts into anti-facts accompanied by a perpetual deluge of propaganda, misinformation, euphemism, and mendacity. The authoritative leader, or “egocrat,” to use the French philosopher Claude Lefort’s term, displaces the rule of law and “common” sense to become the quixotic judge of what is just or unjust, truth or lie, at each moment.

Despite the immediacy of catastrophic evil, Duranty’s article describes the Constitution of the USSR as one of the “the most democratic in the world… a foundation on which a future democracy may be built.” In addition to praiseworthy descriptions of the Red Army, free education and medical care, communal housing, and equality of the sexes, there is an upbeat commentary in which the “great purge” is breezily described as “one of the periodic cleansings of the Communist Party.” Duranty reports that this “cleansing” is “now over,” and people are “repairing the damage,” as if the country were tidying up after a particularly nasty winter storm. In fact, the Stalinist pattern of violence, imprisonment, exile, and execution merely shifted its focus with a swift and terrifying ferocity to the Baltic and eastern Poland.

The most startling feature of Duranty’s essay is its characterization of Stalin himself. There, sandwiched between a celebratory feature on a newly released film called The Wizard of Oz and a lengthy spread on embarrassing celebrity pictures, such as the famed ventriloquist’s dummy Charlie McCarthy with a cigarette in his wooden mouth, is a photo of a handsome smiling Joseph Stalin captioned “Stalin, chairman of the inner circle of the Communist Party… does not lay down the law as Lenin did. Stalin prefers to hear the views of his associates before he makes his own decision.” Stalin’s 1939 lionization in Look as an exemplar of participatory management was followed a few months later with his ascension to Time magazine’s cover as “Man of the Year.” Indeed, between 1930 and his death in 1953, Stalin made a total of ten appearances on the cover of Time.

One reason is that the Soviet government went to a great deal of trouble to present a false picture, including “model prisons” that betrayed no trace of the immense state machinery of torture and death. Another reason was the credulity of the observers themselves. In some cases, like Duranty’s, they were blinded by an ideological allegiance to the idea of a socialist state.

The most compelling reason of all is that in most of these cases, journalists, scholars, and Western governments had a difficult time reckoning the full weight of totalitarianism’s monstrous achievements because the actual facts were so “improbable” that it was difficult even for specialists to grasp their truth. “The Stalin epoch,” writes Conquest, “is replete with what appear as improbabilities to minds unfitted to deal with the phenomena.” This failure of comprehension has immediate significance for us as we learn to reckon with surveillance capitalism and its new instrumentarian power.

Nearly every intellectual who turned to this project in the period immediately following the war cites the feeling of astonishment at the suddenness with which, as Harvard political scientist Carl Friedrich put it, totalitarianism had “burst upon mankind… unexpected and unannounced.” Its manifestations were so novel and unanticipated, so shocking, rapid, and unparalleled, that all of it eluded language, challenging every tradition, norm, value, and legitimate form of action. The systematic accretion of violence and complicity that engulfed whole populations at extreme velocity invoked a kind of bewilderment that ended in paralysis, even for many of the greatest minds of the twentieth century.

Friedrich was among the first scholars of totalitarianism to address this experience of improbability, writing in 1954 that “virtually no one before 1914 anticipated the course of development which has overtaken Western civilization since then… none of the outstanding scholars in history, law, and the social sciences discerned what was ahead… which culminated in totalitarianism. To this failure to foresee corresponds a difficulty in comprehending.” Not even the most farsighted of the early-century interpreters of industrial society, thinkers such as Durkheim and Weber, had anticipated this murderous turn.

“Total loyalty—the psychological basis for domination—can be expected only from the completely isolated human being who, without any other social ties to family, friends, comrades, or even mere acquaintances, derives his sense of having a place in the world only from his belonging to a movement, his membership in the party.”

Totalitarian power cannot succeed by remote control. Mere conformity is insufficient. Each individual inner life must be claimed and transformed by the perpetual threat of punishment without crime. Mass murder warrants economies of scale—the camps, massacres, and gulags—but for the rest it would be a handmade terror that aims to remake every aspect of the individual from the inside out: heart, mind, sexuality, personality, and spirit. This craftwork requires the detailed orchestration of isolation, anxiety, fear, persuasion, fantasy, longing, inspiration, torture, dread, and surveillance. Arendt describes the relentless process of "atomization" and fusion in which terror destroys the ordinary human bonds of law, norms, trust, and affection, "which provide the living space for the freedom of the individual." The "iron band" of terror "mercilessly presses men… against each other so that the very space of free action… disappears." Terror "fabricates the oneness of all men."

Although it is not murderous, instrumentarianism is as startling, incomprehensible, and new to the human story as totalitarianism was to its witnesses and victims. Our encounter with unprecedented power helps to explain why it has been difficult to name and know this new species of coercion, shaped in secret, camouflaged by technology and technical complexity, and obfuscated by endearing rhetoric. Totalitarianism was a political project that converged with economics to overwhelm society. Instrumentarianism is a market project that converges with the digital to achieve its own unique brand of social domination.

It is not surprising, therefore, that instrumentarianism’s specific “viewpoint of observation” was forged in the controversial intellectual domain known as “radical behaviorism” and its antecedents in turn-of-the-century theoretical physics. In the remainder of this chapter, our examination of power in the time of surveillance capitalism pivots to this point of origin far from totalitarianism’s murder and mayhem. It takes us to laboratories and classrooms and the realms of thought spun by men who regarded freedom as a synonym for ignorance and human beings as distant organisms imprisoned in patterns of behavior beyond their own comprehension or control, such as ants, bees, or Stuart MacKay’s herds of elk.

In a 1971 cover story, Time magazine described Burrhus Frederic “B. F.” Skinner as “an institution at Harvard… the most influential of living American psychologists, and the most controversial contemporary figure in the science of human behavior, adored as a messiah and abhorred as a menace.”

His early research broke new ground as he engineered variable “schedules of reinforcement” to produce detailed patterns of activity that were foreign to an animal’s original behavioral repertoire, work that he called “operant conditioning.”

Skinner set out on a new path defined by the promise of behavioral engineering. In 1947 he told a symposium, “It is not a matter of bringing the world into the laboratory, but of extending the practices of an experimental science to the world at large. We can do this as soon as we wish to do it.” The missile project had cast the meaning of his experimental work “in a new light,” he reflected years later in his autobiography. “It was no longer merely an experimental analysis. It had given rise to a technology.”

Skinner was eager to apply his laboratory insights to the world's ills despite precious few grounds for his inferential leaps. As a public intellectual, he spent nearly seven decades trying to persuade the public that his radical behaviorism offered the principles of social organization necessary to defend civilization from cataclysm. He brashly extrapolated from the conduct of beleaguered animals to grand theories of social behavior and human evolution in books such as his 1948 "utopian" novel Walden Two and his 1971 social philosophy Beyond Freedom & Dignity.

It was Watson who famously announced in 1913 the behaviorist point of view: "Psychology as the behaviorist views it is a purely objective experimental branch of natural science. Its theoretical goal is the prediction and control of behavior. Introspection forms no essential part of its methods.… The behaviorist… recognizes… no dividing line between man and brute." But Watson turned out to be as much showman as scientist, and Skinner bitterly criticizes his extreme claims and "shortage of facts" that cast lasting doubt on radical behaviorism.

Meyer had pursued his doctoral studies at the University of Berlin, where his dissertation advisor, Max Planck, was destined to become one of the most celebrated physicists of all time. Planck insisted on the unity of the physical world and the discoverability of the natural laws that would disclose their secrets only through mathematical analysis, including the laws of human behavior. "The outside world is something independent from man, something absolute," Planck wrote. "The quest for the laws which apply to this absolute appeared… as the most sublime scientific pursuit in life." Meyer carried Planck's teaching into his quest for the principles that would finally elevate the study of human behavior to true scientific status.

The phrase that captured the new scientific perspective was the "Other-One." Human behavior would yield to scientific research only if psychologists learned to view humans as others. This "viewpoint of observation" was an absolute requirement for an "objective science of human behavior" that ceased to confuse inner experience with external action. Central to this new viewpoint was his notion of the human being as organism. The human being is recast as an "it," an "other," a "they" of organisms: an "organism among organisms," distinguishable from a lettuce, a moose, or an inchworm only in degree of complexity. A scientific psychology would restrict its interests to the social and therefore visible behaviors of this "organism as an organism." It would be "the study of the life of the Other-One—but of his life in so far as it is of social significance rather than as it is of significance for himself.… We are studying the Other-One in preference to Our-Selves."

The logical consequences of the new viewpoint necessitated a reinterpretation of the higher-order human experiences that we call "freedom" and "will." Meyer echoed Planck in positing that "freedom of action in the animal world signifies the same that is meant by accidents in the world of physics." Such accidents are simply phenomena for which there are insufficient information and understanding. And so it goes with freedom. The liberal idea of freedom persists in an inverse relationship to the growth of scientific knowledge, especially in the field of psychological science. Knowledge and freedom are necessarily adversaries. As Meyer wrote, "The Other-One's conduct is free, uncaused, only in the same sense in which the issue of a disease, the outcome of a war, the weather, the crops are free and uncaused; that is, in the sense of general human ignorance of the particular causes of the particular outcome."

Decades later, this worldview would define the core of the controversial social philosophy espoused in Beyond Freedom & Dignity, in which Skinner argues that knowledge does not make us free but rather releases us from the illusion of freedom. In reality, he writes, freedom and ignorance are synonyms. The acquisition of knowledge is heroic in that it rescues us from ignorance, but it is also tragic because it necessarily reveals the impossibility of freedom.

For Meyer and for Skinner, our attachments to notions such as freedom, will, autonomy, purpose, and agency are defense mechanisms that protect us from the uncomfortable facts of human ignorance. I think of Dickens's Scrooge when he first encounters the doleful, chain-dragging ghost of his deceased partner Jacob Marley and denies the apparition, saying, “You may be an undigested bit of beef, a blot of mustard, a crumb of cheese, a fragment of an underdone potato.” So it is with freedom: an undigested bit of fear, a crumb of denial that, once metabolized, will dispel the apparition and deliver us to reality. The environment determines behavior, and our ignorance of precisely how it does so is the void that we fill with the fantasy of freedom.

Meyer took great pains to insist that the significance of human inwardness—“soul,” “self,” “mind,” “consciousness”—is restricted to the subjective life of the individual. It can have no scientific value because it cannot be observed and measured: “We do not deny the soul, but we do not devote our time to it. We find enough, and more than enough, to do studying the body.” The soul is “your own business,” an intimate experience and irreducible mystery that are formally exempt from scientific inquiry: “Human societies can thus be understood as originating from natural laws, not in the sense of groups of souls tho, but in the sense of groups of organisms.”

In Meyer’s thinking, the shift in perspective from “the human being as a soul” to “the human being as an organism” explained “why the trend of history is in the direction of democracy.” As science overtakes civilization, Meyer assumed a new global recognition of equality and democratic fellowship rooted in the basic fact of our overwhelming similarity as fellow organisms. The divisions that haunt society, politics, and economics based on class, wealth, liberality, race, and so on would become ridiculous: “In real world-wide human life, the differences among individuals are entirely swamped by their likenesses. To him who accepts the scientific view that human society is a group of organisms, it is an absurd proposition to divide… into… classes.…”

Meyer believed that every social science that aspired to true scientific status would embrace the viewpoint of the Other-One—psychology, of course, but also sociology, economics, criminology, history, and the psychology of religion: “Christ going among his fellow men, an organism among organisms.…” Otherizing would pave the way to a rational future, with its bleak satisfactions that resign humanity to the forfeit of freedom as the price of knowledge.

From the opening page of Skinner’s first book, The Behavior of Organisms, published in 1938, he sounds Meyer’s (and Planck’s) caution: freedom is merely ignorance waiting to be conquered. “Primitive systems of behavior” assign causality to “entities beyond man.” Equally inadequate are the “advanced systems of behavior” that ascribe control to vague fictions such as the “self” or “free will.” “The inner organism,” he wrote, “may in resignation be called free, as in the case of ‘free will,’ when no further investigation is held to be possible.”

Skinner called his work “radical behaviorism,” insisting that the only meaningful object of behavioral study was observation of action devoid of subjective attributions. That’s what made it radical. “Behavior is what an organism is doing—or more accurately what it is observed by another organism to be doing,” he declared in his first book’s opening pages. “Operant” behavior was his term for this active, observable “doing.” The vocabulary for rendering descriptions of operant behavior was to be purged of inwardness: an organism cannot be said to “see” but rather to “look toward.” Only such objective descriptions can render measurable behavioral facts that, in turn, lead to patterns and ultimately to the documentation of causal relationships between environment and behavior.

When a man controls himself, chooses a course of action, thinks out the solution to a problem, or strives toward an increase in self-knowledge, he is behaving. He controls himself precisely as he would control the behavior of anyone else—through the manipulation of variables of which behavior is a function. His behavior in so doing is a proper object of analysis, and eventually it must be accounted for with variables lying outside the individual himself.

In nearly every book and article, Skinner declared the truth that Planck taught Meyer and Meyer imparted to his students, the single truth that can be grasped only through the viewpoint of the Other-One: freedom is ignorance. The felt experience of free will is but a bit of undigested denial, produced by a lack of information about the actual determinants of behavior. Like Meyer and Planck before him, Skinner regarded freedom as an “accident,” arguing that the very notion of “accident” is illusory, a snapshot in time that reveals a lacuna waiting to be filled and eventually transformed by advancing knowledge into an expression of a lawful, predictable pattern.

Richard Herrnstein, one of Skinner’s most-accomplished students, later his colleague in Harvard’s Psychology Department, and a luminary of radical behaviorism, once explained to me that any action regarded as an expression of free will is simply one for which “the vortex of stimuli” that produced it cannot yet be adequately specified. We merely lack the means of observation and calculation. I was a twenty-three-year-old student, and the term was new and startling to me. I never forgot that conversation, perhaps because it came as close as any to summarizing the behaviorist’s conception of God. Indeed, there was a time when if you took the elevator to Skinner’s lab on the seventh floor of the Psychology Department, the first thing you would see was a sign reading “God is a VI,” a variable interval of behavioral reinforcement plucked from the vortex.

In this view, “freedom” or “accidents” shrink as our developing capabilities in measurement and computation provide more information about the vortex of stimuli. Ignorance about human behavior is like a melting iceberg on a warming planet, destined to yield to the mounting heat as we invent the means and methods smart enough to first decipher and then control the vortex of stimuli that shapes each fact of human behavior.

The problems imposed by the complexity of a subject matter must be dealt with as they arise. Apparently hopeless cases often become manageable in time. It is only recently that any sort of lawful account of the weather has been possible.… Self-determination does not follow from complexity.… Difficulty in calculating the orbit of the fly does not prove capriciousness, though it may make it impossible to prove anything else.

Skinner would insist that “further investigation” is always possible. In the first pages of Beyond Freedom & Dignity he calls for a technological solution to ignorance: “We need to make vast changes in human behavior, and we cannot make them with the help of nothing more than physics or biology, no matter how hard we try… what we need is a technology of behavior… comparable in power and precision to physical and biological technology.…”

Skinner concluded that the literature of freedom and dignity “stands in the way of further human achievement.” He argued that the missing puzzle piece holding back the urgent development of the “instruments and methods” essential for a technology of behavior was the stubborn allegiance to these antique notions among people determined to preserve “due credit” for their actions. The belief in “autonomous man” is a regressive source of resistance to a rational future, an “alternative explanation of behavior” that obstructs the advancement of society.

The professor believed that humanity desperately needed a plan, and he imagined powerful new instruments that could engineer behavior in every domain. As early as 1953, he anticipated today’s digitally engineered casino environments, whose sophistication in the precise shaping of gamblers’ behavior has made them a testing ground for state security agencies and surveillance capitalists alike: “But with proper instrumentation it should be possible to improve upon established practices in all these fields. Thus gambling devices could be ‘improved’—from the point of view of the proprietor—by introducing devices which would pay off on a variable-interval basis, but only when the rate of play is exceptionally high.”

Skinner understood that the engineering of behavior risked violating individual sensibilities and social norms, especially concerns about privacy. In order to allay these anxieties he advised that observation must be unobtrusive, ideally remaining outside the awareness of the organism: “Behavior may also be observed with a minimum of interaction between subject and scientist, and this is the case with which one naturally tries to begin.” But there would be challenges. New technologies of behavior would have to continually push the envelope of the public-private divide in order to access all the data relevant to behavioral prediction and control. In this he anticipated today’s rendition frontier as new detection systems plumb the depths of personalities and emotions: “But we are still faced with events which occur at the private level and which are important to the organism without instrumental amplification. How the organism reacts to these events will remain an important question, even though the events may some day be made accessible to everyone.”

Such conflicts would be resolved, Skinner reckoned, by the gradual retreat of privacy norms as they fall to the advance of knowledge: “The line between public and private is not fixed.” Like today’s surveillance capitalists, he was confident that the slow drip of technological invention would eventually push privacy to the margins of human experience, where it would join “freedom” and other troublesome illusions. All these would be replaced by the viewpoint of the Other-One embodied in new instruments and methods: “The boundary shifts with every discovery of a technique for making private events public.… The problem of privacy may, therefore, eventually be solved by technical advances.”

Although privacy advocates and many other critics of surveillance capitalism are quick to appropriate Orwellian language in the search for meaning and metaphor that capture the sense of new menace, surveillance capital’s instrumentarian power is best understood as the precise antithesis of Orwell’s Big Brother. We now turn to this stark contrast, most vividly represented in the distinct conceptions of utopia that attach to each of these two species of power.

In the blood-soaked wake of World War II, both Skinner and the journalist and novelist George Orwell aimed curative, “utopian” novels at the mayhem of modernity’s diminished prospects produced by an incomprehensible scale of violence. Viewed from a distance, Skinner’s Walden Two, published in 1948, and Orwell’s 1984, released the following year, had much in common. Each elaborated a complete conception of a distinct logic of power, each imagined a society defined by the full flourishing of that power, and each was utopian from the point of view of the form of power that it described. However, their public reception could not have been more different: Walden Two was dismissed as a dystopian nightmare and ignored by the general public for more than a decade. Orwell’s 1984 was immediately canonized as a dystopian masterpiece and the distillation of the twentieth century’s worst nightmares.

The two utopias have often been confused with each other in their content and aims: Time magazine’s 1971 cover story on Skinner described Walden Two as raising “the specter of a 1984 Orwellian society that might really come to pass.” The great historian and literary critic Lewis Mumford once described Walden Two as a “totalitarian utopia” and a depiction of “hell,” but in fact these characterizations are a persistent and, as we shall see, dangerous confusion. Although both books have been described as depictions of totalitarianism, the forms of power that each describes are profoundly different. In most respects, they are precise opposites.

Big Brother's vigilance is not restricted to the grand continents of armies and statecraft or the observable flows of bodies and crowds. Big Brother is a panvasive consciousness that infects and possesses each individual soul, displacing all attachments once formed in romantic love and good fellowship. The essence of its operation is not simply that it knows every thought and feeling but rather the ruthless tenacity with which it aims to annihilate and replace unacceptable inward experience.

Ultimately, as every reader knows, Winston's stubborn soul is successfully "engineered." Orwell's chilling final passages fulfill the life of that dry seed planted at the turn of the century in Italy's impoverished soil and nourished by war, deprivation, and humiliation to flower in the nightmare of Nazi Germany and the apocalypse of Stalin's Russia, finally to bear fruit in Orwell's imagination: a testament for all time to what Mussolini had called the "ferocious totalitarian will" and the souls on which it feeds. Winston basks in serene elation, "his soul white as snow.… He had won the victory over himself. He loved Big Brother."

In contrast, Walden Two was not intended as a warning but rather as the antidote to totalitarianism and, more broadly, a practical recipe for the challenges of rebuilding Western societies after the war. Skinner understood his utopia as a methodological cure for the nightmare of crushed souls, a cure that, he insisted, was superior to any of the conventional political, economic, or spiritual remedies on offer. He scoffed at the notion that "democracy" held the solution because it is a political system that merely perpetrates the illusion of freedom while impeding the dominion of science. The promise of the "free market" as the curative for postwar society was an equally empty dream, he believed, because it rewards destructive competitiveness between people and classes. Skinner also rejected existentialism's new humanism, seeing it as a breeding ground for passivity, and he regarded religion as the worst cure of all, enshrining ignorance and crippling the advance of science.

Skinner's cure was different and unique: a utopia of technique that promised a future of social equality and dispassionate harmony founded on the viewpoint of the Other-One, the "organism among organisms," as the object of "behavioral engineering." It is the future of Meyer's dreams, in which Frazier, the founder-leader of the fictional Walden Two and Skinner's outspoken avatar, describes that ideal community as a "superorganism" that can be shaped and controlled "as smoothly and efficiently as champion football teams."

Skinner’s utopia was meant to illustrate the possibility of a successful social order that transcends the use of force and rejects the need to dominate human souls. The Walden Two community equally disdains the practices of democratic politics and representative government. Its laws are derived from a science of human behavior, specifically Skinner’s own radical behaviorism, founded on the physicist’s ideal of the Other-One. His utopia was a vehicle for other ambitions as well, intended to illustrate the behavioral solutions that are essential for improvement in every domain of modern life: the nuclear threat, pollution, population control, urban growth, economic equality, crime, education, health care, the development of the individual, effective leisure. It aimed to cultivate “the good life,” for which all the ideals of a liberal society—freedom, autonomy, privacy, a people’s right to self-rule—must be forfeit.

Frazier laments that people “have been kept in their places,” not just by external forces, “but much more subtly by a system of beliefs implanted within their skins. It’s sometimes an almost hopeless task to take the shackles off their souls, but it can be done.… You can’t in the long run enforce anything. We don’t use force! All we need is adequate behavioral engineering.”

These two utopias reflect two distinct species of power, and each novel was intent on rescuing the future from the twentieth-century nightmare of the soul. Orwell was able to draw on the recent past, but Skinner imagined a future that he would not live long enough to enjoy. If surveillance capitalism and its instrumentarian power continue to flourish, then it is we who may see the Walden Two vision realized, as freedom falls to others’ knowledge—but now in the service of others’ wealth and power.

Surveillance capitalism is the puppet master that imposes its will through the medium of the ubiquitous digital apparatus. I now name the apparatus Big Other: it is the sensate, computational, connected puppet that renders, monitors, computes, and modifies human behavior. Big Other combines these functions of knowing and doing to achieve a pervasive and unprecedented means of behavioral modification. Surveillance capitalism's economic logic is directed through Big Other's vast capabilities to produce instrumentarian power, replacing the engineering of souls with the engineering of behavior.

Instrumentarianism's radical indifference is operationalized in Big Other's dehumanized methods of evaluation that produce equivalence without equality. These methods reduce individuals to the lowest common denominator of sameness—an organism among organisms—despite all the vital ways in which we are not the same. From Big Other's point of view we are strictly Other-Ones: organisms that behave. Big Other encodes the viewpoint of the Other-One as a global presence. There is no brother here of any kind, big or little, evil or good; there are no family ties, however grim. There is no relationship between Big Other and its otherized objects, just as there was no relationship between B. F. Skinner's “scientists and subjects.”

Although Big Other can mimic intimacy through the tireless devotion of the One Voice—Amazon-Alexa's chirpy service, Google Assistant's reminders and endless information—do not mistake these soothing sounds for anything other than the exploitation of your needs. I think of elephants, that most majestic of all mammals: Big Other poaches our behavior for surplus and leaves behind all the meaning lodged in our bodies, our brains, and our beating hearts, not unlike the monstrous slaughter of elephants for ivory. Forget the cliché that if it's free, “You are the product.” You are not the product; you are the abandoned carcass. The “product” derives from the surplus that is ripped from your life.

Big Other finally enables the universal technology of behavior that, as Skinner, Stuart MacKay, Mark Weiser, and Joe Paradiso each insisted, accomplishes its aims quietly and persistently, using methods that intentionally bypass our awareness, disappearing into the background of all things.

Under the regime of instrumentarian power, the mental agency and self-possession of the right to the future tense are gradually submerged beneath a new kind of automaticity: a lived experience of stimulus-response-reinforcement aggregated as the comings and goings of mere organisms. Our conformity is irrelevant to instrumentarianism's success. There is no need for mass submission to social norms, no loss of self to the collective induced by terror and compulsion, no offers of acceptance and belonging as a reward for bending to the group. All of that is superseded by a digital order that thrives within things and bodies, transforming volition into reinforcement and action into conditioned response.

In this way instrumentarian power produces endlessly accruing knowledge for surveillance capitalists and endlessly diminishing freedom for us as it continuously renews surveillance capitalism’s domination of the division of learning in society. False consciousness is no longer produced by the hidden facts of class and their relation to production but rather by the hidden facts of instrumentarian power’s command over the division of learning in society as it usurps the rights to answer the essential questions: Who knows? Who decides? Who decides who decides? Power was once identified with the ownership of the means of production, but it is now identified with ownership of the means of behavioral modification that is Big Other.

Though confident that science would eventually overcome the practical challenges of a behavioral technology, Skinner was more troubled by the cultural impediments to a human science of behavioral prediction and control founded on the viewpoint of the Other-One. He resented the friction produced by human beings in their stubborn attachment to the values and ideals of freedom inherited from the eighteenth-century philosophers, and he equally despised the existential project of the postwar philosophies that planted authenticity, free will, and autonomous action at the heart of second-modernity yearning.

In his penultimate essay, written in 1990, barely three years before his death, Skinner mourned the prospects of behavioral prediction as the foundation of a new society built on scientific certainty: "To say that a person is simply a place in which something happens seems even more threatening when it raises questions about what we are likely to do rather than what we have done." In those last years he seemed more resigned to the tenacity of human friction and its stubborn allegiance to something like free will, his voice less outrageous and aggressive than the author of Beyond Freedom & Dignity, two decades earlier. Anger and contempt had shaded into weariness and despair with his final reflections: It may be said that this is a discouraging view of human behavior and that we shall be more likely to do something about the future if we continue to believe that our destiny is in our hands.

It may be said that this is a discouraging view of human behavior and that we shall be more likely to do something about the future if we continue to believe that our destiny is in our hands. That belief has survived for many centuries and has led to remarkable achievements, but the achievements were only the immediate consequences of what was done. We now know that other consequences have followed and that they are threatening our future. What we have done with our destiny may not be a testament we wish to leave to the world.

In our time of surveillance exceptionalism, as astonishment succumbs to helplessness and resignation, the resistance that Skinner lamented appears to be waning. The belief that we can choose our destiny is under siege, and, in a dramatic reversal, the dream of a technology of behavioral prediction and control—for which Skinner had endured such public scorn—is now a flourishing fact. This prize now attracts immense capital, human genius, scientific elaboration, governmental protection, ecosystems of institutionalization, and the glamour that always has and always will attach to power.

The rise of instrumentarianism follows the path of “illuminating events” that, as Hannah Arendt writes, reveal “a beginning in the past which had hitherto been hidden.” It is in the nature of instrumentarian power to operate remotely and move in stealth. It does not grow through terror, murder, the suspension of democratic institutions, massacre, or expulsion. Instead, it grows through declaration, self-authorization, rhetorical misdirection, euphemism, and the quiet, audacious backstage moves specifically crafted to elude awareness as it replaces individual freedom with others’ knowledge and replaces society with certainty. It does not confront democracy but rather erodes it from within, eating away at the human capabilities and self-understanding required to sustain a democratic life.

The narratives of Facebook’s experimental maneuvers, Pokémon Go’s prototype of a behavioral-futures-market–dominated society, and the endless examples of digital innovation crushed under the heel of the surveillance creed may be as close as we get to a public announcement of history-illuminating change that moves through us and among us, irreversibly altering life as we have known it. There is no violence here, only the steady displacement of the will to will that has been embodied in self-determination, expressed in the first-person voice, and nourished in the kind of sanctuary that depends upon the possibility of private life and the promise of public freedom.

Instrumentarian power, like Goethe’s Faust, is morally agnostic. The only moral imperative here is distilled from the point of view of a thin utopian gruel. If there is sin, it is the sin of autonomy: the audacity to reject the flows that herd us all toward predictability. Friction is the only evil. Obstruction in law, action, or rhetoric is simply reactionary. The norm is submission to the supposed iron laws of technological inevitability that brook no impediment. It is deemed only rational to surrender and rejoice in new conveniences and harmonies, to wrap ourselves in the first text and embrace a violent ignorance of its shadow.

This new power is the spawn of an unprecedented convergence: the surveillance and actuation capabilities of Big Other in combination with the discovery and monetization of behavioral surplus.

The trouble with modern theories of behaviorism is not that they are wrong but that they could become true, that they actually are the best possible conceptualization of certain obvious trends in modern society. It is quite conceivable that the modern age—which began with such an unprecedented and promising outburst of human activity—may end in the deadliest, most sterile passivity history has ever known.

What problems? I have argued that surveillance capitalism and its instrumentarian power feed on the volatile "conditions of existence" that I summarize as the "collision." Surveillance capitalism offers solutions to individuals in the form of social connection, access to information, time-saving convenience, and, too often, the illusion of support. These are the resources of the first text. More significantly, it offers solutions to institutions in the form of omniscience, control, and certainty. The idea here is not to heal instability—the corrosion of social trust and its broken bonds of reciprocity, dangerous extremes of inequality, regimes of exclusion—but rather to exploit the vulnerabilities produced by these conditions.

In its absence, the authority of shared values and mutual obligations slips away. The void that remains is a loud signal of societal vulnerability. Confusion, uncertainty, and distrust enable power to fill the social void. Indeed, they welcome it. … In the age of surveillance capitalism it is instrumentarian power that fills the void, substituting machines for social relations, which amounts to the substitution of certainty for society.

The private institutions of capital led the way in this ambitious reformation of collective life and individual experience, but they found necessary support from public institutions, especially as the declaration of a “war on terror” legitimated every inclination to enshrine machine-produced certainty as the ultimate solution to societal uncertainty. These mutual affinities assured that instrumentarian power would not be a stepchild but rather an equal partner or even, with increasing regularity, the lord and master upon whom the state depends in its quest for “total awareness.”

Although we have become desensitized to a seemingly endless train of such examples, it is important to recognize that in these entanglements, state and market institutions demonstrate a shared commitment to a relentless drive toward guaranteed outcomes. Their mutual affinities can help us define the problem that threatens to make instrumentarian power our century's curse.

Unsurprisingly, instrumentarian power is consistently called into action as a solution, if not the solution, to the threat of terrorism. Acts of terror reject the authority of civilizational norms and reveal the impossibility of society without mutual trust. Governments now turn to instrumentarian power as the solution to this new source of societal uncertainty, demanding the certainty machines that promise direct, reliable means of detection, prediction, and even the automatic actuation of countermeasures.

During the sixteen years of the Bush and Obama administrations, “progress in information technology” was understood as the “most effective response” to threat. Peter Swire observes that public officials “know that the private sector is developing many new techniques for collecting and processing data and making decisions based on that data.” The consequence is “a large and ongoing shift toward information-intensive strategies” that avail themselves of these market capabilities.

This status quo was interrupted in 2013, when Edward Snowden revealed the hidden complicity between state security agencies and the tech companies. In the backlash that followed, surveillance capitalists confronted new public relations challenges in what they portrayed as an uneasy or even unwilling alliance between private power and state security needs. Nevertheless, new terrorist threats invariably orient public officials toward the intensification and deployment of Big Other and the instrumentarian power it signifies. However, their ability to access this immense power is fraught with tension. It is not simply theirs to command. They must work, at least in part, through the surveillance capitalists.

A White House briefing memo encouraged the companies to develop a “radicalism algorithm” that would digest social media and other sources of surplus to produce something comparable to a credit score, but aimed at evaluating the “radicalness” of online content.

In 2017 surveillance capitalists, including Facebook, Microsoft, Google, and Twitter, established the Global Internet Forum to Counter Terrorism. The objective was to tighten the net of instrumentarian power through “collaboration on engineering solutions to the problem of sharing content classification techniques,” “counterspeech initiatives,” and a shared database of “unique digital fingerprints” for violent terrorist imagery to accelerate the identification of terrorist accounts.

Another, more prominent surveillance-as-a-service company, Palantir, once touted by Bloomberg Businessweek as “the war on terror’s secret weapon,” was found to be in a secret collaboration with the New Orleans Police Department to test its “predictive policing” technology. Palantir’s software not only identified gang members but also “traced people’s ties to other gang members, outlined criminal histories, analyzed social media, and predicted the likelihood that individuals would commit violence or become a victim.”

Writing in Foreign Policy, journalist Amy Hawkins explains that China's pandemic of social distrust is the problem to which the social credit system is addressed as the cure: "To be Chinese today is to live in a society of distrust, where every opportunity is a potential con and every act of generosity a risk of exploitation." A fascinating empirical study of social trust in contemporary China actually finds high levels of "in-group" trust but discovers that these are correlated with negative health outcomes. The conclusion is that many Chinese trust only the people who are well-known to them. All other relationships are regarded with suspicion and anxiety, with obvious consequences for social trust as well as well-being. This rampant distrust, typically assigned to the traumas of rapid modernization and the shift to a quasi-capitalist economy, is also the legacy of Chinese totalitarianism. The Chinese Communist Party dismantled traditional domains of affiliation, identity, and social meaning—family, religion, civil society, intellectual discourse, political freedom—recalling Arendt's description of the "atomization" that destroys bonds of trust. As Hawkins writes, "But rather than promoting the organic return of traditional morality to reduce the gulf of distrust, the Chinese government has preferred to invest its energy in technological fixes… and it's being welcomed by a public fed up of not knowing who to trust… in part because there's no alternative."

In 2017 the central bank retracted its support for the private-sector personal credit programs, perhaps because they were too successful, their concentrations of knowledge and power too great. Sesame Credit had acquired more than 400 million users in just two years, staking a claim to just about every aspect of those users' lives. A journalist who wrote a book on Ant Financial anticipates that the government is preparing to assert control over the whole system: "The government doesn't want this very important infrastructure of the people's credit in one big company's hands." The Chinese government appears to understand that power accrues to the owners of the means of behavioral modification. It is the owners who set the behavioral parameters that define guaranteed outcomes. Thus, fusion advances.

According to a report in China Daily, debtors on the list were automatically prevented from flying 6.15 million times since the blacklist was launched in 2013. Those in contempt of court were denied sales of high-speed train tickets 2.22 million times. Some 71,000 defaulters have missed out on executive positions at enterprises as a result of their debts. The Industrial and Commercial Bank of China said it had refused loans worth more than 6.97 billion yuan ($1.01 billion) to debtors on the list. No one is sent to a reeducation camp, but they may not be allowed to purchase luxury goods. According to the director of the Institute of the Chinese Academy of International Trade and Economic Cooperation, “Given this inconvenience, 10 percent of people on the list started to pay back the money they owed spontaneously. This shows the system is starting to work.” Economies of action were performing to plan.

The Chinese government’s vision may be impossibly ambitious: the big dream of total awareness and perfect certainty mediated by algorithms that filter a perpetual flood of data flows from private and public supplies, including online and offline experience culled from every domain and able to ricochet back into the individual lives of 1.5 billion people, automating social behavior as the algorithms reward, punish, and shape action right down to the latest bus ticket. So far the project is fragmented across many pilots, not only in the tech companies but also in cities and regions, so there is no real test of the scale that the government envisions. There are plenty of experts who believe that a single system of that scale and complexity will be difficult if not impossible to achieve.

The most common word for privacy, yinsi, didn’t even appear in popular Chinese dictionaries until the mid-1990s. Chinese citizens have accepted national ID cards with biometric chips, “birth permits,” and now social credit rankings because their society has been saturated with surveillance and profiling for decades. For example, the “dang’an” is a wide-ranging personal dossier compiled on hundreds of millions of urban residents from childhood and maintained throughout life. This “Mao-era system for recording the most intimate details of life” is updated by teachers, Communist Party officials, and employers. Citizens have no rights to see its contents, let alone contest them.

As distinct as our politics and cultures may be or have been, the emerging evidence of the Chinese social credit initiatives broadcasts the logic of surveillance capitalism and the instrumentarian power that it produces. Sesame Credit doubles down on every aspect of surveillance capitalist operations, with hundreds of millions of people caught in the gears of an automated behavioral modification machine and its bubbling behavioral futures markets dispersing perks and honors like Pokémon fairy dust in return for guaranteed outcomes.

Chinese users are rendered, classified, and queued up for prediction with every digital touch, and so are we. We are ranked on Uber, on eBay, on Facebook, and on many other web businesses, and those are only the rankings that we see. Chinese users are assigned a “character” score, whereas the US government urges the tech companies to train their algorithms for a “radicalism” score. Indeed, the work of the shadow text is to evaluate, categorize, and predict our behavior in millions of ways that we can neither know nor combat—these are our digital dossiers. When it comes to credit scoring, US and UK banks and financial services firms have floated business models based on the mining and analysis of social media data for credit scores. Facebook itself has hinted of its interest, even filing a patent. These efforts receded only because the Federal Trade Commission threatened regulatory intervention.

Oxford University China scholar Rogier Creemers, who translated some of the first documents on the social credit system, observes that “the trend towards social engineering and 'nudging' individuals towards 'better' behavior is also part of the Silicon Valley approach that holds that human problems can be solved once and for all through the disruptive power of technology.… In that sense, perhaps the most shocking element of the story is not the Chinese government's agenda, but how similar it is to the path technology is taking elsewhere.”

The difference between surveillance capitalism in the West and China's emerging social credit system pivots on the patterns of entanglement and engagement between instrumentarian and state power. There are structural differences. In the West, as we have seen, the patterns have taken on many forms. The state began as bosom and shelter, then eager student and envious cousin. Surveillance capitalism and its instruments have come of age now, producing a fitful but necessary partnership. Key instrumentarian capabilities are docked in the big surveillance capitalist firms, and the state must move with and through these firms to access much of the power it seeks.

In the Chinese context, the state will run the show and own it, not as a market project but as a political one, a machine solution that shapes a new society of automated behavior for guaranteed political and social outcomes: certainty without terror. All the pipes from all the supply chains will carry behavioral surplus to this new, complex means of behavioral modification. The state will assume the role of the behaviorist god, owning the shadow text and determining the schedule of reinforcements and the behavioral routines that it will shape. Freedom will be forfeit to knowledge, but it will be the state's knowledge that it exercises, not for the sake of revenue but for the sake of its own perpetuation.

Recall Carl Friedrich's observation on the challenge of grasping the naked facts of totalitarianism: “Virtually no one before 1914 anticipated the course of development which has overtaken Western civilization since then.… To this failure to foresee corresponds a difficulty in comprehending.” Recall too the grinning, robust “Joe” Stalin planted among Hollywood luminaries on the glossy pages of a 1939 Look magazine. Will we suffer the same lack of foresight as those who could not comprehend totalitarianism's rise, paralyzed by the sheer power of Big Other and its infinite echoes of consequence, distracted by our needs and confused by its speed, secrecy, and success?

Astonishment is a necessary alarm. We need it, but it should not leave us frozen in disbelief. The steady drumbeat of Big Other's manifest destiny, its breathtaking velocities, and the obscurity of its aims and purpose are intended to disarm, disorient, and bewilder. Inevitabilist ideology works to equate surveillance capitalism and its instrumentarian power with nature: not a human construction but something more like a river or a glacier, a thing that can only be joined or endured. All the more reason to ask: Might the banalities of today's declarations (“instruct the vehicular monitoring system not to allow the car to be started”) also reveal themselves in the fullness of time as the seeds of our century's greatest nightmare? What of the authors of the instrumentarian project? How will we appraise the smiling, robust faces of the tech titans when we revisit those images in the glossy pixels of some twenty-first-century version of Look?

Although he did not name it, the visionary of ubiquitous computing, Mark Weiser, foresaw the immensity of instrumentarian power as a totalizing societal project. He did so in a way that suggests both its utter lack of precedent and the danger of confounding it with what has gone before: “hundreds of computers in every room, all capable of sensing people near them and linked by high-speed networks have the potential to make totalitarianism up to now seem like sheerest anarchy.” In fact, all those computers are not the means to a digital hyper-totalitarianism. They are, as I think Weiser sensed, the foundation of an unprecedented power that can reshape society in unprecedented ways.

Seven decades ago, Skinner’s proto-instrumentarian behavioral utopia, Walden Two, was met with revulsion. Today the real thing is inspirational fodder for surveillance capitalist rhetoric as leaders promote the tools and visions that will bring the old professor’s ideas to life… to our lives. The processes of normalization and habituation have begun.

Although instrumentarianism and totalitarianism are distinct species, they each yearn toward totality, though in profoundly different ways. Totalitarianism seeks totality as a political condition and relies on violence to clear its path. Instrumentarianism seeks totality as a condition of market dominance, and it relies on its control over the division of learning in society, enabled and enforced by Big Other, to clear its path. The result is the application of instrumentarian power to societal optimization for the sake of market objectives: a utopia of certainty.

In the spring of 2017, Microsoft CEO Satya Nadella bounded onstage to open the company's annual developers' conference, his slender profile accentuated by the requisite black polo shirt, black jeans, and trendy black high-tops. He quickly dazzled the audience with a roll call of his troops. Nadella recounted the 500 million Windows 10 devices; 100 million monthly users of its Office software; 140 million monthly users of the corporation's digital "assistant," Cortana; and more than 12 million organizations signed on to its cloud services, including 90 percent of the Fortune 500.

Most unusual, however, is that Page portrays Google's totalistic ambitions as a logical consequence of its commitment to the perfection of society. From his point of view, we should welcome the opportunity to lean on Big Other and willingly subordinate all knowledge and decision rights to Google's plan. For the sake of the plan, the totality of society-every person, object, and process-must be corralled into the supply chains that feed the machines, which, in turn, spin the algorithms that animate Big Other to manage and mitigate our frailty: What you should want us to do is to really build amazing products and to really do that...

What you should want us to do is to really build amazing products and to really do that… we have to understand apps and we have to understand things you could buy, and we have to understand airline tickets. We have to understand anything you might search for. And people are a big thing you might search for.… We’re going to have people as a first class object in search… if we’re going to do a good job meeting your information needs, we actually need to understand things and we need to understand things pretty deeply.

Google originated in the prospect of optimally organizing the world’s information, but Page wants the corporation to optimize the organization of society itself: “In my very long-term worldview,” he said in 2013, “our software understands deeply what you’re knowledgeable about, what you’re not, and how to organize the world so that the world can solve important problems.”

Facebook CEO Mark Zuckerberg shares these totalistic ambitions, and he is increasingly frank about “society,” not just the individuals within it, as subordinate to Facebook’s embrace. His “three big company goals” include “connecting everyone; understanding the world; and building the knowledge economy, so that every user will have ‘more tools’ to share ‘different kinds of content.’” Zuckerberg’s keen appreciation of second-modernity instabilities—and the yearning for support and connection that is among its most-vivid features—drives his confidence, just as it did for Google economist Hal Varian. The corporation would know every book, film, and song a person had ever consumed. Predictive models would enable the corporation to “tell you what bar to go to” when you arrive in a strange city. The vision is detailed: when you arrive at the bar, the bartender has your favorite drink waiting, and you’re able to look around the room and identify people just like you.

In 2017 Zuckerberg went even further in articulating his societal ambitions, this time aiming straight at the heart of second-modernity anxieties: “People feel unsettled. A lot of what was settling in the past doesn’t exist anymore.” Zuckerberg believes that he and his company can provide a future “that works for everyone” and fulfills “personal, emotional, and spiritual needs” for “purpose and hope,” “moral validation,” and “comfort that we are not alone.” “Progress now requires humanity coming together not just as cities or nations,” Zuckerberg urged, “but also as a global community… the most important thing we at Facebook can do is develop the social infrastructure… to build a global community.…” Citing Abraham Lincoln, Facebook’s founder located his company’s mission in the evolutionary time line of civilization, during which humanity organized itself first in tribes, then cities, then nations. The next phase of social evolution would be “global community,” and Facebook was to lead the way, constructing the means and overseeing the ends.

As Nadella and other surveillance capitalists spin their utopian dreams, the surveillance capitalists fail to mention that the magical age they envision comes at a price: Big Other must expand toward totality as it deletes all boundaries and overwhelms every source of friction in the service of its economic imperatives. All power yearns toward totality, and only authority stands in the way: democratic institutions; laws; regulations; rights and obligations; private governance rules and contracts; the normal market constraints exercised by consumers, competitors, and employees; civil society; the political authority of the people; and the moral authority of individual human beings who have their bearings. … Instrumentarian power, like the apprentice's broom, has flourished in the sorcerer's absence with little authority to check its action, and the surveillance capitalists' appetite for totality has grown with this success. The utopian rhetoric of a magical age has been critical to this progress. The notion that Big Other will solve all of humanity's problems while empowering each individual is usually dismissed as mere "techno utopianism," but it would be a mistake for us to ignore this rhetoric without examining its purpose. Such discourse is no mere hogwash. It is the minesweeper that precedes the foot soldiers and the canny diplomat sent ahead to disarm the enemy and smooth the way for a quiet surrender. The promise of a magical age plays a critical strategic role, simultaneously distracting us from and legitimating surveillance capitalism's totalistic ambitions that necessarily include "people" as a "first class object."

The Manuels observe a sixth characteristic of the future-oriented modern visionary, and this is where the men and the corporations examined here represent powerful exceptions to the rule: “Often a utopian foresees the later evolution and consequences of technological development already present in an embryonic state; he may have antennae sensitive to the future. His gadgets, however, rarely go beyond the mechanical potentialities of his age. Try as he may to invent something wholly new, he cannot make a world out of nothing.” In our time, however, surveillance capitalists can and do make such a world—a genuinely historic deviation from the norm.

Individually and collectively, the knowledge, power, and wealth that surveillance capitalists command would be the envy of any ancient potentate, just as they are now coveted by the modern state. With 2017 balance sheets reporting $126 billion in cash and securities for Microsoft, $92 billion for Google, and about $30 billion for Facebook, and the financial markets endorsing their ever-expanding instrumentarian regimes with more than $1.6 trillion in market capitalization in mid-2017, these are the rare utopianists who can oversee the translation of their imaginations into fact without soldiers to pave the way in blood.

The surveillance capitalists reverse the normal sequence of theory and practice. Their practices move ahead at high velocity in the absence of an explicit and contestable theory. They specialize in displays of instrumentarianism’s unique brand of shock and awe, leaving onlookers dazed, uncertain, and helpless. The absence of a clear articulation of their theory leaves the rest of us to ponder its practical effects: the vehicular monitoring system that shuts down your engine; the destination that appears with the route; the suggested purchase that flashes on your phone the moment your endorphins peak; Big Other’s continuous tracking of your location, behavior, and mood; and its cheerful herding of city dwellers toward surveillance capitalism’s customers.

The only way to grasp the theory advanced in their applied utopistics is to reverse engineer their operations and scrutinize their meaning, as we have done throughout these chapters.

Applied utopistics are on the move at Facebook, Google, and Microsoft as the frontier of behavioral surplus extraction moves into realms of life traditionally understood as societal and elaborated under some combination of civil institutions and public leadership. Zuckerberg’s 2017 mission statement for Facebook, introduced as “building global community,” announced a new phase of applied utopistics: “Overall, it is important that the governance of our community scales with the complexity and demands of its people. We are committed to always do better, even if that involves building a worldwide voting system to give you more voice and control. Our hope is that this model provides examples of how collective decision-making may work in other aspects of the global community.”

Microsoft’s Nadella encouraged his developers: “Whether it’s precision medicine or precision agriculture, whether it’s digital media or the industrial internet, the opportunity for us as developers to have broad, deep impact on all parts of society and all parts of the economy has never been greater.” The vision that Nadella unveiled that day is emblematic of the wider surveillance capitalist template for our futures.

In order to decipher the true measure of an instrumentarian society, let’s set aside the razzle-dazzle of a “magic age” and focus instead on the practices of applied utopistics and the social vision they imply. Nadella provided a valuable opportunity when he unveiled a series of practical applications that imply a sweeping new vision of machine relations as the template for a new era’s social relations.

Just as the division of labor migrated from the economic domain to society in the twentieth century, Nadella's construction site is the economic petri dish in which a new division of learning mutates into life, ready for translation to society. In the twentieth century the critical success factors of industrial capitalism—efficiency, productivity, standardization, interchangeability, the minute division of labor, discipline, attention, scheduling, conformity, hierarchical administration, the separation of knowing and doing, and so forth—were discovered and crafted in the workplace and then transposed to society, where they were institutionalized in schools, hospitals, family life, and personality. As generations of scholars have documented, society became more factory-like so that we might train and socialize the youngest among us to fit the new requirements of a mass production order.

We have entered this cycle anew, but now the aim is to remake twenty-first-century society as a "first-class thing" organized in the image of the machine hive for the sake of others' certainty. The connectedness that we once sought for personal sustenance and effectiveness is recast as the medium for a new species of power and the social confluence that translates into guaranteed outcomes.

The imaginative range of this new thinking is demonstrated in a 2013 Microsoft patent application updated and republished in 2016 and titled "User Behavior Monitoring on a Computerized Device." With conspicuously thin theory complemented by thick practice, the patented device is designed to monitor user behavior in order to preemptively detect "any deviation from normal or acceptable behavior that is likely to affect the user's mental state. A prediction model corresponding to features of one or more mental states may be compared with features based upon current user behavior."

All these behavioral data are stored for future historical analyses in order to improve the prediction model. If the user normally restrains the volume of his or her voice, then sudden excessive shouting may indicate a “psychosocial event.” Alternatively, the behavior could be assessed in relation to a “feature distribution representing normal and/or acceptable behavior for an average member of a population… a statistically significant deviation from that behavior baseline indicates a number of possible psychological events.” The initial proposition is that in the event of an anomaly, the device would alert “trusted individuals” such as family members, doctors, and caregivers. But the circle widens as the patent specifications unfold. The scientists note the utility of alerts for health care providers, insurance companies, and law-enforcement personnel. Here is a new surveillance-as-a-service opportunity geared to preempt whatever behavior clients choose.

Microsoft’s patent returns us to Planck, Meyer, and Skinner and the viewpoint of the Other-One. In their physics-based representation of human behavior, anomalies are the “accidents” that are called freedom but actually denote ignorance; they simply cannot yet be explained by the facts. Planck/Meyer/Skinner believed that the forfeit of this freedom was the necessary price to be paid for the “safety” and “harmony” of an anomaly-free society in which all processes are optimized for the greater good. Skinner imagined that with the correct technology of behavior, knowledge could preemptively eliminate anomalies, driving all behavior toward preestablished parameters that align with social norms and objectives.

In this template of social relations, behavioral modification operates just beyond the threshold of human awareness to induce, reward, goad, punish, and reinforce behavior consistent with “correct policies.” Thus, Facebook learns that it can predictably move the societal dial on voting patterns, emotional states, or anything else that it chooses. Niantic Labs and Google learn that they can predictably enrich McDonald’s bottom line or that of any other customer. In each case, corporate objectives define the “policies” toward which confluent behavior harmoniously streams.

Schmidt and Thrun emphasize the “crucial insight that differentiates AI from the way people learn.” Instead of the typical assurances that machines can be designed to be more like human beings and therefore less threatening, Schmidt and Thrun argue just the opposite: it is necessary for people to become more machine-like. Machine intelligence is enthroned as the apotheosis of collective action in which all the machines in a networked system move seamlessly toward confluence, all sharing the same understanding and thus operating in unison with maximum efficiency to achieve the same outcomes.

When driving, people mostly learn from their own mistakes, but they rarely learn from the mistakes of others. People collectively make the same mistakes over and over again. As a result, hundreds of thousands of people die worldwide every year in traffic collisions. AI evolves differently. When one of the self-driving cars makes an error, all of the self-driving cars learn from it. In fact, new self-driving cars are “born” with the complete skill set of their ancestors and peers. So collectively, these cars can learn faster than people.

In this way, the machine hive becomes the role model for a new human hive in which we march in peaceful unison toward the same direction based on the same “correct” understanding in order to construct a world free of mistakes, accidents, and random messes. In this world the “correct” outcomes are known in advance and guaranteed in action. The same ubiquitous instrumentation and transparency that define the machine system must also define the social system, which in the end is simply another way of describing the ground truth of instrumentarian society.

In this human hive, individual freedom is forfeit to collective knowledge and action. Nonharmonious elements are preemptively targeted with high doses of tuning, herding, and conditioning, including the full seductive force of social persuasion and influence. We march in certainty, like the smart machines. We learn to sacrifice our freedom to collective knowledge imposed by others and for the sake of their guaranteed outcomes. This is the signature of the third modernity offered up by surveillance capital as its answer to our quest for effective life together.

Pentland’s research lab is funded by a who’s who of global corporations, consultancies, and governments: Google, Cisco, IBM, Deloitte, Twitter, Verizon, the EU Commission, the US government, the Chinese government, “and various entities who are all concerned with why we don’t know what’s going on in the world.…”

Although Pentland is not alone in this field, he is something of a high priest among an exclusive group of priests. Unlike Hal Varian, Pentland does not speak of Google in the first-person plural, but his work is showcased in surveillance capitalist enclaves, where it provides the kind of material and intellectual support that helps to legitimate instrumentarian practices.

Pentland’s academic credentials and voluble intelligence help legitimate a social vision that repelled and alarmed intellectuals, public officials, and the general public just decades ago. Most noteworthy is that Pentland “completes” Skinner, fulfilling his social vision with big data, ubiquitous digital instrumentation, advanced mathematics, sweeping theory, numerous esteemed coauthors, institutional legitimacy, lavish funding, and corporate friends in high places without having attracted the worldwide backlash, moral revulsion, and naked vitriol once heaped on Harvard’s outspoken behaviorist. This fact alone suggests the depth of psychic numbing to which we have succumbed and the loss of our collective bearings.

Professor Pentland began his intellectual journey as Skinner did, in the study of animal behavior. Where Skinner trained his reasoning on the detailed behaviors of blameless individual creatures, Pentland concerned himself with the gross behavior of animal populations. As a part-time researcher at NASA’s Environmental Research Institute while still an undergraduate, he developed a method for assessing the Canadian beaver population from space by counting the number of beaver ponds: “You’re watching the lifestyle, and you get an indirect measure.”

Skinner bitterly lamented the absence of “instruments and methods” for the study of human behavior comparable to those available to physicists. As if in response, Pentland and his students have spent the last two decades determined to invent the instruments and methods that can transform all of human behavior, especially social behavior, into highly predictive math. An early milestone was a 2002 collaboration with then-doctoral student Tanzeem Choudhury, in which the coauthors wrote, “As far as we know, there are currently no available methods to automatically model face-to-face interactions. This absence is probably due to the difficulty of obtaining reliable measurements from real-world interactions within a community.… We believe sensing and modeling physical interactions among people is an untapped resource.” In other words, the “social” remained an elusive domain even as data and computers had become more commonplace.

The researchers’ response was to introduce the “sociometer,” a wearable sensor that combines a microphone, accelerometer, Bluetooth connection, analytic software, and machine learning techniques designed to infer “the structure and dynamic relationships” in human groups. (Choudhury would eventually run the People Aware Computing group at Cornell University.) From that point onward, Pentland and his teams have worked to crack the code on the instrumentation and instrumentalization of social processes in the name of a totalistic social vision founded on a comprehensive means of behavior modification.

Pentland and Eagle began with 100 MIT students and faculty centered at the Media Lab, equipping them with 100 Nokia phones preloaded with special software in a project that would become the basis of Eagle’s doctoral dissertation. The two researchers exposed the revelatory power of continuously harvested behavioral data, which they validated with survey information collected directly from each participant. Their analyses produced detailed portraits of individual and group life: the “social system,” as the authors called it. They were able to specify regular temporal and spatial patterns of location, activity, and communication-use patterns, which together enabled predictions of up to 90 percent accuracy regarding where someone was likely to be and what that person was likely to be doing within the hour, as well as highly accurate predictions about an individual’s colleagues, casual friends, and close relationships. The team identified patterns of communication and interaction within work groups, as well as the broad “organizational rhythms and network dynamics” of the Media Lab. (Eagle became CEO of Jana, a mobile advertising company that offers free internet to emerging markets in exchange for behavioral surplus.)

Pentland understands the rapid encroachments of Big Other and instrumentarian power as a “light-speed, hyperconnected world” where virtual crowds of millions from anywhere in the world “can form in minutes.” He views the MIT community as the avant-garde: brilliant pioneers of light speed, already in sync with its extreme velocities and thus a model for the rest of society. Reflecting on his students and colleagues, Pentland writes that “I have also gotten to see how creative cultures must change in order to thrive in the hyperconnected, warp-speed world that is MIT, an environment that the rest of the world is now entering.” Pentland reasons that his group’s adaptation to MIT’s norms of rapid deployment simply foreshadows what is in store for the rest of us.

As we go about our daily lives, we leave behind virtual breadcrumbs—digital records of the people we call, the places we go, the things we eat and the products we buy. These breadcrumbs tell a more accurate story of our lives than anything we choose to reveal about ourselves.… Digital breadcrumbs… record our behavior as it actually happened.

Pentland was among the first to recognize the commercial relevance of behavioral surplus. Although he does not discuss it explicitly, he appears to embrace the realpolitik of surveillance capitalism as the necessary condition for an instrumentarian society. Pentland’s own companies are extensions of his applied utopistics: proving grounds for instrumentarian techniques and the habituation of populations to pervasive rendition, monitoring, and modification in pursuit of surveillance revenues.

The goal, the authors said, was to make machines that can “monitor social communication and provide real-time intervention.” To that end, twenty-two office employees were “instrumented” with the badge for one month in order to “automatically measure individual and collective patterns of behavior, predict human behavior from unconscious social signals, identify social affinity among individuals working in the same team, and enhance social interactions by providing feedback to the users of our system.” The research provided credible results, revealing patterns of communication and behavior that the authors concluded “would not be available without the use of a device such as the sociometric badge. Our results… argue strongly for the use of automatic sensing data collection tools to understand social systems.”

In 2010 Pentland and his 2009 coauthors founded a company, Sociometric Solutions, to bring Skinner’s longed-for “instruments and methods” to the marketplace. It was one of many companies that Pentland would create to apply the rigors of his social physics to captive populations of office workers. Sociometric Solutions’ CEO, Ben Waber, one of Pentland’s doctoral students, calls his operation “people analytics,” and in his book of the same name, he anticipates a future of “connection, collaboration, and data” with the badge or something like it “deployed across millions of individuals at different companies in countries all over the world for not minutes but years or decades.… Imagine what we could learn about to help people collaborate more effectively.…”

Pentland and his crew continued to develop the sociometer and its applications, and by 2013 the device had been used by dozens of research groups and companies, including members of the Fortune 1000. A 2014 study, authored with Waber and colleagues from Harvard and Northeastern University, quantified gender differences in interaction patterns. The success of the analysis occasioned this announcement: “It is now possible to actively instrument human behavior to collect detailed data on various dimensions of social interaction.” The authors signaled their aim to employ MacKay’s cardinal rule of unobtrusive surveillance for effective monitoring of herds, flocks, and packs, acknowledging that the continuous pervasive collection of human behavioral data could succeed only when conducted outside the boundaries of human awareness, thus eliminating possible resistance, just as we saw at Facebook.

Writing in Scientific American on the power of sociometric data, Pentland says, “I persuaded the manager of a Bank of America call center to schedule coffee breaks simultaneously. The goal was to promote more engagement between employees. This single change resulted in a productivity increase of $15 million a year.”

Pentland understands his experiments and paid interventions in workplace settings as emblematic of the larger challenges of social relations in an instrumentarian society. Once again we see the intended path from the economic to the social domain. Those instrumented office workers function as living labs for the translation of instrumentarian relations to the wider society. … Like Nadella, Pentland described his aims as developing the social systems that would work along the same lines as the machine systems, using behavioral data flows to judge the "correctness" of action patterns and to intervene when it is necessary to change "bad" action to "correct" action. "If people aren't interacting correctly and information isn't spreading correctly," Pentland warns, "people make bad decisions.… What you're trying to do is make a human-machine symbiote, where the humans understand more about the network of interactions because of the computers, and the computers are able to understand more about how humans work."

New digital systems are required that must be “integrated,” “holistic,” “responsive,” “dynamic,” and “self-regulating”: “We need a radical rethinking of societies’ systems. We must create a nervous system for humanity that maintains the stability of our societies’ systems throughout the globe.”

But Pentland saw a problem. Although ubiquitous technologies are well on the way to solving the technical challenges of a global nervous system, Big Other will not be complete until it also understands human behavior on a global scale: “What is missing… are the dynamic models of demand and reaction,” along with an architecture that guarantees “safety, stability, and efficiency.… The models required must describe human demand and reactions, since humans are at the core of all of these systems… the necessary observations are observations of individual behavior.…”

Pentland had identified a dangerous void, which foreshadows the “profound shift” that Nadella extolled to Microsoft developers in 2017 when he said: “People and their relationship with other people is now a first-class thing in the cloud!” “People” would have to become part of Big Other’s purview, lest they fall prey to “incorrect” behavior. Society’s safety, stability, and efficiency hang in the balance.

For the first time in history, the majority of humanity is linked.… As a consequence, our mobile wireless infrastructure can be "reality mined" in order to… monitor our environments, and plan the development of our society.… Reality mining of the "digital breadcrumbs" left behind as we go about our daily lives offers potential for creating remarkable, second-by-second models of group dynamics and reactions over extended periods of time.… In short, we now have the capacity to collect and analyze data about people with a breadth and depth that was previously inconceivable.

Pentland insists that the "old" social categories of status, class, education, race, gender, and generation are obsolete, as irrelevant as the energy, food, and water systems that he wants to replace. Those categories describe societies through the lens of history, power, and politics, but Pentland prefers "populations" to societies, "statistics" to meaning, and "computation" to law. He sees the "stratification of the population" coded not by race, income, occupation, or gender but rather by "behavior patterns" that produce "behavior subgroups" and a new "behavior demographics" that can predict disease, financial risk, consumer preferences, and political views with "between 5 and 10 times the accuracy" of the standard measures.

A final question is urgently posed: "how to get the humans in these systems to participate in the plan?" His answers do not lie in persuasion or education but in behavioral modification. He says we need "new predictive theories of human decision making" as well as "incentive mechanism design," an idea that is comparable to Skinner's "schedules of reinforcement." Regarding how to get humans to follow the plan, Pentland offers the principle of "social influence" to explain the design mechanisms through which millions of human beings can be herded toward the guaranteed outcomes of safety, stability, and efficiency.

Pentland understands that Big Other is not only an architecture that monitors and controls things. Big Other's instrumentation and data flows also make people mutually visible to one another, from the updates on your breakfast to the population flows in cities. Back in 2011, Pentland enthused, "Revolutionary new… infrastructures are providing us with a God's eye view of ourselves." The aim is a computer-mediated society where our mutual visibility becomes the habitat in which we attune to one another, producing social patterns based on imitation that can be manipulated for confluence, just as the logic of the machine hive suggests.

For society, the hope is that we can use this new in-depth understanding of individual behavior to increase the efficiency and responsiveness of industries and governments. For individuals, the attraction is the possibility of a world where everything is arranged for your convenience—your health checkup is magically scheduled just as you begin to get sick, the bus comes just as you get to the bus stop, and there is never a line of waiting people at city hall. As these new abilities become refined by the use of more sophisticated statistical models and sensor capabilities, we could well see the creation of a quantitative, predictive science of human organizations and human society.

Pentland is sanguine on this point: total knowledge is within reach. As he states, “In just a few short years we are likely to have incredible rich data available about the behavior of virtually all of humanity—on a continuous basis. The data mostly already exists.” The right to the future tense—and with it social trust, authority, and politics—is surrendered to Big Other and the transcendent computational systems that rule society under the watchful eye of a group that Pentland calls “we.” He never defines this “we,” which imposes an us-them relationship, introducing the exclusivity of the shadow text and its one-way mirror. It is an omission that haunts his text. Does it refer to the priesthood of data scientists like Pentland? The priesthood in collaboration with the owners of the means of behavior modification?

Pentland understands instrumentarian society as an historical turning point comparable to the invention of the printing press or the internet. It means that for the first time in human history, "We will have the data required to really know ourselves and understand how society evolves." Pentland says that "continuous streams of data about human behavior" mean that everything from traffic, to energy use, to disease, to street crime will be accurately forecast, enabling a "world without war or financial crashes, in which infectious disease is quickly detected and stopped, in which energy, water, and other resources are no longer wasted, and in which governments are part of the solution rather than part of the problem." This new "collective intelligence" operates to serve the greater good as we learn to act "in a coordinated manner" based on "social universals."

Pentland avoids the question "Whose greater good?" How is the greater good determined when surveillance capitalism owns the machines and the means of behavioral modification? "Goodness" arrives already oriented toward the interests of the owners of the means of behavioral modification and the clients whose guaranteed outcomes they seek to achieve. The greater good is someone's, but it may not be ours.

Smooth operations leave no room for unreasonable or unintentional outcomes, and Skinner viewed the creative and often messy conflicts of politics, especially democratic politics, as a source of friction that threatens the rational efficiency of the community as a single, high-functioning "superorganism." He laments our inclination to try to change things with "political action," and he endorses what he perceives as a widespread loss of faith in democracy. In Walden Two Frazier insists that "I don't like the despotism of ignorance. I don't like the despotism of neglect, of irresponsibility, the despotism of accident, even. And I don't like the despotism of democracy!"

In Walden Two a "plan" replaces politics, overseen by a "noncompetitive" group of "Planners" who eschew power in favor of the dispassionate administration of the schedules of reinforcement aimed at the greater good. Planners exercise unique control over society but "only because that control is necessary for the proper functioning of the community."

Pentland worries that our political-economic constructs such as “market” and “class” hail from an old, slow world of the eighteenth and nineteenth centuries. The new, “light-speed hyperconnected world” leaves no time for the kind of rational deliberation and face-to-face negotiation and compromise that characterized the social milieu in which such political concepts originated: “We can no longer think of ourselves as only individuals reaching carefully considered decisions; we must include the dynamic social effects that influence our individual decisions and drive economic bubbles, political revolutions, and the internet economy.”

The velocity of instrumentarian society leaves us no time to get our bearings, and that speed is repurposed here as a moral imperative demanding that we relinquish individual agency to the automated systems that can keep up the pace in order to quickly perceive and impose correct answers for the greater good. There is no room for politics in this instrumentarian society because politics means establishing and asserting our bearings. Individual moral and political bearings are a source of friction that wastes precious time and diverts behavior from confluence.

Instead of politics, markets, classes, and so on, Pentland reduces society to his laws of social physics: a reincarnation of Skinner’s “communal science.” Indeed, Pentland regards his work as the practical foundation of a new “computational theory of behavior” capable of producing a “causal theory of social structure… a mathematical explanation of why society reacts as it does and how these reactions may (or may not) solve human problems.…” These new mathematical analyses not only reveal the deep “mechanisms of social interactions” (Skinner’s “special capacities of the group”) but also combine with “our newly acquired massive amounts of behavior data” in order to reveal the patterns of causality that make it possible to “engineer better social systems,” all of it based on “unprecedented instrumentation.”

Computation thus replaces the political life of the community as the basis for governance. The depth and breadth of instrumentation make it possible, Pentland says, to calculate idea flow, social network structure, the degree of social influence between people, and even “individual susceptibilities to new ideas.” Most important, instrumentation makes it possible for those with the God view to modify others’ behavior. The data provide a “reliable prediction of how changing any of these variables will change the performance of all the people in the network” and thus achieve the optimum performance of Skinner’s superorganism. This mathematics of idea flow is the basis for Pentland’s version of a “plan” that dictates the targets and objectives of behavior change. Human behavior must be herded and penned within the parameters of the plan, just as behavior at Nadella’s construction site was continuously and automatically molded to policy parameters. Pentland calls this “tuning the network.”

In the community of Walden Two, reinforcement is precisely orchestrated to eliminate emotions that threaten cooperation. Only “productive and strengthening emotions—joy and love” are allowed. Feelings of sorrow and hate “and the high-voltage excitements of anger, fear, and rage” are considered “wasteful and dangerous” threats to “the needs of modern life.” Any form of distinction between persons undermines the harmony of the whole and its capacity to bend to collective purpose. Frazier acknowledges that you cannot coerce people into doing the right thing. The solution is far more subtle and sophisticated, based upon scientifically calibrated schedules of reinforcement: “Instead you have to set up certain behavioral processes which will lead the individual to design his own ‘good’ conduct.… We call that sort of thing ‘self-control.’ But don’t be misled, the control always rests in the last analysis in the hands of society.”

Pentland finds Facebook’s “contagion” experiments particularly enlightening, seeing all sorts of practical insights in their complexities. For example, in the corporation’s 61-million-person voting experiment, Pentland sees confirmation that social pressure can be effectively instrumentalized in social networks, especially among people with “strong ties”: “The knowledge that our face-to-face friends had already voted generated enough social pressure that it convinced people to vote.” … With this knowledge and more like it, Pentland’s “we,” the tuners, will be able to activate the “right incentives.”

The God view persuades him that assessing human action really is comparable to counting beavers: “We can observe humans in just the same way we observe apes or bees and derive rules of behavior, reaction, and learning.” In all of these populations, the collective exerts pressure on each organism to go with the flow, stay with the herd, return to the hive, and take flight with the flock. Idea flows mimic the pattern of the machine hive, the edge conflates with the hub, identity yields to and synchrony, the parts dissolve in the whole. … I believe that we can we think of each stream of ideas as a swarm or collective intelligence, flowing through time, with all the humans in it learning from each other’s experiences in order to jointly discover the patterns of preferences and habits of action that best suit the surrounding physical and social environment. This is counter to the way most modern Westerners understand themselves, which is as rational individuals, people who know what they want and who decide for themselves what actions to take in order to accomplish their goals. … This shift from society to swarm and from individuals to organisms is the cornerstone upon which the structure of an instrumentarian society rests.

Pentland ignores the role of empathy in emulation because empathy is a felt experience that is not subject to the observable metrics required for computational governance. Instead, Pentland subscribes to the label Homo imitans to convey that it is mimicry, not empathy, and certainly not politics, which defines human existence. The term itself derives from studies of infant learning, but for Pentland it is a fitting explanation of all human behavior all the time: an assertion, like Skinner’s, that control always rests with society. “The largest single factor driving adoption of new behaviors,” he writes, is “the behavior of peers.”

Because we are born to imitate one another, Pentland argues, the whole species is attuned to social pressure as an efficient means of behavioral modification. This model of human learning is a throwback to bees and apes but also a forward pass to the machine hive. Machines do not learn by empathy; learning is automatically updated in a lockstep progression of collective intelligence.

Instrumentarian society is a planned society, produced through total control of the means of behavioral modification. Neither Skinner's planners nor Pentland's tuners shrink from their responsibility to wield the power that shapes the superorganism.

Here again the behaviorist from Harvard had already sounded the first and most eloquent iteration of this message, elevating the Other-One and denouncing the autonomous self. In Beyond Freedom & Dignity, Skinner freely displayed his contempt for this most transcendent ideal of the Sartrean age: the will to will oneself into first-person voice and action. Skinner argued that the differences between humans and other species are greatly exaggerated, and he would have found Pentland entirely justified in his rejection of the individual in favor of the distant, computer-mediated gaze. Beavers or people, the variance hardly matters once we shed the destructive fiction of individual autonomy. The surrender of the individual to manipulation by the planners clears the way for a safe and prosperous future built on the forfeit of freedom for knowledge.

What is being abolished is autonomous man—the inner man, the homunculus, the possessing demon, the man defended by the literatures of freedom and dignity. His abolition has long been overdue.… He has been constructed from our ignorance, and as our understanding increases, the very stuff of which he is composed vanishes… and it must do so if it is to prevent the abolition of the human species. To man qua man we readily say good riddance. Only by dispossessing him can we turn… from the inferred to the observed, from the miraculous to the natural, from the inaccessible to the manipulable.

Unlike Skinner, though, Pentland is careful to round the square, perhaps in the hope of evading a withering review by the likes of Noam Chomsky. … Pentland sidesteps the risks of Skinner’s jeremiad by taking a softer tone: “Some people react negatively to the phrase social physics, because they feel that it implies that people are machines without free will and without the ability to move independently of our role in society.” Like Meyer, Pentland acknowledges that humans have a “capacity for independent thought” but insists that social physics “does not need to try to account for it.” As Pentland sees it, the problem is not that “independent thought” is omitted from the picture but rather that “internal, unobservable” thought processes are just friction that “will occasionally emerge to defeat our best social physics models.” Fortunately, the models are not really in danger because “the data tell us that deviations from our regular social patterns occur only a few percent of the time.” The autonomous individual is but a statistical blip, a slip of the pen that is easily overridden in the march toward confluent action and someone’s greater good.

One important study of Bitcoin, the cryptocurrency that relies on blockchain, suggests that such machine solutions both express and contribute to the general erosion of the social fabric in ways that are both consistent with instrumentarianism and further pave the way for its success. Information scholars Primavera De Filippi and Benjamin Loveluck conclude that contrary to popular belief, “Bitcoin is neither anonymous nor privacy-friendly… anyone with a copy of the blockchain can see the history of all Bitcoin transactions… every transaction ever done on the Bitcoin network can be traced back to its origin.” Such systems rely on “perfect information,” but the kinds of coordination processes that build open democratic societies, such as “social trust” or “loyalty,” are “expunged” in favor of “a profoundly market-driven approach.”

It is no surprise that capitalism shapes social relations. A century ago it was the new means of mass production that fashioned mass society in its image. Today, surveillance capitalism offers a new template for our future: the machine hive in which our freedom is forfeit to perfect knowledge administered for others' profit.

Surveillance capitalists work hard to camouflage their purpose as they master the uses of instrumentarian power to shape our behavior while evading our awareness. That is why Google conceals the operations that turn us into the objects of its search and Facebook distracts us from the fact that our beloved connections are essential to the profit and power that flow from its network ubiquity and totalistic knowledge.

Without knowing it, we sent the least formed and most vulnerable among us to scout the hive and settle its wilderness. Now their messages are filtering in from the frontier.

All grew so fast his life was overgrown, Till he forgot what all had once been made for: “I felt so lonely… I could not sleep well without sharing or connecting to others,” a Chinese girl recalled. “Emptiness,” an Argentine boy moaned. “Emptiness overwhelms me.” A Ugandan teenager muttered, “I felt like there was a problem with me,” and an American college student whimpered, “I went into absolute panic mode.” These are but a few of the lamentations plucked from one thousand student participants in an international study of media use that spanned ten countries and five continents. They had been asked to abstain from all digital media for a mere twenty-four hours, and the experience released a planet-wide gnashing of teeth and tearing of flesh that even the study’s directors found disquieting. Capping the collective cri de coeur, a Slovakian university student reflected, “Maybe it is unhealthy that I can’t be without knowing what people are saying and feeling, where they are, and what’s happening.” … The students’ accounts are a message in a bottle for the rest of us, narrating the mental and emotional milieu of life in an instrumentarian society with its architectures of behavioral control, social pressure, and asymmetrical power. Most significantly, our children are harbingers of the emotional toll of the viewpoint of the Other-One as young people find themselves immersed in a hive life, where the other is an “it” to me, and I experience myself as the “it” that others see.

Facebook has learned to bite hard on the psychological needs of young people, creating new challenges for the developmental processes that build individual identity and personal autonomy.

The international "unplug" study helps to set the stage, for it reveals a range of emotional anguish summarized in six categories: addiction, failure to unplug, boredom, confusion, distress, and isolation. The students' sudden disconnection from the network produced the kinds of cravings, depression, and anxiety that are characteristic of clinically diagnosed addictions. The result was that a majority in every country admitted that they could not last out the day unplugged. Their angst was compounded by the same Faustian pact with which we are all too familiar, as they discovered that nearly all daily logistical, communicative, and informational requirements were dependent upon their connected devices: "Meeting with friends became difficult or impossible, finding the way to a destination without an online map or access to the internet became a problem, and simply organizing an evening at home became a challenge."

Facebook’s early advantage in this work arose in no small measure from the simple fact that its founders and original designers were themselves adolescents and emerging adults. They designed practices for an imagined universe of adolescent users and college students, and those practices were later institutionalized for the rest of us, reducing the social world to a tally of “friends” who are not friends and “likes” that provide a continuous ticker tape of one’s value on the social market, stoking the anxieties of pre-adulthood and anticipating the mesmerizing social disciplines of the hive.

It wasn’t simply that they didn’t know what to do with themselves but rather that “they had problems articulating what they were feeling or even who they were if they couldn’t connect.” The students felt as though “they had lost part of themselves.”

These feelings of disorientation and isolation suggest a psychological dependency on the “others,” and additional studies only deepen our understanding of how “Generation Z,” the demographic cohort born in and after 1996—the first group of digital natives, with no memory of life before the rise of surveillance capitalism—relies on a range of social media for psychological sustenance as they bounce between four or five platforms more or less simultaneously. Consider first the older cohorts. A 2012 survey concluded that emerging adults devote more time to using media than any other daily activity, spending nearly twelve hours each day with media of some form. By 2018 Pew Research reported that nearly 40 percent of young people ages 18–29 report being online “almost constantly,” as do 36 percent of those ages 30–49. Generation Z intensifies the trend: 95 percent use smartphones, and 45 percent of teens say they are online “on a near-constant basis.” If that is how you spend your days and nights, then the findings of a 2016 study are all too logical, as 42 percent of teenage respondents said that social media affects how people see them, having adopted what the researchers call an outside-looking-in approach to how they express themselves. Their dependency penetrates deeply into their sense of well-being, affecting how they feel about themselves (42 percent) and their happiness (37 percent).

In a subsequent elaboration on the psychological consequences of experiencing oneself from the “outside looking in,” a 2017 survey of young British women ages 11–21 suggests that the social principles of instrumentarian society, so enthusiastically elaborated by Pentland and endorsed by surveillance capitalist leaders, appear to be working effectively. Thirty-five percent of the women said that their biggest worry online was comparing themselves and their lives with others as they are drawn into “constant comparisons with often idealized versions of the lives, and bodies, of others.”

In light of these findings, one UK medical specialist comments on the young people in her practice: “People are growing up to want to be influencers and that is now a job role.… I am not sure if parents are fully aware of the pressure people face.…” Indeed, only 12 percent of respondents in that 2017 survey reckoned that their parents understood these pressures. The reports confirm that social pressure is well institutionalized as the means of online social influence, but contrary to Pentland’s belief that “class” divisions would disappear, life in the hive produces new cleavages and forms of stratification: not only tune or be tuned but also pressure or be pressured.

Nothing summarizes young life in the hive better than the insights of Facebook's own North American marketing director, Michelle Klein, who told an audience in 2016 that while the average adult checks his or her phone 30 times a day, the average millennial, she enthusiastically reported, checks more than 157 times daily. Generation Z, we now know, exceeds this pace. Klein described Facebook's engineering feat: “a sensory experience of communication that helps us connect to others, without having to look away,” noting with satisfaction that this condition is a boon to marketers. She underscored the design characteristics that produce this mesmerizing effect: design is narrative, engrossing, immediate, expressive, immersive, adaptive, and dynamic.

If you are over the age of thirty, you know that Klein is not describing your adolescence, or that of your parents, and certainly not that of your grandparents. Adolescence and emerging adulthood in the hive are a human first, meticulously crafted by the science of behavioral engineering; institutionalized in the vast and complex architectures of computer-mediated means of behavior modification; overseen by Big Other; directed toward economies of scale, scope, and action in the capture of behavioral surplus; and funded by the surveillance capital that accrues from unprecedented concentrations of knowledge and power. Our children endeavor to come of age in a hive that is owned and operated by the applied utopianists of surveillance capitalism and is continuously monitored and shaped by the gathering force of instrumentarian power. Is this the life that we want for the most open, pliable, eager, self-conscious, and promising members of our society?

The answer lies in a combination of behavioral science and high-stakes design that is precision-tooled to bite hard on the felt needs of this age and stage: a perfectly fitted hand and glove. Social media is designed to engage and hold people of all ages, but it is principally molded to the psychological structure of adolescence and emerging adulthood, when one is naturally oriented toward the “others,” especially toward the rewards of group recognition, acceptance, belonging, and inclusion. For many, this close tailoring, combined with the practical dependencies of social participation, turns social media into a toxic milieu. Not only does this milieu extract a heavy psychological toll, but it also threatens the course of human development for today's young and the generations that follow, all spirits of a Christmas Yet to Come.

The hand-and-glove relationship of technology addiction was not invented at Facebook, but rather it was pioneered, tested, and perfected with outstanding success in the gaming industry, another setting where addiction is formally recognized as a boundless source of profit. Skinner had anticipated the relevance of his methods to the casino environment, which executives and engineers have transformed into as vivid an illustration as one can muster of the startling power of behavioral engineering and its ability to exploit individual inclinations and transform them into closed loops of obsession and compulsion.

Schüll learned that addictive players seek neither entertainment nor the mythical jackpot of cash. Instead, they chase what Harvard Medical School addiction researcher Howard Shaffer calls “the capacity of the drug or gamble to shift subjective experience,” pursuing an experiential state that Schüll calls the “machine zone,” a state of self-forgetting in which one is carried along by an irresistible momentum that feels like one is “played by the machine.” The machine zone achieves a sense of complete immersion that recalls Klein’s description of Facebook’s design principles—engrossing, immersive, immediate—and is associated with a loss of self-awareness, automatic behavior, and a total rhythmic absorption carried along on a wave of compulsion.

The key, one casino executive says in words that are all too familiar, “is figuring out how to leverage technology to act on customers’ preferences [while making] it as invisible—or what I call auto-magic—as possible.”

These innovations drive up revenues with the sheer volume of extended play as each machine is transformed into a “personalized reward device.” The idea, as the casinos came to understand it, is to avoid anything that distracts, diverts, or interrupts the player’s fusion with the machine; consoles “mold to the player’s natural posture,” eliminating the distance between the player’s body and frictionless touch screens: “Every feature of a slot machine—its mathematical structure, visual graphics, sound dynamics, seating and screen ergonomics—is calibrated to increase a gambler’s ‘time on device’ and to encourage ‘play to extinction.’” The aim is a kind of crazed machine sex, an intimate closed-loop architecture of obsession, loss of self, and auto-gratification.

There are some chinks in the armor. For example, in 2017 Napster cofounder and one-time Facebook president Sean Parker frankly admitted that Facebook was designed to consume the maximum possible amount of users' time and consciousness. The idea was to send you "a little dopamine hit every once in a while"—a.k.a. "variable reinforcement—in the form of 'likes' and comments. The goal was to keep users glued to the hive, chasing those hits while leaving a stream of raw materials in their wake."

Shaffer, the addiction researcher, has identified five elements that characterize this state of compulsion: frequency of use, duration of action, potency, route of administration, and player attributes. We already know quite a bit about the high frequency and long duration of young people's engagement in social media. What we need to understand is something of (1) the psychological attributes that draw them to social media in the first place (the hand), (2) the design practices that ratchet up potency in order to transform inclinations into unquenchable need (the glove), and (3) the mental and emotional consequences of Facebook's ever-more-exquisite ability to enmesh young people in chasing their own kind of zone.

Consider the final moments of a 2017 Washington Post profile on a thirteen-year-old girl, part of a series chronicling "what it's like to grow up in the age of likes, lols, and longing." It is the girl's birthday, and only one question will decide her happiness: do her friends like her enough to post pictures of her on their pages in appreciation of the occasion? "She scrolls, she waits. For that little notification box to appear." Regardless of your age, who among us does not feel a painful blast of recognition? Adolescence has always been a time when acceptance, inclusion, and recognition from the "others" can feel like matters of life and death, and social media has not been required to make it so. Is adolescence really any different today than in any other era? The answer is yes… and no.

Adolescence was officially “discovered” in the United States in 1904 by G. Stanley Hall, and even then, Hall, the first doctor of psychology in the country, located the challenges of youth in the rapidly changing context of “our urbanized hothouse life that tends to ripen everything before its time.” While writing about teenagers in 1904, he observed that adolescence is a period of extreme orientation toward the peer group: “Some seem for a time to have no resource in themselves, but to be abjectly dependent for their happiness upon their mates.” He also pointed to the potential for cruelty within the peer group, a phenomenon that contemporary psychologists refer to as “relational aggression.” Decades later, the central challenge of adolescence was famously characterized as “identity formation” by the developmental psychologist Erik Erikson, who did much to explain twentieth-century adolescence.

Today, most psychologists agree that our longer lives combined with the challenges of an information-intensive society have further lengthened the time between childhood and adulthood. Many have settled on the notion of “emerging adulthood” to denote the years between eighteen and the late twenties as a new life stage: emerging adulthood is to the twenty-first century what adolescence was to the twentieth.

As developmental psychologist Robert Kegan described the adolescent experience long before the advent of Facebook, "There is no self independent of the context of 'other people liking.'" This is not a moral or emotional shortcoming but a fact of life in this developmental moment, and it entails certain predictable consequences. For example, one tends to operate through social comparison. One can be easy prey to manipulation, with few defenses against social pressure and other forms of social influence. The fixed belief system of an established group can all too easily fill the inner void, substituting an externally sourced identity for the work of self-construction.

Moving on from "fusion" means a transition from being someone who is their relationships to someone who has their relationships. It entails a deep reconfiguration of how we make sense of our experience. In Kegan's language, this means a shift away from a "culture of mutuality" to a more complex "culture of identity, self-authorship, and personal autonomy." This shift depends upon encountering people and life experiences that demand something more than our reflection in the mirror. It requires individuals and situations that insist on our first-person voice, provoking us to carve out our own unique response to the world.

This is an inner act that eludes rendition or datafication as we begin to compose an inward sense of valid truth and moral authority. This is the reference point from which we can say, "I think," "I feel," "I believe." Gradually, this "I" learns to feel authorship and ownership of its experiences. It can reflect on itself, know itself, and regulate itself with intentional choices and purposive action. Research shows that these big leaps in self-construction are stimulated by experiences such as structured reflection, conflict, dissonance, crisis, and failure. The people who help trigger this new inward connection refuse to act as our mirrors. They reject fusion in favor of genuine reciprocity. "Who comes into a person's life," Kegan observes, "may be the single greatest factor to influence what that life becomes."

Facebook relies on specific practices that feed the inclinations of people, especially young people, to know themselves from “the outside looking in.” Most critical is that the more the need for the “others” is fed, the less able one is to engage the work of self-construction. So devastating is the failure to attain that positive equilibrium between inner and outer life that Lapsley and Woodbury say it is “at the heart” of most adult personality disorders.

Nodder highlights Facebook’s precocious mastery of “social proof”: “Much of our behavior is determined by our impressions of what is the correct thing to do… based on what we observe others doing.… This influence is known as social proof.” The company instrumentalizes this aspect of adolescent nature by using messages from “friends” to make a product, service, or activity feel “more personal and emotional.” This ubiquitous tactic, much admired by Pentland, was used in the Facebook voting experiment.

Facebook’s single most momentous innovation in behavioral engineering is the now equally ubiquitous “Like” button, adopted in 2009. According to contemporaneous blog posts by longtime Facebook executive Andrew Bosworth, the “Like” button had been debated internally for more than a year and a half before Zuckerberg’s final decision to incorporate it. He had rejected the idea more than once, fearing that it would detract from other features intended to lift monetization, such as the controversial Beacon program. Significantly, the founder embraced the button only when new data revealed it as a powerful source of behavioral surplus that helped to ratchet up the magnetism of the Facebook News Feed, as measured by the volume of comments.

On the demand side, Facebook’s “likes” were quickly coveted and craved, morphing into a universal reward system or what one young app designer called “our generation’s crack cocaine.” “Likes” became those variably timed dopamine shots, driving users to double down on their bets “every time they shared a photo, web link, or status update. A post with zero likes wasn’t just privately painful, but also a kind of public condemnation.” In fact, most users craved the reward more than they feared humiliation, and the “Like” button became Facebook’s signature, spreading across the digital universe and actively fusing users in a new kind of mutual dependency expressed in a pastel orgy of giving and receiving reinforcement.

The “Like” button was only the start of what was to be an historic construction of a new social world that for many users is defined by fusion with the social mirror, especially among the young. Just as gamblers chase the zone of fusion with the machine, a young person embedded in the culture of mutuality chases the zone of fusion with the social mirror. For anyone already struggling with the challenge of the self-other balance, the “Like” button and its brethren continuously tip the scales toward regression.

When News Feed was first launched in 2006, it transformed Facebook from a site where users had to visit friends’ pages to see their updates to having those messages automatically shared in a stream on each person’s home page. Hundreds of thousands of users joined opposition groups, repelled by the company’s unilateral invasion of privacy. “No one was prepared for their online activity to suddenly be fodder for mass consumption,” recalled the tech news site TechCrunch on News Feed’s tenth anniversary in 2016, as it offered readers “The Ultimate Guide to the News Feed,” with instructions on “how you can get your content seen by more people,” how to appear “prominently,” and how to resonate with your “audience.”

Playing to the fear of invisibility and abandonment worked in 2006, when Facebook had just 9.5 million users (and required a college e-mail address to sign up), and it has driven the acceptance of every subsequent tweak to News Feed as Facebook has amassed more than 2 billion users. News Feed grew to become the “epicenter” of the corporation’s revenue success and “the most valuable billboard on Earth,” as Time magazine stated in 2015, just three years after Facebook’s IPO.

Facebook has Pentland’s prized God view on its side, an unparalleled resource that is drawn upon to remake this naturally longed-for fusion into a space of no escape. Science and capital are united in this long-game project. Yesterday it was the “Like” button, today it is augmented reality, and tomorrow there will be new innovations added to this repertoire. The company’s growth in user engagement, surplus capture, and revenue are evidence that these innovations have hit their marks.

But what happens when the forces of surveillance capital turn the mountain into a mountain range? Look at us! Yes, you are alive! Do not look away! Why would you? How could you? Today, we might "like" you!

According to the 302 most significant quantitative research studies on the relationships between social media use and mental health (most of them produced since 2013), the psychological process that most defines the Facebook experience is what psychologists call "social comparison." It is usually considered a natural and virtually automatic process that operates outside of awareness, "effectively forced upon the individual by his social environment" as we apply evaluative criteria tacitly internalized from our society, community, group, family, and friends. As one research review summarizes, "Almost at the moment of exposure, an initial holistic assessment of the similarity between the target and the self is made." As we go through life being exposed to other people, we naturally compare ourselves along the lines of similarity and contrast—I am like you. I am different from you—subliminal perceptions that translate into judgments—I am better than you. You are better than I.

For most of human history, people lived in small enclaves and were typically surrounded by others very much like themselves. Social comparisons with little variation are unlikely to entail great psychological risk. Research suggests that the diffusion of television in the second half of the twentieth century dramatically increased the intensity and negativity of social comparison, as it brought vivid evidence of varied and more-affluent lives dramatically different from one's own. One study found an increase in criminal larceny as television diffused across society, awakening an awareness of and desire for consumer goods. A related issue was that increased exposure to television programs depicting affluence led to "the overestimation of others' wealth and more dissatisfaction with one's own life."

Social media marks a new era in the intensity, density, and pervasiveness of social comparison processes, especially for the youngest among us, who are "almost constantly online" at a time of life when one's own identity, voice, and moral agency are a work in progress. In fact, the psychological tsunami of social comparison triggered by the social media experience is considered unprecedented.

Unlike television, however, social media entails active self-presentation characterized by “profile inflation,” in which biographical information, photos, and updates are crafted to appear ever more marvelous in anticipation of the stakes for popularity, self-worth, and happiness. Profile inflation triggers more negative self-evaluation among individuals as people compare themselves to others, which then leads to more profile inflation, especially among larger networks that include more “distant friends.” As one study concluded, “Expanding one’s social network by adding a number of distant friends through Facebook may be detrimental by stimulating negative emotions for users.”

One consequence of the new density of social comparison triggers and their negative feedback loops is a psychological condition known as FOMO (“fear of missing out”). It is a form of social anxiety defined as “the uneasy and sometimes all-consuming feeling that… your peers are doing, in the know about, or in possession of more or something better than you.” It’s a young person’s affliction that is associated with negative mood and low levels of life satisfaction. Research has identified FOMO with compulsive Facebook use: FOMO sufferers obsessively checked their Facebook feeds—during meals, while driving, immediately upon waking or before sleeping, and so on. This compulsive behavior is intended to produce relief in the form of social reassurance, but it predictably breeds more anxiety and more searching.

A 2011 study found that social media users exposed to pictures of “beautiful users” developed a more negative self-image than those who were shown less attractive profile pictures. Men who were shown profiles of high-career-status men judged their own pursuits as inadequate, compared to others who saw profiles of less successful men. By 2013, researchers found that Facebook use could predict negative shifts in both how their young subjects felt moment to moment as well as their overall life satisfaction. That year, German researchers found that the “astounding… wealth of social information” presented on Facebook produces “a basis for social comparison and envy on an unprecedented scale.” Their work demonstrated that “passive following” on Facebook exacerbates feelings of envy and decreases life satisfaction. More than 20 percent of all recent experiences of envy reported by the students in the research study had been triggered by Facebook exposure.

A three-phase investigation in 2014 found that spending a lot of time browsing profiles on Facebook produced a negative mood immediately afterward. Then, upon reflection, those users felt worse, reckoning that they had wasted their time. Instead of walking away, they typically chose to spend even more time browsing the network in the hope of feeling better, chasing the dream of a sudden and magical reversal of fortune that would justify past suffering. This cycle not only leads to more social comparison and more envy, but it can also predict depressive symptoms.

First we present ourselves as data objects for inspection, and then we experience ourselves as the “it” that others see. One 2014 study demonstrated the deleterious effect of these loops on body consciousness. An analysis of young men and women who had used Facebook for at least six years concluded that, regardless of gender, more Facebook participation leads to more body surveillance. A sense of self-worth comes to depend on physical appearance and being perceived as a sex object.

Life in the hive favors those who most naturally orient toward external cues rather than toward one’s own thoughts, feelings, values, and sense of personal identity. When considered from the vantage point of the self-other balance, positive social comparisons are just as pernicious as negative comparisons. Both are substitutes for the “hard bargain” of carving out a self that is capable of reciprocity rather than fusion. Whether the needle moves up or down, social comparison is the flywheel that powers the closed loop between the inclination toward the social mirror and its reinforcement. Both ego gratification and ego injury drive the chase for more external cues.

"Liking others' content and clicking links to posts by friends," the researchers summarized, "were consistently related to compromised well-being, whereas the number of status updates was related to reports of diminished mental health." So strong was this relationship that "a 1-standard-deviation increase in 'likes clicked'… 'links clicked'… or 'status updates' was associated with a decrease of 5%–8% of a standard deviation in self-reported mental health," even controlling for a person's initial state of well-being. The researchers' definitive conclusion? "Facebook use does not promote well-being.… Individual social media users might do well to curtail their use of social media and focus instead on real-world relationships."

This is not a rehearsal. This is the show. Facebook is a prototype of instrumentarian society, not a prophecy. It is the first frontier of a new societal territory, and the youngest among us are its vanguard. The frontier experience is an epidemic of the viewpoint of the Other-One, a hyper-objectification of one's own personhood shaped by the relentless amplification of life lived from the "outside looking in." The consequence is a pattern of overwhelming anxiety and disorientation in the simple act of digital disconnection, while connection itself is haunted by fresh anxieties that paradoxically leave too many feeling isolated, diminished, and depressed.

But this time it is not a question of simply packing their lunch and crossing our fingers as they head into the school-day maze of adolescent cliques, or sending them off to college knowing that they may stumble or fall but eventually find their passions and their people as they find themselves. This time, we have sent them into the raw heart of a rogue capitalism that amassed its fortune and power through behavioral dispossession parlayed into behavior modification in the service of others' guaranteed outcomes.

Innocent hangouts and conversations are embedded in a behavioral engineering project of planetary scope and ambition that is institutionalized in Big Other’s architectures of ubiquitous monitoring, analysis, and control. In their encounter with the self-other balance, teenagers step onto a playing field already tilted by surveillance capital to tip them into the social mirror and keep them fixed on its reflections. Everything depends upon feeding the algorithms that can effectively and precisely bite on him and bite on her and not let go. All those outlays of genius and money are devoted to this one goal of keeping users, especially young users, plastered to the social mirror like bugs on the windshield.

The research studies and first-person accounts that we have reviewed reveal the coercive underbelly of the instrumentarian’s much revered “confluence,” in which harmonies are achieved at the expense of the psychological integrity of participants. This is the world of Pentland’s “social learning,” his theory of “tuning” little more than the systematic manipulation of the rewards and punishments of inclusion and exclusion. It succeeds through the natural human inclination to avoid psychological pain. Just as ordinary consumers can become compulsive gamblers at the hands of the gaming industry’s behavioral technologies, psychologically ordinary young people are drawn into an unprecedented vortex of social information that automatically triggers social comparison on an equally unprecedented scale. This mental and emotional milieu appears to produce a virus of insecurity and anxiety that drives a young person deeper into this closed loop of escalating compulsion as he or she chases relief in longed-for signals of valorization.

This cycle unnaturally exacerbates and intensifies the natural orientation toward the group. And although we all share in this disposition to varying degrees, it is most pronounced in the stages of life that we call adolescence and emerging adulthood. Ethologists call this orientation “homing to the herd,” an adaptation of certain species, such as passenger pigeons and herring, that home to the crowd rather than to a particular territory. In the confrontation with human predators, however, this instinct has proven fatal.

For example, biologist Bernd Heinrich describes the fate of the passenger pigeons, whose “social sense was so strong that it drew the new predator, technologically equipped humans, from afar. It made them not only easy targets, but easily duped.” Commercial harvesters followed the pigeons’ flight and nesting patterns, and then used huge nets to catch thousands of pigeons at a time, shipping millions by rail each year to the markets from St. Louis to Boston. The harvesters used a specific technique, designed to exploit the extraordinary bonds of empathy among the birds and immortalized in the term “stool pigeon.” A few birds would be captured first and attached to a perch with their eyes sewn shut. As these birds fluttered in panic, the flock would descend to “attend to them.” This made it easy for the harvesters to “catch and slaughter” thousands at once. The last passenger pigeon died in the Cincinnati Zoo in 1914: “The pigeon had no home boundaries over which to spread itself and continued to orient only to itself, so it could be everywhere, even to the end.… To the pigeons, the only ‘home’ they knew was in the crowd, and now they had become victims of it… the lack of territorial boundaries of human predators had tipped the scales to make their adaptation their doom.”

This artificial intensification of homing to the herd can only complicate, delay, or impede the hard psychological bargain of the self-other balance. When we multiply this effect by hundreds of millions and distribute it across the globe, what might it portend for the prospects of human and societal development?

Facebook is the crucible of this new dark science. It aims to perfect the relentless stimulation of social comparison in which natural empathy is manipulated and instrumentalized to modify behavior toward others’ ends. This synthetic hive is a devilish pact for a young person. In terms of sheer everyday effectiveness—contact, logistics, transactions, communications—turn away, and you are lost. And if you simply crave the fusion juice that is proof of life at a certain age and stage—turn away, and you are extinguished.

It is a new phenomenon to live continuously in the milieu of the gaze of others, to be followed by hundreds or thousands of eyes, augmented by Big Other’s devices, sensors, beams, and waves rendering, recording, analyzing, and actuating. The unceasing pace, density, and volume of the gaze deliver a perpetual stream of evaluative metrics that raise or lower one’s social currency with each click. In China, these rankings are public territory, shiny badges of honor and scarlet letters that open or shut every door. In the West, we have “likes,” “friends,” “followers,” and hundreds of other secret rankings that invisibly pattern our lives.

Facebook entered the world bypassing old institutional boundaries, offering us freedom to connect and express ourselves at will. It is impossible to say what the Facebook experience might have been had the company chosen a path that did not depend upon surveillance revenues. Instead, we confront the sudden accretion of an instrumentarian power that spins our society in an unanticipated direction. Facebook's applied utopistics are a prototype of an instrumentarian future, showcasing feats of behavioral engineering that groom populations for the rigors of instrumentarianism's coercive harmonies.

Its operations are designed to exploit the human inclination toward empathy, belonging, and acceptance. The system tunes the pitch of our behavior with the rewards and punishments of social pressure, herding the human heart toward confluence as a means to others' commercial ends.

From this vantage point, we see that the full scope of the Facebook operation constitutes a vast experiment in behavior modification designed not only to test the specific capabilities of its tuning mechanisms, as in its official "large-scale experiments," but also to do so on the broadest possible social and psychological canvas. Most significantly, the applied utopistics of social pressure, its flywheel of social comparison, and the closed loops that bind each user to the group system vividly confirm Pentland's theoretical rendering of the case. Instrumentarian social principles are evident here, not as hypotheses but as facts, the facts that currently constitute the spaces where our children are meant to "grow up."

What we witness here is a bet-the-farm commitment to the socialization and normalization of instrumentarian power for the sake of surveillance revenues. Just as Pentland stipulated, these closed loops are imposed outside the realm of politics and individual volition. They move in stealth, crafting their effects at the level of automatic psychological responses and tipping the self-other balance toward the pseudo-harmonies of the hive mind. In this process, the inwardness that is the necessary source of autonomous action and moral judgment suffers and suffocates. These are the preparatory steps toward the death of individuality that Pentland advocates.

What we have seen in Facebook is a living example of the third modernity that instrumentarianism proffers, defined by a new collectivism owned and operated by surveillance capital. The God view drives the computations. The computations enable tuning. Tuning replaces private governance and public politics, without which individuality is merely vestigial. And just as the uncontract bypasses social mistrust rather than healing it, the post-political societal processes that bind the hive rely on social comparison and social pressure for their durability and predictive certainty, eliminating the need for trust. Rights to the future tense, their expression in the will to will, and their sanctification in promises are drawn into the fortress of surveillance capital.

The young people we have considered in this chapter are the spirits of a Christmas Yet to Come. They live on the frontier of a new form of power that declares the end of a human future, with its antique allegiances to individuals, democracy, and the human agency necessary for moral judgment. Should we awaken from distraction, resignation, and psychic numbing with Scrooge's determination, it is a future that we may still avert.

This is the meaning of the hand and glove. The closed loop and the tight fit are meant to create the conditions of no exit. Once, it was no exit from God's total knowledge and power. Today, it is no exit from the others, from Big Other, and from the surveillance capitalists who decide. This condition of no exit creeps on slippered feet. First we do not even have to look away, and later we cannot.

This was not intended as a statement of misanthropy but rather a recognition that the self-other balance can never be adequately struck as long as the "others" are constantly "watching." Another mid-century social psychologist, Erving Goffman, took up these themes in his immortal The Presentation of Self in Everyday Life. Goffman developed the idea of the "backstage" as the region in which the self retreats from the performative demands of social life.

The difference is that Goffman assumed a backstage where you could be your true self. For us, the backstage is shrinking. There is almost no place left where I can be my true self. Even when I am walking by myself, and I think I am backstage, something happens—an ad appears on my phone or someone takes a photo, and, I discover that I am onstage, and everything changes. … The “everything” that changes is the sudden cognizance, part realization and part reminder, that Big Other knows no boundaries. Experience is seamlessly rendered across the once-reliable borders of the virtual and real worlds. This accrues to the immediate benefit of surveillance capital—“Welcome to McDonald’s!” “Buy this jacket!”—but any worldly experience can just as immediately be delivered to the hive: a post here, a photo there. Ubiquitous connection means that the audience is never far, and this fact brings all the pressures of the hive into the world and the body.

Recent research has begun to turn to this dour fact that a team of British researchers describes as the “extended chilling effect.” The idea here is that people—especially, though not exclusively, young people—now censor and curate their real-world behavior in consideration of their own online networks as well as the larger prospect of the internet masses. The researchers conclude that participation in social media “is profoundly intertwined with the knowledge that information about our offline activities may be communicated online, and that the thought of displeasing ‘imagined audiences’ alters our ‘real-life’ behavior.”

When I catch myself wanting to cheer the students who are anguished by connection and terrified of its loss, I consider the meaning of “no exit” as recounted in a personal recollection of the social psychologist Stanley Milgram regarding an experiment that demonstrated “the power of immediate circumstances on feelings and behavior.” … Milgram's class was studying the force with which social norms control behavior. He had the idea of examining the real-life phenomenon by having his students approach a person on the subway and, without providing any justification, simply look the person in the eye and ask for his or her seat. One afternoon, Milgram himself boarded the subway ready to make his contribution. Despite his years of observing and theorizing disturbing patterns of human behavior, it turned out that he was unprepared for his own moment of social confrontation. Assuming that it would be an "easy" caper, Milgram approached a passenger and was about to utter the "magical phrase" when "the words seemed lodged in my trachea and would simply not emerge. I stood there frozen, then retreated... I was overwhelmed by paralyzing inhibition." The psychologist eventually hectored himself into trying again. He recounts what occurred when he finally approached a passenger and "choked out" his request: "Excuse me, sir, may I have your seat?"

“Excuse me, sir, may I have your seat?” A moment of stark anomic panic overcame me. But the man got right up and gave me the seat.… Taking the man’s seat, I was overwhelmed by the need to behave in a way that would justify my request. My head sank between my knees, and I could feel my face blanching. I was not role-playing. I actually felt as if I were going to perish. … Moments later the train pulled into the next station, and Milgram exited. He was surprised to discover that as soon as he left the train, “all the tension disappeared.” Milgram left the subway, where he vibrated in tune with the “others,” and that exit enabled a return to his “self.”

Unlike Milgram, we face an intolerable situation. Like the gamblers in their machine wombs, we are meant to fuse with the system and play to extinction: not the extinction of our funds but rather the extinction of our selves. Extinction is a design feature formalized in the conditions of no exit. The aim of the tuners is to contain us within “the power of immediate circumstances” as we are compelled by the “logic of social relations” in the hive to bow to social pressure exerted in calculated patterns that exploit our natural empathy. Continuously tightening feedback loops cut off the means of exit, creating impossible levels of anxiety that further drive the loops toward confluence. What is to be killed here is the inner impulse toward autonomy and the arduous, exciting elaboration of the autonomous self as a source of moral judgment and authority capable of asking for a subway seat or standing against rogue power.

Inside the hive, it is easy to forget that every exit is an entrance. To exit the hive means to enter that territory beyond, where one finds refuge from the artificially tuned-up social pressure of the others. Exit leaves behind the point of view of the Other-One in favor of entering a space in which one’s gaze can finally settle inward. To exit means to enter the place where a self can be birthed and nurtured. History has a name for that kind of place: sanctuary.

Refuge and prospect are opposites: refuge is small and dark; prospect is expansive and bright.… We need them both and we need them together.

The old house, for those who know how to listen, is a sort of geometry of echoes. The voices of the past do not sound the same in the big room as in the little bed chamber.… Among the most difficult memories, well beyond any geometry that can be drawn, we must recapture the quality of the light; then come the sweet smells that linger in the empty rooms.

Home is our school of intimacy, where we first learn to be human. Its corners and nooks conceal the sweetness of solitude; its rooms frame our experience of relationship. Its shelter, stability, and security work to concentrate our unique inner sense of self, an identity that imbues our day dreams and night dreams forever. Its hiding places—closets, chests, drawers, locks, and keys—satisfy our need for mystery and independence. Doors—locked, closed, half shut, wide open—trigger our sense of wonder, safety, possibility, and adventure. Bachelard plumbs not only the imagery of the human house but also of nests and shells, the “primal images” of home that convey the absolute “primitiveness” of the need for a safe refuge: “Well-being takes us back to the primitiveness of the refuge. Physically, the creature endowed with a sense of refuge huddles up to itself, takes to cover, hides away, lies snug, concealed… a human being likes to ‘withdraw into his corner’… it gives him physical pleasure to do so.”

The shelter of home is our original way of living in space, Bachelard discovers, shaping not only the existential counterpoint of “home” and “away” but also many of our most fundamental ways of making sense of experience: house and universe, refuge and world, inside and outside, concrete and abstract, being and nonbeing, this and that, here and elsewhere, narrow and vast, depth and immensity, private and public, intimate and distant, self and other.

That our walls are dense and deep is of no importance now because the boundaries that define the very experience of home are to be erased. There can be no corners in which to curl up and taste the pleasures of solitary inwardness. There can be no secret hiding places because there can be no secrets. Big Other swallows refuge whole, along with the categories of understanding that originate in its elemental oppositions: house and universe, depth and immensity. Those ageless polarities in which we discover and elaborate our sense of self are casually eviscerated as immensity installs itself in my refrigerator, the world chatters in my toothbrush, elsewhere stands watch over my bloodstream, and the garden breeze stirs the chimes draped from the willow tree only to be broadcast across the planet. The locks? They have vanished. The doors? They are open.

In the march of institutional interests intent on implementing Big Other, the very first citadel to fall is the most ancient: the principle of sanctuary. The sanctuary privilege has stood as an antidote to power since the beginning of the human story. Even in ancient societies where tyranny prevailed, the right of sanctuary stood as a fail-safe. There was an exit from totalizing power, and that exit was the entrance to a sanctuary in the form of a city, a community, or a temple. By the time of the Greeks, sanctuaries were sacred sites built across the ancient Greek world and consecrated to the purposes of asylum and religious sacrifice. The Greek word asylon means “unplunderable” and founds the notion of a sanctuary as an inviolable space. The right of asylum survived into the eighteenth century in many parts of Europe, attached to holy sites, churches, and monasteries. The demise of the sanctuary privilege was not a repudiation but rather a reflection of social evolution and the firm establishment of the rule of law. One historian summarized this transformation: “justice as sanctuary.”

Those who would eviscerate sanctuary are keen to take the offensive, putting us off guard with the guilt-inducing question “What have you got to hide?” But as we have seen, the crucial developmental challenges of the self-other balance cannot be negotiated adequately without the sanctity of “disconnected” time and space for the ripening of inward awareness and the possibility of reflexivity: reflection on and by oneself. The real psychological truth is this: If you’ve got nothing to hide, you are nothing.

In “Psychological Functions of Privacy,” Darhl Pedersen defines privacy as a “boundary control process” that invokes the decision rights associated with “restricting and seeking interaction.” Pedersen’s research identifies six categories of privacy behaviors: solitude, isolation, anonymity, reserve, intimacy with friends, and intimacy with family. His study shows that these varied behaviors accomplish a rich array of complex psychological “privacy functions” considered salient for psychological health and developmental success: contemplation, autonomy, rejuvenation, confiding, freedom, creativity, recovery, catharsis, and concealment. These are experiences without which we can neither flourish nor usefully contribute to our families, communities, and society.

It’s the wall, not the tunnels, that requires our attention. So far US privacy laws have failed to keep pace with the march of instrumentarianism.

As a 2016 article on the “internet of things” by Fourth Amendment scholar Andrew Guthrie Ferguson concludes, “If billions of sensors filled with personal data fall outside of Fourth Amendment protections, a large-scale surveillance network will exist without constitutional limits.” As we have seen, it already does. Dutch scholars make a similar case for the inadequacy of Dutch law as it falls behind Big Other, no longer able to effectively assert the sanctity of the home from the invasive action of either industry or the state: “The walls no longer shield the individual effectively from the outside in the pursuing of… personal life without intrusion.…”

Many hopes today are pinned on the new body of EU regulation known as the General Data Protection Regulation (GDPR), which became enforceable in May 2018. The EU approach fundamentally differs from that of the US in that companies must justify their data activities within the GDPR’s regulatory framework. The regulations introduce several key new substantive and procedural features, including a requirement to notify people when personal data is breached, a high threshold for the definition of “consent” that puts limits on a company’s reliance on this tactic to approve personal data use, a prohibition on making personal information public by default, a requirement to use privacy by design when building systems, a right to erasure of data, and expanded protections against decision making authored by automated systems that imposes “consequential” effects on a person’s life. The new regulatory framework also imposes substantial fines for violations, which will rise to a possible 4 percent of a company’s global revenue, and it allows for class-action lawsuits in which users can combine to assert their rights to privacy and data protection.

These are vital and necessary accomplishments, and the question that is most important to our story is whether this new regulatory regime can be a springboard to challenging the legitimacy of surveillance capitalism and ultimately vanquishing its instrumentarian power. In time, the world will learn if the GDPR can move out in front of Big Other, reasserting a division of learning aligned with the values and aspirations of a democratic society. Such a victory would depend upon society’s rejection of markets based on the dispossession of human experience as a means to the prediction and control of human behavior for others’ profit.

In March 2018, fifteen months after his initial request, Dehaye finally received an e-mail from Facebook's Privacy Operations Team. He was told that the information he sought "is not available through our self-service tools" but is stored in "Hive," Facebook's "log storage area," where it is retained for "data analytics" and maintained as separate from "the data bases that power the Facebook site." The company insisted that accessing the data required it to surmount "huge technical challenges." "This data," the company writes, "is also not used to directly serve the live Facebook website which users experience."

Facebook makes clear that the shadow text is not available to users, despite the promotion of its self-service download tools that promise to give users access to their personal data retained by the company. Indeed, the competitive dynamics of surveillance capitalism make the shadow text a crucial proprietary source of advantage. Any attempt to breach its content will be experienced as an existential threat; no surveillance capitalist will voluntarily provide data from the shadow text. Only law can compel this challenge to the pathological division of learning.

In the wake of the Cambridge Analytica scandal in March 2018, Facebook announced it would expand the range of personal data that it allows users to download, but even these data remain wholly contained within the first text, composed largely of the information that users themselves have provided, including information that they have deleted: friends, photos, video, ads that have been clicked, pokes, posts, location, and so on. These data do not include behavioral surplus, prediction products, and the fate of those predictions as they are used for behavioral modification, bought, and sold. When you download your “personal information,” you access the stage, not the backstage: the curtain, not the wizard.

Facebook’s response to Dehaye illustrates another consequence of the extreme asymmetries of knowledge at play. The company insisted that access to the requested data required it to surmount “huge technical challenges.” As behavioral surplus flows converge in machine-learning–based manufacturing operations, the sheer volume of data inputs and methods of analysis moves beyond human comprehension. Consider something as trivial as the case of Instagram’s machines selecting what images to show you. Its computations are based on varied streams of behavioral surplus from a subject user, then more streams from the friends in that user’s network, then more from the activities of people who follow the same accounts as the subject user, then the data and social links from the user’s Facebook activity. When it finally applies a ranking logic to predict what images the user will want to see next, that analysis also includes data on the past behavior of the subject user. Instagram has machines doing this “learning” because humans cannot.

This recalls our discussion of Facebook’s “prediction engine” FBLearner Flow, where the machines are fed tens of thousands of data points derived from behavioral surplus, diminishing the very notion of the right to contest “automatic decision making.” If the algorithms are to be contestable in any meaningful way, it will require new countervailing authority and power, including machine resources and expertise to reach into the core disciplines of machine intelligence and construct new approaches that are available for inspection, debate, and combat. Indeed, one expert has already proposed the creation of a government agency—an “FDA for algorithms”—to oversee the development, distribution, sale, and use of complex algorithms, arguing that existing laws “will prove no match for the difficult regulatory puzzles algorithms pose.”

Dehaye is an activist, and his aim is not only to access data but also to document the arduousness and even absurdity of the undertaking. Given these realities, he suggests that the data-protection regulations are comparable to freedom-of-information laws. The procedures for requesting and receiving information under these laws are imperfect and onerous, typically undertaken by legal specialists, but nevertheless essential to democratic freedom. Although effective contest will require determined individuals, the individual alone cannot shoulder the burden of justice, any more than an individual worker in the first years of the twentieth century could bear the burden of fighting for fair wages and working conditions. Those twentieth-century challenges required collective action, and so do our own.

It is already possible to see a new awakening to empowering collective action, at least in the privacy domain. One example is None of Your Business (NOYB), a nonprofit organization led by privacy activist Max Schrems. After many years of legal contest, Schrems made history in 2015 when his challenge to Facebook’s data-collection and data-retention practices—which he asserted were in violation of EU privacy law—led the Court of Justice of the European Union to invalidate the Safe Harbor agreement that governed data transfers between the US and the EU. In 2018 Schrems launched NOYB as a vehicle for “professional privacy enforcement.”

This is at least one conclusion from the past decade: despite far more stringent privacy and data-protection laws in the EU as compared to the US, as well as a forceful commitment to antitrust, Facebook and Google have continued to flourish in Europe. For example, between 2010 and 2017 the compounded annual growth rate for Facebook daily active users was 15 percent in Europe compared to 9 percent in the US and Canada.

In practice, however, the company was making changes to ensure that the GDPR would not circumscribe the majority of its operations. Until then, 1.5 billion of its users, including those in Africa, Asia, Australia, and Latin America, were governed by terms of service issued by the company's international headquarters in Ireland, meaning that these terms fell under the EU framework. It was in late April that Facebook quietly issued new terms of service, placing those 1.5 billion users under US privacy laws and thus eliminating their ability to file claims in Irish courts.

What life is left to us if taming fails? Without protection from surveillance capitalism and its instrumentarian power—their behavioral aims and societal goals—we are trapped in a condition of "no exit," where the only walls are made of glass. The natural human yearning for refuge must be extinguished and the ancient institution of sanctuary deleted.

"No exit" is the necessary condition for Big Other to flourish, and its flourishing is the necessary condition for all that is meant to follow: the tides of behavioral surplus and their transformation into revenue, the certainty that will meet every market player with guaranteed outcomes, the bypass of trust in favor of the uncontract's radical indifference, the paradise of effortless connection that exploits the needs of harried second-modernity individuals and transforms their lives into the means to others' ends, the plundering of the self, the extinction of autonomous moral judgment for the sake of frictionless control, the actuation and modification that quietly drains the will to will, the forfeit of your voice in the first person in favor of others' plans, the destruction of the social relations and politics of the old and slow and still-unfulfilled ideals of self-determining citizens bound to the legitimate authority of democratic governance.

In the absence of synthetic declarations that secure the road to a human future, the intolerability of glass life turns us toward a societal arms race of counter-declarations in which we search for and embrace increasingly complex ways to hide in our own lives, seeking respite from lawless machines and their masters. We do this to satisfy our enduring need for sanctuary and as an act of resistance with which to reject the instrumentarian disciplines of the hive, its “extended chilling effects,” and Big Other’s relentless greed. In the context of government surveillance, the practices of “hiding” have been called “privacy protests” and are well-known for drawing the suspicion of law-enforcement agencies. Now, hiding is also invoked by Big Other and its market masters, whose reach is far and deep as they install themselves in our walls, our bodies, and on our streets, claiming our faces, our feelings, and our fears of exclusion.

I have suggested that too many of the best and brightest of a new generation devote their genius to the intensification of the click-stream. Equally more poignant is the way in which a new generation of activists, artists, and inventors feels itself called to create the art and science of hiding. The intolerable conditions of glass life compel these young artists to dedicate their genius to the prospects of human invisibility, even as their creations demand that we aggressively seek and find our bearings. Their provocations already take many forms: signal-blocking phone cases, false fingerprint prosthetics that prevent your fingertips from being “used as a key to your life,” LED privacy visors to impede facial-recognition cameras, a quilted coat that blocks radio waves and tracking devices, a scent diffuser that releases a metallic fragrance when an unprotected website or network is detected on any of your devices, a “serendipitor app” to disrupt any surveillance “that relies on subjects maintaining predictable routines,” a clothing line called “Glamouflage” featuring shirts covered with representations of celebrity faces to confuse facial-recognition software, anti-neuroimaging surveillance headgear to obstruct digital invasion of brain waves, and an anti-surveillance coat that creates a shield to block invasive signals. Chicago artist Leo Selvaggio produces 3-D–printed resin prosthetic masks to confound facial recognition. He calls his effort “an organized artistic intervention.”

Perhaps most poignant is the Backslash Tool Kit: “a series of functional devices designed for protests and riots of the future,” including a smart bandana for embedding hidden messages and public keys, independently networked wearable devices, personal black-box devices to register abuse of law enforcement, and fast deployment routers for off-grid communication. Backslash was created as part of a master’s thesis project at New York University, and it perfectly reflects the contest for the third modernity that this generation faces. The designer writes that for young, digitally native protesters, “connectivity is a basic human right.” Yet, he laments, “the future of technology in protests looks dark” because of overwhelming surveillance.

Take a casual stroll through the shop at the New Museum for Contemporary Art in Manhattan, and you pass a display of its bestseller: table-top mirrors whose reflecting surface is covered with the bright-orange message “Today’s Selfie Is Tomorrow’s Biometric Profile.” This “Think Privacy Selfie Mirror” is a project of the young Berlin-based artist Adam Harvey, whose work is aimed at the problem of surveillance and foiling the power of those who surveil. Harvey’s art begins with “reverse engineering… computer vision algorithms” in order to detect and exploit their vulnerabilities through camouflage and other forms of hiding.

Harvey is one among a growing number of artists, often young artists, who direct their work to the themes of surveillance and resistance. Artist Benjamin Grosser’s Facebook and Twitter “demetricators” are software interfaces that present each site’s pages with their metrics deleted: “The numbers of ‘likes,’ ‘friends,’ followers, retweets… all disappear.” How is an interface that foregrounds our friend count changing our conceptions of friendship? he asks. “Remove the numbers and find out.” Grosser’s “Go Rando” project is a web browser extension that “obfuscates your feelings on Facebook” by randomly choosing an emoji each time you click “Like,” thus undermining the corporation’s surplus analyses as they compute personality and emotional profiles. Trevor Paglen’s richly orchestrated performance art combines music, photography, satellite imagery, and artificial intelligence to reveal Big Other’s omnipresent knowing and doing. “It’s trying to look inside the software that is running an AI… to look into the architectures of different computer vision systems and trying to learn what it is that they are seeing,” Paglen says. Chinese artist Ai Weiwei’s 2017 installation “Hansel & Gretel” created a powerful experience in which participants viscerally confront the surveillance implications of their own innocent picture taking, Instagramming, tweeting, texting, tagging, and posting.

The greatest danger is that we come to feel at home in glass life or in the prospect of hiding from it. Both alternatives rob us of the life-sustaining inwardness, born in sanctuary, that finally distinguishes us from the machines. This is the well from which we draw the capacities to promise and to love, without which both the private bonds of intimacy and the public bonds of society wither and die. If we do not alter this course now, we leave a monumental work for the generations that follow us. Industrial capitalism commandeered nature only to saddle the coming generations with the burden of a burning planet. Will we add to this burden with surveillance capitalism’s invasion and conquest of human nature? Will we stand by as it subtly imposes the life of the hive while demanding the forfeit of sanctuary and the right to the future tense for the sake of its wealth and power?

Paradiso calls it a revolution, and Pentland says it is the death of individuality. Nadella and Schmidt advocate the machine hive as our role model, with its coercive confluence and preemptive harmonies. Page and Zuckerberg understand the transformation of society as a means to their commercial ends. There are dissenters among us, to be sure, but the declaration of life without walls has thus far failed to trigger a mass withdrawal of agreement. This is in part the result of our dependency and in part because we do not yet appreciate the breadth and depth of what the architects have in store, let alone the consequences that this “revolution” might entail.

Our sensibilities grow numb to the monstrosity of Big Other as its features are developed, tested, elaborated, and normalized. We become deaf to the lullaby of walls. Hiding from the machines and their masters drifts from the obsessions of the vanguard to a normal theme of social discourse and eventually our conversations around the dinner table. Each step down this path occurs as if in the fog of war: scattered fragments and incidents that appear abruptly and often in obscurity. There is little room to perceive the pattern, let alone its origins and meaning. Nonetheless, each deletion of the possibility of sanctuary leaves a void that is seamlessly and soundlessly filled by the new conditions of instrumentarian power.

The notion that ignorance and freedom are essential characteristics of capitalism is rooted in the conditions of life before the advent of modern systems of communication and transportation, let alone global digital networks, the internet, or the ubiquitous computational, sensate, actuating architectures of Big Other. Until the last few moments of the human story, life was necessarily local, and the "whole" was necessarily invisible to the "part."

Adam Smith's famous metaphor of the "invisible hand" drew on these enduring realities of human life. Each individual, Smith reasoned, employs his capital locally in pursuit of immediate comforts and necessities. Each one attends to "his own security... his own gain... led by an invisible hand to promote an end which was no part of his intention." That end is the efficient employ of capital in the broader market: the wealth of nations. The individual actions that produce efficient markets add up to a staggeringly complex pattern, a mystery that no one person or entity could hope to know or understand, let alone to direct: "The statesman, who should attempt to direct private people in what manner they ought to employ their capitals, would...

Something is awry. It is true that many capitalists, including surveillance capitalists, vigorously employ these centuries-old justifications for their freedom when they reject regulatory, legislative, judicial, societal, or any other form of public interference in their methods of operation. However, Big Other and the steady application of instrumentarian power challenge the classic quid pro quo of freedom for ignorance.

Recall Mark Zuckerberg’s boast that Facebook would know every book, film, and song a person had ever consumed and that its predictive models would tell you what bar to go to when you arrive in a strange city, where the bartender would have your favorite drink waiting. As the head of Facebook’s data science team once reflected, "This is the first time the world has seen this scale and quality of data about human communication.… For the first time, we have a microscope that… lets us examine social behavior at a very fine level that we’ve never been able to see before.…" A top Facebook engineer put it succinctly: "We are trying to map out the graph of everything in the world and how it relates to each other."

As Google’s Eric Schmidt observed in 2010, "You give us more information about you, about your friends, and we can improve the quality of our searches. We don’t need you to type at all. We know where you are. We know where you’ve been. We can more or less know what you’re thinking about." Satya Nadella of Microsoft understands all physical and institutional spaces, people, and social relationships as indexable and searchable: all of it subject to machine reasoning, pattern recognition, prediction, preemption, interruption, and modification.

Although there is nothing unusual about the prospect of capitalist enterprises seeking every kind of knowledge advantage in a competitive marketplace, the surveillance capitalist capabilities that translate ignorance into knowledge are unprecedented because they rely on the one resource that distinguishes the surveillance capitalists from traditional utopianists: the financial and intellectual capital that permits the actual transformation of the world, materialized in the continuously expanding architectures of Big Other. More astonishing still is that surveillance capital derives from the dispossession of human experience, operationalized in its unilateral and pervasive programs of rendition: our lives are scraped and sold to fund their freedom and our subjugation, their knowledge and our ignorance about what they know.

We have carefully examined surveillance capitalism's novel foundational mechanisms, economic imperatives, gathering power, and societal objectives. One conclusion of our investigations is that surveillance capitalism's command and control of the division of learning in society are the signature feature that breaks with the old justifications of the invisible hand and its entitlements. The combination of knowledge and freedom works to accelerate the asymmetry of power between surveillance capitalists and the societies in which they operate. This cycle will be broken only when we acknowledge as citizens, as societies, and indeed as a civilization that surveillance capitalists know too much to qualify for freedom.

In another decisive break with capitalism’s past, surveillance capitalists abandon the organic reciprocities with people that have long been a mark of capitalism’s endurance and adaptability. Symbolized in the twentieth century by Ford’s five-dollar day, these reciprocities hearken back to Adam Smith’s original insights into the productive social relations of capitalism, in which firms rely on people as employees and customers. Smith argued that price increases had to be balanced with wage increases “so that the labourer may still be able to purchase that quantity of those necessary articles which the state of the demand for labour… requires that he should have.” The shareholder-value movement and globalization went a long way toward destroying this centuries-old social contract between capitalism and its communities, substituting formal indifference for reciprocity. Surveillance capitalism goes further. It not only jettisons Smith, but it also formally rescinds any remaining reciprocities with its societies.

Second, by historical standards the large surveillance capitalists employ relatively few people compared to their unprecedented computational resources. This pattern, in which a small, highly educated workforce leverages the power of a massive capital-intensive infrastructure, is called “hyperscale.”

From the time they went public to 2016, Google and Facebook steadily climbed to the heights of market capitalization, with Google reaching $532 billion by the end of 2016 and Facebook reaching $332 billion, without Google ever employing more than 75,000 people or Facebook more than 18,000. General Motors took four decades to reach its highest market capitalization of $225.15 billion in 1965, when it employed 735,000 women and men. Most startling is that GM employed more people during the height of the Great Depression than either Google or Facebook employs at their heights of market capitalization.

The GM pattern is the iconic story of the United States in the twentieth century, before globalization, neoliberalism, the shareholder-value movement, and plutocracy unraveled the public corporation and the institutions of the double movement. Those institutions rationalized GM's employment policies with fair labor practices, unionization, and collective bargaining, emblematic of stable reciprocities during the pre-globalization decades of the twentieth century.

Hyperscale firms have become emblematic of modern digital capitalism, and as capitalist inventions they present significant social and economic challenges, including their impact on employment and wages, industry concentration, and monopoly. In 2017, 24 hyperscale firms operated 320 data centers with anywhere between thousands and millions of servers (Google and Facebook were among the largest).

Not all hyperscale firms are surveillance capitalists, however, and our focus here is restricted to the convergence of these two domains. The surveillance capitalists that operate at hyperscale or outsource to hyperscale operations dramatically diminish any reliance on their societies as sources of employees, and the few for whom they do compete, as we have seen, are drawn from the most-rarefied strata of data science.

The absence of organic reciprocities with people as either sources of consumers or employees is a matter of exceptional importance in light of the historical relationship between market capitalism and democracy. In fact, the origins of democracy in both America and Britain have been traced to these very reciprocities. In America the violation of consumer reciprocities awakened an unstoppable march toward liberty as economic power translated into political power. A half century later in Britain, a grudging, practical, self-interested respect for the necessary interdependence of capital and labor translated into new patterns of political power, expressed in the gradual expansion of the franchise and the nonviolent shift to more-inclusive democratic institutions. Even a brief glance at these world-altering histories can help us grasp the degree to which surveillance capitalism diverges from capitalism's past.

The American Revolution is the outstanding example of how the reciprocities of consumption contributed to the rise of democracy. Historian T. H. Breen argues in his pathbreaking study The Marketplace of Revolution that it was the violation of these reciprocities that set the Revolution into motion, uniting disparate provincial strangers into a radical new patriotic force. Breen explains that American colonists had come to depend upon the “empire of goods” imported from England and that this dependency instilled the sense of a reciprocal social contract: “For ordinary people, the palpable experience of participating in an expanding Anglo-American consumer market” intensified their sense of a “genuine partnership” with England. Eventually, the British Parliament famously misjudged the rights and obligations of this partnership, imposing a series of taxes that turned imported goods such as cloth and tea into “symbols of imperial oppression.” Breen describes the originality of a political movement born in the shared experience of consumption, the outrage at the violation of essential producer-consumer interdependencies, and the determination to make “goods speak to power.” … The translation of consumer expectations into democratic revolution occurred in three waves, beginning in 1765, when the Stamp Act triggered popular protests, riots, and organized resistance finally expressed in the “nonimportation movement” (today we would call it a consumer boycott). As Breen tells it, the details of the Stamp Act were less important than the colonists’ realization that England did not perceive them as political or economic equals bound in mutually beneficial reciprocities: “By compromising the Americans’ ability to purchase the goods they desired, Parliament had revealed an intention to treat the colonists like second-class subjects,” levying a heavy price “on the pursuit of material happiness.” The Stamp Act was experienced as a violation of the colonists’ rights not only as subjects of the empire but also as consumers of the empire: it was the first translation of consumers’ economic power into political power, a “radically new form of politics” in which the most ordinary members of colonial society experienced “an exhilarating surge of empowerment.” … When the Townshend Acts were passed just two years later, in 1767, this time imposing taxes on a range of imported goods, a new wave of outrage mobilized people in every colony. Detailed nonimportation agreements turned consumer sacrifice into the front line of political resistance. The shared experience of violated expectations cut across regional, religious, and cultural differences, providing a new basis for social solidarity. … The 1773 Tea Act plunged the colonies into a new phase of resistance that shifted the political focus from nonimportation, which depended upon merchants holding the line, to nonconsumption, which demanded the participation of all individuals in the unique solidarity of their shared status as “customers.” It was in this context that Samuel Adams proclaimed that the cause of liberty “depended on the ability of the American people to free themselves from ‘the Baubles of Britain.’”

The Stamp Act was experienced as a violation of the colonists’ rights not only as subjects of the empire but also as consumers of the empire: it was the first translation of consumers’ economic power into political power, a “radically new form of politics” in which the most ordinary members of colonial society experienced “an exhilarating surge of empowerment.”

A year later, in 1774, the First Continental Congress convened in Philadelphia and produced a "grand scheme" to abolish trade with England. "It brought to fruition a brilliantly original strategy of consumer resistance to political oppression," Breen writes, "one that had invited Americans to think of themselves as Americans even before they entertained a thought of independence."

In early-nineteenth-century Britain, as Daron Acemoglu and James A. Robinson have shown, the rise of democracy was inextricably bound to industrial capitalism's dependency on the "masses" and their contribution to the prosperity necessitated by the new organization of production. The rise of volume production and its wage-earning labor force established British workers' economic power and led to a growing appreciation of their political legitimacy and power. This produced a new sense of interdependence between ordinary people and elites. … In sharp contrast to the pragmatic concessions of Britain's early industrial capitalists, surveillance capitalists' extreme structural independence from people breeds exclusion rather than inclusion and lays the foundation for the unique approach that we have called "radical indifference."

The accumulation of freedom and knowledge combines with the lack of organic reciprocities with people to shape a third unusual feature of surveillance capitalism: a collectivist orientation that diverges from the long-standing values of market capitalism and market democracy, while also sharply departing from surveillance capitalism's origins in the neoliberal worldview. For the sake of its own commercial success, surveillance capitalism aims us toward the hive collective. This privatized instrumentarian social order is a new form of collectivism in which it is the market, not the state, which concentrates both knowledge and freedom within its domain.

This collectivist orientation is an unexpected development in light of surveillance capitalism's origins in a neoliberal creed conceived sixty years ago as a reaction to the collectivist totalitarian nightmares of the mid-twentieth century. Later, with the demise of the fascist and socialist threats, neoliberal ideology cunningly succeeded in redefining the modern democratic state as a fresh source of collectivism to be resisted by any and all means. Indeed, the evisceration of the double movement has been prosecuted in the name of defeating the supposed collectivist hazards of "too much democracy." Now the hive emulates the "termite state," which even the democracy-despising Hayek derided as incompatible with human freedom.

We connect people. That can be good if they make it positive. Maybe someone finds love. Maybe it even saves the life of someone on the brink of suicide. So we connect more people. That can be bad if they make it negative. Maybe it costs a life by exposing someone to bullies. Maybe someone dies in a terrorist attack coordinated on our tools. And still we connect people. The ugly truth is that… anything that allows us to connect more people more often is de facto good. It is perhaps the only area where the metrics do tell the true story as far as we are concerned.… That's why all the work we do in growth is justified. All the questionable contact importing practices. All the subtle language that helps people stay searchable by friends. All of the work we do to bring more communication in.… The best products don't win. The ones everyone uses win… make no mistake, growth tactics are how we got here.

A significant result of the systematic application of radical indifference is that the public-facing “first text” is vulnerable to corruption with content that would normally be perceived as repugnant: lies, systematic disinformation, fraud, violence, hate speech, and so on. As long as content contributes to “growth tactics,” Facebook “wins.” This vulnerability can be an explosive problem on the demand side, the user side, but it breaks through the fortifications of radical indifference only when it threatens to interrupt the flow of surplus into the second “shadow” text: the one that is for them but not for us. The norm is that information corruption is not catalogued as problematic unless it poses an existential threat to supply operations—Bosworth’s imperative of connection—either because it might trigger user disengagement or because it might attract regulatory scrutiny. This means that any efforts toward “content moderation” are best understood as defensive measures, not as acts of public responsibility.

In a formal sense, professional journalism is the precise opposite of radical indifference. The journalist’s job is to produce news and analysis that separate truth from falsehood. This rejection of equivalence defines journalism’s raison d’être as well as its organic reciprocities with its readers. Under surveillance capitalism, though, these reciprocities are erased. A consequential example was Facebook’s decision to standardize the presentation of its News Feed content so that “all news stories looked roughly the same as each other… whether they were investigations in The Washington Post, gossip in the New York Post, or flat-out lies in the Denver Guardian, an entirely bogus newspaper.” This expression of equivalence without equality made Facebook’s first text exceptionally vulnerable to corruption from what would come to be called “fake news.”

Economists Hunt Allcott and Matthew Gentzkow, who have studied these phenomena in detail, define “fake news” as “distorted signals uncorrelated with the truth” that impose “private and social costs by making it more difficult… to infer the true state of the world.…” They found that in the lead-up to the 2016 US election there were 760 million instances of a user reading these intentionally orchestrated lies online, or about three such stories for each adult American.

As radical indifference would predict, however, “fake news” and other forms of information corruption have been perennial features of Google and Facebook’s online environments. There are countless examples of disinformation that survived and even thrived because it fulfilled economic imperatives, and I point out just a few. In 2007 a prominent financial analyst worried that the subprime mortgage bust would harm Google’s lucrative ad business. It seems a strange observation until you learn that in the years prior to the Great Recession, Google eagerly welcomed shady subprime lenders into its behavioral futures markets, anxious to net the lion’s share of the $200 million in monthly revenue that mortgage lenders were spending on online advertising. A 2011 Consumer Watchdog report on Google’s advertising practices leading up to and during the Great Recession concluded that “Google has been a prominent beneficiary of the national home loan and foreclosure crisis… by accepting deceptive advertising from fraudulent operators who falsely promise unwary consumers that they can solve their mortgage and credit problems.” Despite these increasingly public facts, Google continued to serve its fraudulent business customers until 2011, when the US Treasury Department finally required the company to suspend advertising relationships with “more than 500 internet advertisers associated with the 85 alleged online mortgage fraud schemes and related deceptive advertising.” … Only a few months earlier, the Department of Justice had fined Google $500 million, “one of the largest financial forfeiture penalties in history,” for accepting ads from online Canadian pharmacies that encouraged Google’s US users to illegally import controlled drugs, despite repeated warnings.

Information corruption has also been a continuous feature of the Facebook environment. The turmoil associated with the 2016 US and UK political disinformation campaigns on Facebook was a well-known problem that had disfigured elections and social discourse in Indonesia, the Philippines, Columbia, Germany, Spain, Italy, Chad, Uganda, Finland, Sweden, Holland, Estonia, and the Ukraine.

That radical indifference produces equivalence without equality also affects the high science of targeted advertising. For example, journalist Julia Angwin and her colleagues at ProPublica discovered that Facebook “enabled advertisers to direct their pitches to the news feeds of almost 2,300 people who expressed interest in the topics of ‘Jew hater,’ ‘How to burn jews,’ or ‘History of why jews ruin the world.’” As the journalists explained, “Facebook has long taken a hands-off approach to its advertising business.… Facebook generates its ad categories automatically based both on what users explicitly share with Facebook and what they implicitly convey through their online activity.”

Zuckerberg also used his super-voting power to reject a shareholder proposal that would have required the company to report on its management of disinformation and the societal consequences of its practices, and Google executives successfully fought back a similar shareholder proposal that year.

Radical indifference means that it doesn’t matter what is in the pipelines as long as they are full and flowing. Camouflaged as a retreat from corruption, the new strategy doubled down on activities rich in behavioral surplus, especially the live videos that Zuckerberg had long coveted.

Surveillance capitalism’s successful claims to freedom and knowledge, its structural independence from people, its collectivist ambitions, and the radical indifference that is necessitated, enabled, and sustained by all three now propel us toward a society in which capitalism does not function as a means to inclusive economic or political institutions. Instead, surveillance capitalism must be reckoned as a profoundly antidemocratic social force.

Surveillance capitalism’s antidemocratic and antiegalitarian juggernaut is best described as a market-driven coup from above. It is not a coup d’état in the classic sense but rather a coup de gens: an overthrow of the people concealed as the technological Trojan horse that is Big Other. On the strength of its annexation of human experience, this coup achieves exclusive concentrations of knowledge and power that sustain privileged influence over the division of learning in society: the privatization of the central principle of social ordering in the twenty-first century. … In a surreal paradox, this coup is celebrated as “personalization,” although it defiles, ignores, overrides, and displaces everything about you and me that is personal.

Surveillance capitalism rules by instrumentarian power through its materialization in Big Other, which, like the ancient tyrant, exists out of mankind while paradoxically assuming human shape. Surveillance capitalism’s tyranny does not require the despot’s whip any more than it requires totalitarianism’s camps and gulags. All that is needed can be found in Big Other’s reassuring messages and emoticons, the press of the others not in terror but in their irresistible inducements to confluence, the weave of your shirt saturated with sensors, the gentle voice that answers your queries, the TV that hears you, the house that knows you, the bed that welcomes your whispers, the book that reads you.… Big Other acts on behalf of an unprecedented assembly of commercial operations that must modify human behavior as a condition of commercial success. It replaces legitimate contract, the rule of law, politics, and social trust with a new form of sovereignty and its privately administered regime of reinforcements.

The success of this coup de gens stands as sour testimony to the thwarted needs of the second modernity, which enabled surveillance capitalism to flourish and still remains its richest vein for extraction and exploitation. In this context it is not difficult to understand why Facebook’s Mark Zuckerberg offers his social network as the solution to the third modernity. He envisions a totalizing instrumentarian order—he calls it the new global “church”—that will connect the world’s people to “something greater than ourselves.” It will be Facebook, he says, that will address problems that are civilizational in scale and scope, building “the long-term infrastructure to bring humanity together” and keeping people safe with “artificial intelligence” that quickly understands “what is happening across our community.”

Right now we are at the beginning of a new arc that I have called information civilization, and it repeats the same dangerous arrogance. The aim now is not to dominate nature but rather human nature. The focus has shifted from machines that overcome the limits of bodies to machines that modify the behavior of individuals, groups, and populations in the service of market objectives. This global installation of instrumentarian power overcomes and replaces the human inwardness that feeds the will to will and gives sustenance to our voices in the first person, incapacitating democracy at its roots.

The rise of instrumentarian power is intended as a bloodless coup, of course. Instead of violence directed at our bodies, the instrumentarian third modernity operates more like a taming. Its solution to the increasingly clamorous demands for effective life pivots on the gradual elimination of chaos, uncertainty, conflict, abnormality, and discord in favor of predictability, automatic regularity, transparency, confluence, persuasion, and pacification.

Instrumentarian power has gathered strength outside of mankind but also outside of democracy. There can be no law to protect us from the unprecedented, and democratic societies, like the innocent world of the Tainos, are vulnerable to unprecedented power.

Repulsed by the social degradation and climate chaos produced by nearly four decades of neoliberal policy and practice, an important and varied group of scholars and activists argues that the era of capitalism is at end. Some propose more-humane economic alternatives, some anticipate protracted decline, and others, repelled by social complexity, favor a blend of elite power and authoritarian politics in closer emulation of China's authoritarian system.

These developments alert us to a deeper truth: just as capitalism cannot be eaten raw, people cannot live without the felt possibility of homecoming. Hannah Arendt explored this territory more than sixty years ago in The Origins of Totalitarianism, where she traced the path from a thwarted individuality to a totalizing ideology. It was the individual's experience of insignificance, expendability, political isolation, and loneliness that stoked the fires of totalitarian terror. Such ideologies, Arendt observed, appear as “a last support in a world where nobody is reliable and nothing can be relied upon.” Years later, in his moving 1966 essay “Education after Auschwitz,” social theorist Theodor Adorno attributed the success of German fascism to the ways in which the quest for effective life had become an overwhelming burden for too many people: “One must accept that fascism and the terror it caused are connected with the fact that the old established authorities… decayed and were toppled, while the people psychologically were not yet ready for self-determination. They proved to be unequal to the freedom that fell into their laps.”

Should we grow weary of our own struggle for self-determination and surrender instead to the seductions of Big Other, we will inadvertently trade a future of homecoming for an arid prospect of muted, sanitized tyranny. A third modernity that solves our problems at the price of a human future is a cruel perversion of capitalism and of the digital capabilities it commands. It is also an unacceptable affront to democracy. I repeat Thomas Piketty's warning: “A market economy… if left to itself… contains powerful forces of divergence, which are potentially threatening to democratic societies and to the values of social justice on which they are based.” This is precisely the whirlwind that we will reap at the hands of surveillance capitalism, an unprecedented form of raw capitalism that is surely contributing to the tempering of commitment to the democratic prospect as it successfully bends populations to its soft-spoken will. It gives so much, but it takes even more.

Surveillance capitalism arrived on the scene with democracy already on the ropes, its early life sheltered and nourished by neoliberalism’s claims to freedom that set it at a distance from the lives of people. Surveillance capitalists quickly learned to exploit the gathering momentum aimed at hollowing out democracy’s meaning and muscle. Despite the democratic promise of its rhetoric and capabilities, it contributed to a new Gilded Age of extreme wealth inequality, as well as to once-unimaginable new forms of economic exclusivity and new sources of social inequality that separate the tuners from the tuned. Among the many insults to democracy and democratic institutions imposed by this coup des gens, I count the unauthorized expropriation of human experience; the hijack of the division of learning in society; the structural independence from people; the stealthy imposition of the hive collective; the rise of instrumentarian power and the radical indifference that sustains its extractive logic; the construction, ownership, and operation of the means of behavior modification that is Big Other; the abrogation of the elemental right to the future tense and the elemental right to sanctuary; the degradation of the self-determining individual as the fulcrum of democratic life; and the insistence on psychic numbing as the answer to its illegitimate quid pro quo. We can now see that surveillance capitalism takes an even more expansive turn toward domination than its neoliberal source code would predict, claiming its right to freedom and knowledge, while setting its sights on a collectivist vision that claims the totality of society. Though still sounding like Hayek, and even Smith, its antidemocratic collectivist ambitions reveal it as an insatiable child devouring its aging fathers.

Cynicism is seductive and can blind us to the enduring fact that democracy remains our only channel for reformation. It is the one idea to have emerged from the long story of human oppression that insists upon a people’s inalienable right to rule themselves. Democracy may be under siege, but we cannot allow its many injuries to deflect us from allegiance to its promise. It is precisely in recognition of this dilemma that Piketty refuses to concede defeat, arguing that even “abnormal” dynamics of accumulation have been—and can again be—mitigated by democratic institutions that produce durable and effective countermeasures: “If we are to regain control of capital, we must bet everything on democracy.…”

Democracy is vulnerable to the unprecedented, but the strength of democratic institutions is the clock that determines the duration and destructiveness of that vulnerability. In a democratic society the debate and contest afforded by still-healthy institutions can shift the tide of public opinion against unexpected sources of oppression and injustice, with legislation and jurisprudence eventually to follow.

The professor was an optimist and a tireless educator who believed that legislative and judicial action invariably reflect the public opinion of twenty to thirty years earlier. It was an insight that he and Hayek—the two have been described as “soul mates and adversaries”—had crafted and transformed into systematic strategies and tactics. As Hayek told Robert Bork in a 1978 interview, “I’m operating on public opinion. I don’t even believe that before public opinion has changed, a change in the law will do any good… the primary thing is to change opinion.…”

This is precisely the lesson of Ford’s discovery and the logic behind successive episodes of revitalization over many centuries. “There is no single variety of capitalism or organization of production,” Piketty writes. “This will continue to be true in the future, no doubt more than ever: New forms of organization and ownership remain to be invented.” Harvard philosopher Roberto Unger enlarges on this point, arguing that market forms can take any number of distinct legal and institutional directions, “each with dramatic consequences for every aspect of social life” and “immense importance for the future of humanity.”

Capitalism has survived the longue durée less because of any specific capability and more because of its plasticity. It survives and thrives by periodically renewing its roots in the social, finding new ways to generate new wealth by meeting new needs. Its evolution has been marked by a convergence of basic principles—private property, the profit motive, and growth—but with new forms, norms, and practices in each era.

What is at stake here is the human expectation of sovereignty over one's own life and authorship of one's own experience. What is at stake is the inward experience from which we form the will to will and the public spaces to act on that will. What is at stake is the dominant principle of social ordering in an information civilization and our rights as individuals and societies to answer the questions Who knows? Who decides? Who decides who decides?

Whatever has gone wrong, the responsibility to right it is renewed with each generation. Pity us and those who come next if we forfeit a human future to powerful companies and a rogue capitalism that fail to honor our needs or serve our genuine interests. Worse still would be our own voiceless capitulation to the message of inevitability that is power's velvet-gloved right hand.

So it is for me and perhaps for you: the bare facts of surveillance capitalism necessarily arouse my indignation because they demean human dignity. The future of this narrative will depend upon the indignant citizens, journalists, and scholars drawn to this frontier project; indignant elected officials and policy makers who understand that their authority originates in the foundational values of democratic communities; and, especially, indignant young people who act in the knowledge that effectiveness without autonomy is not effective, dependency-induced compliance is no social contract, a hive with no exit can never be a home, experience without sanctuary is but a shadow, a life that requires hiding is no life, touch without feel reveals no truth, and freedom from uncertainty is no freedom.

Burnham’s cowardice is a cautionary tale. We are living in a moment when surveillance capitalism and its instrumentarian power appear to be invincible. Orwell’s courage demands that we refuse to cede the future to illegitimate power. He asks us to break the spell of enthrallment, helplessness, resignation, and numbing. We answer his call when we bend ourselves toward friction, rejecting the smooth flows of coercive confluence. Orwell’s courage sets us against the relentless tides of dispossession that demean all human experience. Friction, courage, and bearings are the resources we require to begin the shared work of synthetic declarations that claim the digital future as a human place, demand that digital capitalism operate as an inclusive force bound to the people it must serve, and defend the division of learning in society as a source of genuine democratic renewal.

Arendt, like Orwell, asserts the possibility of new beginnings that do not cleave to already visible lines of power. She reminds us that every beginning, seen from the perspective of the framework that it interrupts, is a miracle. The capacity for performing such miracles is entirely human, she argues, because it is the source of all freedom: “What usually remains intact in the epochs of petrification and foreordained doom is the faculty of freedom itself, the sheer capacity to begin, which animates and inspires all human activities and is the hidden source… of all great and beautiful things.”

Surely the Age of Surveillance Capitalism will meet the same fate as it teaches us how we do not want to live. It instructs us in the irreplaceable value of our greatest moral and political achievements by threatening to destroy them. It reminds us that shared trust is the only real protection from uncertainty. It demonstrates that power untamed by democracy can only lead to exile and despair. Friedman’s cycle of public opinion and durable law now reverts to us: it is up to us to use our knowledge, to regain our bearings, to stir others to do the same, and to found a new beginning. In the conquest of nature, industrial capitalism’s victims were mute. Those who would try to conquer human nature will find their intended victims full of voice, ready to name danger and defeat it.

The Berlin Wall fell for many reasons, but above all it was because the people of East Berlin said, “No more!” We too can be the authors of many “great and beautiful” new facts that reclaim the digital future as humanity’s home. No more! Let this be our declaration.

When a lightning fire destroyed our home in 2009, thousands of books along with every trace of my scholarly career and new work in progress vanished in a few hours. I thought that I would never write again, but my brilliant and beloved husband, Jim Maxmin, insisted that time would bring rebirth. It did.

Frank urged me to write for FAZ while I pursued the longer work, insisting on publishing essays that, in my monkish way, I would have incubated for many more months or years. I learned so much from our endless discussions, and it is because of Frank that my work on Big Other and surveillance capitalism became useful public frameworks long before this book was finalized.

Every author acknowledges what I am about to write, because it is true: In the end, one faces the page in solitude. Anything in this book that falls short of the trust invested in me is my responsibility alone.

From the very first page I was consumed with an overwhelming imperative: everyone needs to read this book as an act of digital self-defense. With tremendous lucidity and moral courage, Zuboff demonstrates not only how our minds are being mined for data but also how they are being rapidly and radically changed in the process. The hour is late and much has been lost already—but as we learn in these indispensable pages, there is still hope for emancipation.

I will make a guarantee: Assuming we survive to tell the tale, The Age of Surveillance Capitalism has a high probability of joining the likes of Adam Smith's The Wealth of Nations and Max Weber's Economy and Society as defining social-economics texts of modern times. It is not a 'quick read'; it is to be savored and re-read and discussed with colleagues and friends.

The Age of Surveillance Capitalism is brilliant and essential. Shoshana Zuboff reveals capitalism's most dangerous frontier with stunning clarity: The new economic order of surveillance capitalism founded on extreme inequalities of knowledge and power. Her sweeping analysis demonstrates the unprecedented challenges to human autonomy, social solidarity, and democracy perpetrated by this rogue capitalism.

IV. The Discovery of Behavioral Surplus … VI. A Human Invention … VII. The Secrets of Extraction … VIII. Summarizing the Logic and Operations of Surveillance Capitalism CHAPTER FOUR The Moat Around the Castle I.

CHAPTER SEVEN The Reality Business … V. Certainty for Profit … VI. Executing the Uncontract … VII. Inevitabilism … VIII. Men Made It IX.

CHAPTER ELEVEN The Right to the Future Tense … I. I Will to Will … II. We Will to Will … III. How Did They Get Away with It?

CHAPTER SEVENTEEN The Right to Sanctuary I. Big Other Outruns Society II. Justice at the New Frontier of Power III. Every Unicorn Has a Hunter CONCLUSION CHAPTER EIGHTEEN A Coup from Above I.

CHAPTER FOURTEEN A Utopia of Certainty I. Society as the Other-One II. Totality Includes Society III. Applied Utopistics IV. Confluence as Machine Relations V. Confluence as Society CHAPTER FIFTEEN The Instrumentarian Collective I.

CHAPTER THIRTEEN Big Other and the Rise of Instrumentarian Power I. Instrumentarianism as a New Species of Power II. A Market Project of Total Certainty III. This Century's Curse IV. The China Syndrome V. A Fork in the Road CHAPTER FOURTEEN A Utopia of Certainty I.

CHAPTER SIXTEEN Of Life in the Hive I. Our Canaries in the Coal Mine II. The Hand and the Glove III. Proof of Life IV. The Next Human Nature V. Homing to the Herd VI. No Exit CHAPTER SEVENTEEN The Right to Sanctuary I.

CHAPTER TWELVE Two Species of Power I. A Return to the Unprecedented II. Totalitarianism as a New Species of Power III. An Opposite Horizon IV. The Other-One V. Against Freedom VI. A Technology of Human Behavior VII. Two Utopias CHAPTER THIRTEEN Big Other and the Rise of Instrumentarian Power I.

By 2014, about twenty years after the invention of the world wide web, an extensive survey by Pew Research found 87 percent of Americans using the internet. Among those, 76 percent regarded it as “a good thing for society” and 90 percent as “a good thing for me.” Indeed, people routinely call 911 when Facebook is down. In less than two decades after the Mosaic browser was released to the public, enabling easy access to the world wide web, a 2010 BBC poll found that 79 percent of people in twenty-six countries considered internet access to be a fundamental human right. Six years later, the United Nations adopted specific language on internet access: “Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers."

6. Lizabeth Cohen, A Consumers' Republic: The Politics of Mass Consumption in Postwar America (New York: Knopf, 2003); Martin J. Sklar, The Corporate Reconstruction of American Capitalism: 1890–1916: The Market, the Law, and Politics (New York: Cambridge University Press, 1988).

10. For readers interested in a more detailed analysis of the rise of this phenomenon, I recommend the extended discussion in Zuboff and Maxmin, The Support Economy. See also Ulrich Beck and Elisabeth Beck-Gernsheim, Individualization: Institutionalized Individualism and Its Social and Political Consequences (London: Sage, 2002); Ulrich Beck, “Why 'Class' Is Too Soft a Category to Capture the Explosiveness of Social Inequality at the Beginning of the Twenty-First Century,” British Journal of Sociology 64, no. 1 (2013): 63–74; Ulrich Beck and Edgar Grande, “Varieties of Second Modernity: The Cosmopolitan Turn in Social and Political Theory and Research,” British Journal of Sociology 61, no. 3 (2010): 409–43.

Thomas Piketty, Capital in the Twenty-First Century (Cambridge, MA: Belknap Press, 2014). Piketty integrated years of income data to conclude that income inequality in the US and the UK has reached levels not seen since the nineteenth century. The top decile of US wage earners steadily increased its share of national income from 35 percent in the 1980s to over 46 percent in 2010. The bulk of this increase comes from the top 1 percent, whose share rose from 9 percent to 20 percent, about half of which went to the 0.1 percent. Piketty calculates that 60–70 percent of the top 0.1 percent of the income hierarchy is composed of managers who have succeeded in obtaining “historically unprecedented” compensation thanks to new value-maximizing incentive structures.

Studies of financial deepening—liberalization, financialization—in both developed and less developed economies have shown that it is linked to new instabilities, including bankruptcies, bank failures, extreme asset volatility, and recession in the real sectors of the economy.

Historian Nancy MacLean and journalist Jane Mayer document hidden operations of radical-right ideologues and their billionaire backers who command unlimited funds for the purposes of political and public manipulation, relying on clandestine networks of think tanks, donor organizations, and media outlets to skillfully exploit citizen unrest and drive it toward extremist views.

A national poll conducted by Software Advice found that 61 percent of Americans believe that some version of the right to be forgotten is necessary, 39 percent want a European-style blanket right to be forgotten, and nearly half were concerned that “irrelevant” search results can harm a person’s reputation. A survey by YouGov found that 55 percent of Americans would support legislation similar to the right to be forgotten, compared to only 14 percent who would not. A US survey by Benenson Strategy Group and SKDKnickerbocker published nearly a year after the EU decision found that 88 percent of respondents somewhat (36 percent) or strongly (52 percent) supported a US law that would let them petition companies such as Google, Yahoo!, and Bing to remove certain personal information that appears in search results.

Although it’s important to note that Hal Varian is not an executive leader at Google, there is a great deal in the public record to suggest that he has played a leading role in helping Google’s leaders grasp the operations and implications of their own commercial logic as well as its extension and elaboration. I compare Varian’s insights to those of James Couzens at Ford. Couzens was an investor and businessman—he would later become a US senator—who served as general manager at Ford. He helped shepherd Ford’s spectacular success with his clear grasp of the new logic of mass production and its economic significance. He was not a theoretician or a prolific writer like Varian, but his correspondence and articles were graced with unusual insight and have remained a vital source for scholars of mass production.

Varian spent several years as a consultant to Google before becoming the firm's chief economist in 2007. He notes in his biographical material that "since 2002 he has been involved in many aspects of the company, including auction design, econometric analysis, finance, corporate strategy, and public policy." When the Wall Street Journal reported on Varian's new position at Google in 2007, it noted that the position entailed building "a team of economists, statisticians, and analysts to assist the company in 'marketing, in human resources, in strategy, in policy related stuff.'" In his book on Google, Steven Levy quotes Eric Schmidt reflecting on how the firm learned to exploit its new "click economy": "We have Hal Varian and we have the physicists." In Levy's 2009 Wired article on "Googlenomics," Schmidt credits Varian's early examination of the firm's ad auctions with providing the eureka moment that clarified the true nature of Google's business: "All of a sudden, we realized we were in the auction business."

In the work that I cite here, Varian frequently illustrates his points with examples from Google. He often uses the first-person plural in these instances, such as "Google has been so successful with our own experiments that we have made them available to our advertisers and publishers in two programs." Therefore, it seems fair to assume that Varian's perspectives provide critical insights into the premises and objectives that define this new market form.

Among the first generation of Silicon Valley venture-backed internet startups, 12.5 percent had completed IPOs by the end of 2001, compared to 7.3 percent in the rest of the country, while only 4.2 percent of valley startups attained profitability, a significantly lower proportion than in the rest of the country.

Bharat, Lawrence, and Sahami, 16–17. The list of attributes included the content (e.g., words, Anchortext, etc.) of websites that the user has visited (or visited in a certain time period); demographic information; geographic information; psychographic information; previous queries (and/or associated information) that the user has made; information about previous advertisements that the user has been shown, has selected, and/or has made purchases after viewing; information about documents (e.g., word processor files) viewed/requested and/or edited by the user; user interests; browsing activity; and previous purchasing behavior.

A typical example is this statement from the Economist: “Google exploits information that is a by-product of user interactions, or data exhaust, which is automatically recycled to improve the service or create an entirely new product.”

According to Alphabet's account in 2016, its revenues were $90,272,000,000. This included "Google segment revenues of $89.5 billion with revenue growth of 20% year over year and Other Bets revenues of $0.8 billion with revenue growth of 82% year over year." Google Segment advertising revenues were $79,383,000,000, or 88.73 percent of Google Segment revenues.

This translated into 76 percent of all desktop searches and 96 percent of mobile searches in the US and corresponding shares of 87 percent and 95 percent worldwide.

When shareholders objected to the system at the company's annual general meeting by casting 180 million votes for an equal-voting-rights resolution, they were swamped by the 551 million votes owned by the founders.

In 2017 Snap's IPO offered only nonvoting shares, leaving its founders with 70 percent control of all votes and the remainder in the hand of pre-IPO investors. … Other IPOs featured super-voting shares with as many as thirty to ten thousand times as many votes per share as the ordinary class of stock.

Wiebe summarizes the worldview put forth by the millionaires as they coordinated to repulse the electoral threat to industrial capital, and it will sound familiar to anyone who has read the justifications of the Silicon Valley tycoons and their adulation of all things "destructive" and "entrepreneurial." According to the nineteenth-century catechism, only the "highest types of their race discovered more effective ways to combine land, labor, and capital, and drew society upward as the rest reorganized behind their leaders." The majority of "ordinary talent" was left to divide what remained after the requirements of capital, and "the weakest simply disappeared." The result was to be "an ever improving race winnowed by competition." Any violation of these "natural laws" would only benefit "the survival of the unfittest" and reverse the evolution of the race.

Economic historian Philip Mirowski summarizes the “metatheses” that since the 1980s helped to constitute neoliberalism as a loose “paradigm,” despite its amorphous, multifaceted, and sometimes contradictory theories and practices. Among these, several became essential shelter for the surveillance capitalists’ bold actions, secret operations, and rhetorical misdirection: (1) democracy was to be constrained in favor of actively reconstructing the state as the agent of a stable market society; (2) the entrepreneur and the corporation were conflated, enshrining “corporate personhood,” rather than the rights of citizens, as the focus of legal protections; (3) freedom was defined negatively, as “freedom from” interference in the natural laws of competition, and all control was understood as coercive, except for market control; and (4) inequality of wealth and rights was accepted and even celebrated as a necessary feature of a successful market system and a force for progress.

Another detailed study, by Luigi Vigneri and his colleagues at EURECOM in 2015, looked closely at the 5,000 newest and most popular applications in the Google Play store. Researchers found that 500 of these apps connect to more than 500 distinct URLs, and 25 connect to more than 1,000 URLs. Multiple URLs can connect to the same “domain.” So the researchers also examined the domains that were most often the source of these connections. Nine of the top twenty domains behind these hidden connections are web services run by Google. Of the remaining eleven, three are owned or affiliated with Google. The other eight are Google competitors in behavioral futures markets, including Facebook, Samsung, and Scorecard Research, a data broker that sells surplus behavior to its customers.

The researchers then took another valuable step. They characterized each of the URLs visited by these apps as either “ad-related” or “user-tracking related” and found that 66 percent of the apps contact an average of 40 ad-related URLs, although in some cases it’s more than 1,000. Of the five top domains represented by these URLs, three belong to Google. When it comes to tracking, the data suggest that the competition for behavioral surplus is even more contested. Among the apps in the study, 73 percent did not connect to tracking sites, but 16 percent connected to 100 or more tracking sites. Google remains the dominant force here, with 44 percent of the tracker-related domains, followed by 32 percent operated by AT Internet, a privately held “digital intelligence” firm that specializes in “behavior analysis.” Four of the ten most-intensive tracking apps in Google Play were also awarded Google’s “Top Developer Badge.”

A team of researchers from the University of Washington led by Adam Lerner and Anna Simpson studied the growth of web trackers from 1996 to 2016. Not surprisingly from our point of view, web tracking increased in tandem with the rise and institutionalization of surveillance capitalism, as did third-party connections. The researchers note that although earlier trackers recorded routine data oriented toward product stability, the more recent rise in trackers is of those that capture and analyze personal information. In 2000 only about 5 percent of sites contacted at least five third parties, but by 2016, 40 percent of sites sent data to third parties. Among trackers with the “most power to capture profiles of user behavior across many sites,” google-analytics.com is cited as a “remarkable outlier,” gathering more data from more sites than any other entity does. The researchers conclude that despite the privacy concerns that have received so much attention in recent years, tracking has substantially expanded in “scope and complexity” on a clear upward trend line. In other words, there is more tracking now than at any time since the launch of the internet, even as citizens and governments try to protect individual privacy.

In 2002, a one-point increase in digitalization score predicted a $166.20 (in 2016 dollars) increase in real annual average wages for occupations with the same education requirements. By 2016 this wage premium had almost doubled to $292.80. In sum, workers with superior digital skills are more and more earning higher wages (all other things being equal) than similarly educated workers with fewer digital skills.

It is a question of these laws themselves, of these tendencies working with iron necessity towards inevitable results. The country that is more developed industrially only shows, to the less developed, the image of its own future.

In 1997 data privacy researcher Latanya Sweeney famously demonstrated that using publicly available population registers (e.g., a voter list), she could identify the medical records of Massachusetts Governor William Weld from medical information that had already been stripped of all explicit identifiers, such as name, address, and Social Security number.

In 2015, 29.5 million US adults used wearable devices—mostly fitness trackers like Under Armour's and smart watches—representing an increase of 57.7 percent over 2014.

Forbes reported on the excitement with which industry met this news: "Big pharm companies are launching hundreds of mobile apps… while corporations take extra mobile strides to keep employees' health insurance costs down.…"

In 2012 Pew Research reported that 53 percent of Americans owned smartphones and, of those, 20 percent had downloaded at least one health-related app. By 2015, a national survey found that 71 percent of Americans owned a smartphone or other wireless device and, of those, 32 percent had downloaded at least one health-related app, fulfilling "the public's desire for 'anywhere, anytime' monitoring, diagnosis, and treatment."

As the results of this growing body of work were pulled into the orbit of surveillance capitalism's supply operations, it is worth noting, the Psychometrics Centre was invited to relocate to the University's Judge School of Business. The Judge School's embrace of the centre was explicitly linked to its commercial prospects and above all to the prospects of depth surplus and personality prediction as they could be applied to surveillance capitalism's prediction requirements.

Today, the digital traces we leave behind allow machines to treat all of our online activity as a 'test.' Our Facebook Likes, the words we use in tweets and emails and the images we upload all provide 'items' from which the machine can learn who we are, what drives and motivates us and how we differ from each other. Psychometrics is at the forefront of developments in ambient intelligence and the Internet of Things, powering connected environments that are sensitive and responsive to our needs.

The commercial theory and practice of behavior modification assume an inescapable networked presence and its cornucopia of digital tools. A team of British researchers surveyed fifty-five behavioral experts in order to compile "a consensually agreed hierarchically structured taxonomy of techniques used in behavior change interventions." This exercise identified ninety-three distinct behavior-change techniques, which were grouped into sixteen methodological clusters: "scheduled consequences," "reward and threat," "repetition and substitution," "antecedents," "associations," "feedback and monitoring," "goals and planning," "social support," "comparison of behavior," "communication of natural consequences," "self-belief," "comparison of outcomes," "shaping knowledge," "regulation," "identity," and "covert learning."

The researchers warn that behavior modification is a "fast-moving field." As illustration, they note that the first such taxonomy, published only four years earlier, had identified just twenty-two behavior-change techniques, many of which were individually oriented and required face-to-face interaction and relationship building. In contrast, the newer techniques are aimed at "community and population-level interventions," a fact that speaks to the migration of behavior-change operations to the novel capabilities (contextual control, automated digital nudges, operant conditioning at scale) of the internet-enabled tools upon which economies of action depend.

Duranty won the Pulitzer Prize in 1932 for New York Times articles written from Moscow in 1931. Subsequently, Duranty's reporting was challenged by anti-Stalinist groups who regarded Duranty as a tool for Stalinist propaganda. The Pulitzer committee investigated the allegations for six months and in the end decided not to revoke the award. Years later, the New York Times would conclude that Duranty's work was some of the worst reporting in the history of that newspaper. None of this prevented Look and other periodicals from continuing to rely on Duranty's accounts of life in the Soviet Union, certainly a significant contribution to the slow pace of public recognition of the unique features of totalitarian power.

Skinner reiterated Meyer's definitive importance to the radical behaviorist point of view. See, for example, his 1967 debate with the philosopher Brand Blanshard, where he stated the following: "A special problem arises from the inescapable fact that a small part of the universe is enclosed within the skin of each of us. It is not different in kind from the rest of the universe, but because our contact with it is intimate and in some ways exclusive, it receives special consideration. It is said to be known in a special way, to contain the immediately given, to be the first thing a man knows and according to some the only thing he can really know. Philosophers, following Descartes, begin with it in their analysis of mind. Almost everyone seems to begin with it in explaining his own behavior. There is, however, another possible starting point—the behavior of what Max Meyer used to call the Other-One. As a scientific analysis grows more effective, we no longer explain that behavior in terms of inner events. The world within the skin of the Other-One loses its preferred status."

In 1946 Planck's long-held views on the unity of science and scientific reasoning would be summarized in his paper on "phantom problems" in science, including the "mind-body problem" and the "problem of free will." Planck viewed the mind-body controversy, like all phantom problems in philosophy and science, as a failure to specify "the viewpoint of the observation" and adhere to it consistently (italics mine). He argued that the "internal" or "psychological" viewpoint and the "external" or "physiological" viewpoint were too frequently confused: "What you feel, think, want, only you can know as firsthand information. Other people can conclude it only indirectly, from your words, conduct, actions and mannerisms. When such physical manifestations are entirely absent, they have no basis whatever to enable them to know your momentary mental state." The external viewpoint was thus the only one admissible "as the basis of our scientific observation of volitional processes." The establishment of this "external viewpoint" as the basis for the scientific study of human behavior is critical to our story, and Planck's paper suggests the influence that his thinking exerted on Meyer (and even the possibility that Planck had read Meyer's 1921 work). In any case, the resonance between Planck's argument and Meyer's is plain.

44. Christopher Lunt, United States Patent: 9100400—Authorization and authentication based on an individual’s social network, issued August 4, 2015, http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d =PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1 =9100400.PN.&OS=PN/9100400&RS=PN/9100400. The critical paragraph in the patent reads as follows: “In a fourth embodiment of the invention, the service provider is a lender. When an individual applies for a loan, the lender examines the credit ratings of members of the individual’s social network who are connected to the individual through authorized nodes. If the average credit rating of these members is at least a minimum credit score, the lender continues to process the loan application. Otherwise, the loan application is rejected.”

5. The term social physics originates in the positivist philosophy of Auguste Comte, who preceded Planck in his programmatic vision of a scientific approach to the study of society that would equal the precision of the natural sciences. In the 1830s Comte wrote the following: “Now that the human mind has founded celestial physics, terrestrial physics… and organic physics… it only remains to complete the system of observational sciences by the foundation of social physics.” See Auguste Comte, Introduction to Positive Philosophy, ed. Frederick Ferré (Indianapolis: Hackett, 1988), 13.

Nearly two hundred years later, Pentland’s theory and research in social physics have made him the focus of articles in the New York Times, the Harvard Business Review, and the New Yorker, as well as a prominent speaker on the global circuit, from the UN and the World Economic Forum to corporations and international conferences. At Microsoft and Google he has been featured as the “presiding genius” of the “Big Data revolution,” whose “groundbreaking experiments” and “remarkable discoveries” have made his work “the bedrock of a whole new scientific field.” With the publication of his book Social Physics, Pentland was introduced at the popular Digital-Life-Design Conference in 2014 by the well-known social media analyst Clay Shirky, who opined that Pentland’s Human Dynamics Lab “has done more to explain human behavior in groups in the last ten years than any other institution in the world.”

These data are drawn from my own compilation of General Motors market capitalization and employment data from 1926 to 2008, Google from 2004 to 2016, and Facebook from 2012 to 2016. All market capitalization values are adjusted for inflation to 2016 dollars, as per the Consumer Price Index from Federal Reserve Economic Data, Economic Research Division, Federal Reserve Bank of St. Louis. The sources used to compile these data include Standard & Poor's Capital IQ (Google Market Capitalization and Headcount), Wharton Research Data Services—CRSP (General Motors Market Capitalization), Standard & Poor's Compustat (General Motors Headcount), Thomas Eikon (Facebook Market Capitalization), Company Annual Reports (General Motors Headcount), and SEC Filings (Facebook Headcount).