{"id":220,"date":"2014-07-01T10:41:37","date_gmt":"2014-07-01T14:41:37","guid":{"rendered":"http:\/\/www.shoshanazuboff.com\/?p=220"},"modified":"2014-07-01T10:41:37","modified_gmt":"2014-07-01T14:41:37","slug":"dark-facebook","status":"publish","type":"post","link":"https:\/\/shoshanazuboff.com\/book\/dark-facebook\/","title":{"rendered":"Dark Facebook: Facebook\u2019s Secret Experiment in Emotional Manipulation Provides a Fresh Glimpse of its Radical Politics and Absolutist Ambitions"},"content":{"rendered":"

Celebrate the Friction:<\/strong> Dark Facebook, like Dark Google, stands tall in the lineup of new absolutist information powers who imagine their users as passive sheep so hungry for their services that we\u2019ll acquiesce to any manipulation.\u00a0 But worldwide outrage over Facebook\u2019s secret \u201cresearch\u201d into emotional contagion is a sign that we are not passive and will not quietly accept a narcotized robot world owned and operated by Big Tech. This friction is cause for celebration. The absolutism and concentration of information power represented by Facebook, Google, and other tech companies will not stand without a contest. We are in the realm of politics now, and that\u2019s exactly where we should be.<\/p>\n

There are many reasons for outrage in this stew, and the experiment itself is only one. The experiment does more than violate well-established principles of human subjects research. It provides a window into Facebook\u2019s absolutist assumptions and how those are expressed in its daily policies and practices.\u00a0 Despite its history of apologizing for privacy violations, Facebook once again appears to regard its users as so many anonymous eyeballs and revenue streams who will happily succumb to any manipulation as long as they get to play online.\u00a0 But the more we learn about the true price we pay for the \u2018free\u2019 services of Facebook, Google, etc., the more we contest their legitimacy.\u00a0 This contest is the key to a future in which the networked world is compatible with principles of democracy and individual rights.<\/p>\n

The Experiment:<\/strong> Facebook\u2019s internal research board signed off on research in 2012 that aimed to test for mechanisms of emotional contagion through online text. The research systematically manipulated the emotional content of daily feeds without users\u2019 knowledge, let alone their informed consent. To do this, they varied the number of posts with positive or negative words in the daily feeds of nearly 700,000 users for one week. They found that exposure to more positive words produced more positive words in users\u2019 communications and more negatives produced more negatives.\u00a0 They drew the inference that their manipulation of the content of daily feeds effected the psychological states of users: their moods and emotions.\u00a0 Ergo, \u201cemotional contagion.\u201d You can read the article in the Proceedings of the National Academy of Sciences here<\/a>.<\/p>\n

The article itself makes bold claims for its findings that I do not find persuasive, but that\u2019s a subject for another discussion. The key point here is twofold: 1) Facebook\u2019s research manipulated not the quantity of messages or the design of the page, both routine, but rather the actual content of feeds. This is messing with the intimate reality of a user\u2019s contact with his or her personal world. One wonders how many other instances of such secret manipulation there are. The history of this subject suggests that once such capabilities are developed they do not lie fallow. They grow in scale and scope as they become available for colonization by ever more powerful interests\u00a0 2) The researchers, and the Facebook internal review board that cleared the research, believed that it was okay to conduct such research secretly, without the knowledge or consent of those nearly 700,000 users whose feed would be affected. They claimed that Facebook\u2019s generic user agreement covered the problem of informed consent.\u00a0 That\u2019s not acceptable. But even on FB\u2019s own terms, subsequent reporting<\/a> reveals that \u201cresearch\u201d was not even included in those terms until four months after the experiment was conducted. The researchers, Adam D. I. Kramer from Facebook\u2019s Core Data Science Team, James Gilroy from UCSF, and Jeffry T. Hancock from Cornell, and their academic editor, Susan Fiske from Princeton, should have known better. They probably did know better.\u00a0 Indeed, Adam Kramer issued an apology of sorts<\/a>\u2014a strange document for many reasons. The grandiose claims of the article are repudiated and recast as weak results. But more interesting still is the obvious fact that he did not imagine the wave of outrage that greeted his triumphant publication in a respected academic journal. All those hapless users were not fooled by the academic white-wash.\u00a0 Adam tried to put the toothpaste back in the tube, but it can\u2019t be done.<\/p>\n

Facebook Absolutism:<\/strong>\u00a0 As I wrote about in my recent Frankfurter Allgemeine Zeitung essay, \u201cDark Google<\/a>,\u201d\u00a0 Facebook, like Google, represents a new kind of business entity whose power is simultaneously ubiquitous, hidden, and unaccountable.\u00a0 I encourage you to read the full article (drawn from my new book still in progress), but I\u201dll summarize a few of the themes here.<\/p>\n

The procedures and effects of the new information barons are not well understood, and therefore they can not be effectively restrained.\u00a0 Individuals and societies are vulnerable to these new powers in ways that we are only gradually coming to light.\u00a0 Facebook, Google, and other tech companies provide services that many have come to regard as essential for basic social participation.\u00a0 Indeed, many of us viewed these firms as harbingers of a new more democratic world enabling unprecedented voice and connection.\u00a0 We thought they were a new kind of company, aligned with our interests. Now the hidden costs of their new goods\u2013\u2013in lost privacy and autonomy at the hands of commercial and even state-sponsored surveillance\u2013\u2013are slowly being revealed.\u00a0 The new firms have reverted to the old GM paradigm but with powers and scope beyond anything the world has known. Now, it\u2019s increasingly the case that the ambition of these companies is shifting from collecting information about us to intervening in and shaping our daily reality for their commercial benefit.\u00a0 I have called this \u201cthe reality business\u201d because \u201creality\u201d is the next big thing that the tech firms want to carve up and sell. In the data business, the payoff is in data patterns that help target ads. In the reality business, the payoff is in shaping and communicating real life behaviors of people and things in millions of ways that drive revenue.\u00a0 The business model is expanding to encompass the digital \u2018you\u2019 as well as the \u2018actual\u2019 you. This is precisely what\u2019s exemplified in Facebook\u2019s secret experiment.<\/p>\n

The \u201creality business\u201d reflects a shift in the frontier of data science from data mining to \u201creality mining\u201d\u00a0 described in one academic paper<\/a> as \u201ca God\u2019s eye view.\u201d\u00a0 But the reality business aims beyond the God\u2019s eye view to the God-like interventions that can shape and control reality.\u00a0 Facebook\u2019s evident interest in mastering the mechanisms of emotional contagion, like Google\u2019s glasses, self-driving cars, or investments in everything from the wired home to drones and satellites share this purpose: to influence and shape human behavior along the lines that feed their bottom line.<\/p>\n

This brings us to the precipice of a new development in the scope of the market economy. A new \u201cfictional commodity\u201d is emerging as a dominant characteristic of market dynamics in the 21st century. \u201cReality\u201d is about to undergo a fictional transformation and be reborn as \u201cbehavior.\u201d\u00a0 This includes the behavior of creatures, their bodies, and their things. It includes actual behavior and data about behavior. It is the world-spanning organism reborn as information and all the tiniest elements within it. This dominion over “reality”\u00a0 is the new currency of the networked surveillance sphere.<\/p>\n

The big question we are left with: Is reality for sale?\u00a0 Right now the outrage of Facebook users says NO! The question remains: how do we translate our outrage into the kinds of institutional and legal frameworks that insure democratic principles and humanistic values in a networked world?<\/p>\n","protected":false},"excerpt":{"rendered":"

Celebrate the Friction: Dark Facebook, like Dark Google, stands tall in the lineup of new absolutist information powers who imagine their users as passive sheep so hungry for their services that we\u2019ll acquiesce to any manipulation.\u00a0 But worldwide outrage over Facebook\u2019s secret \u201cresearch\u201d into emotional contagion is a sign that we are not passive and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-220","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/posts\/220","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/comments?post=220"}],"version-history":[{"count":0,"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/posts\/220\/revisions"}],"wp:attachment":[{"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/media?parent=220"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/categories?post=220"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/shoshanazuboff.com\/book\/wp-json\/wp\/v2\/tags?post=220"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}