… then you know that I have three series of posts in progress right now. Two relate to the “Internet of Things,” automation, and technological enchantment; the third deals with the religious/cultural matrix of technological innovation. As it happens, though, each of these will be on hold for the next week or so during which I’ll be away and without Internet connection–on a digital sabbath of sorts, although more by circumstance than by design. In any case, the blog will be silent for the next several days.
As I follow the train of thought that took the dream of a smart home as a point of departure, I’ve come to a fork in the tracks. Down one path, I’ll continue thinking about the distinctions among Mechanization, Automation, and Animation. Down the other, I’ll pursue the technological enchantment thesis that arose incidentally in my mind as a way of either explaining or imaginatively characterizing the evolution of technology along those three stages.
Separating these two tracks is a pragmatic move. It’s easier for me at this juncture to consider them separately, particularly to weigh the merits of the latter. It may be that the two tracks will later converge, or it may be that one or both are dead ends. We’ll see. Right now I’ll get back to the three stages.
In his comment on my last post, Evan Selinger noted that my schema was Borgmannesque in its approach, and indeed it was. If you’ve been reading along for awhile, you know that I think highly of Albert Borgmann’s work. I’ve drawn on it a time or two of late. Borgmann looked for a pattern that might characterize the development of technology, and he came up with what he called the device paradigm. Succinctly put, the device paradigm described the tendency of machines to become simultaneously more commodious and more opaque, or, to put it another way, easier to use and harder to understand.
In my last post, I used heating as an example to walk through the distinctions among mechanization, automation, and animation. Borgmann also uses heating to illustrate the device paradigm: lighting and sustaining a fire is one thing, flipping a switch to turn on the furnace is another. Food and music also serve as recurring illustrations for Borgmann. Preparing a meal from scratch is one thing, popping a TV dinner in the microwave is another. Playing the piano is one thing, listening to an iPod is another. In each case a device made access to the end product–heat, food, music–easier, instantaneous, safer, more efficient. In each case, though, the workings of the device beneath the commodious surface became more complex and opaque. (Note that in the case of food preparation, both the microwave and the TV dinner are devices.) Ease of use also came at the expense of physical engagement, which, in Borgmann’s view, results in an impoverishment of experience and a rearrangement of the social world.
Keep that dynamic in mind as we move forward. The device paradigm does a good job, I think, of helping us think about the transition to mechanization and from mechanization to automation and animation, chiefly by asking us to consider what we’re sacrificing in exchange for the commodiousness offered to us.
Ultimately, we want to avoid the impulse to automate for automation’s sake. As Nick Carr, whose forthcoming book, The Glass Cage: Automation and Us, will be an excellent guide in these matters, recently put it, “What should be automated is not what can be automated but what should be automated.”
That principle came at the end of a short post reflecting on comments made by Google’s “Android guru,” Sundar Pichai. Pichai offered a glimpse at how Google envisions the future when he described how useful it would be if your car could sense that your child was now inside and automatically changed the music playlists accordingly. Here’s part of Carr’s response:
“With this offhand example, Pichai gives voice to Silicon Valley’s reigning assumption, which can be boiled down to this: Anything that can be automated should be automated. If it’s possible to program a computer to do something a person can do, then the computer should do it. That way, the person will be ‘freed up’ to do something ‘more valuable.’ Completely absent from this view is any sense of what it actually means to be a human being. Pichai doesn’t seem able to comprehend that the essence, and the joy, of parenting may actually lie in all the small, trivial gestures that parents make on behalf of or in concert with their kids — like picking out a song to play in the car. Intimacy is redefined as inefficiency.”
But how do we come to know what should be automated? I’m not sure there’s a short answer to that question, but it’s safe to say that we’re going to need to think carefully about what we do and why we do it. Again, this is why I think Hannah Arendt was ahead of her time when she undertook the intellectual project that resulted in The Human Condition and the unfinished The Life of the Mind. In the first she set out to understand our doing and in the second, our thinking. And all of this in light of the challenges presented by emerging technological systems.
One of the upshots of new technologies, if we accept the challenge, is that they lead us to look again at what we might have otherwise taken for granted or failed to notice altogether. New communication technologies encourage us to think again about the nature of human communication. New medical technologies encourage us to think again about the nature of health. New transportation technologies encourage us to think again about the nature of place. And so on.
I had originally used the word “forced” where I settled for the word “encourage” above. I changed the wording because, in fact, new technologies don’t force us to think again about the realms of life they impact. It is quite easy, too easy perhaps, not to think at all, simply to embrace and adopt the new technology without thinking at all about its consequences. Or, what amounts to the same thing, it is just as easy to reject new technologies out of hand because they are new. In neither case would we be thinking at all. If we accept the challenge to think again about the world as new technologies cast aspects of it in a new light, we might even begin to see this development as a great gift by leading us to value, appreciate, and even love what was before unnoticed.
Returning to the animation schema, we might make a start at thinking by simply asking ourselves what exactly is displaced at each transition. When it comes to mechanization, it seems fairly straightforward. Mechanization, as I’m defining it, ordinarily displaces physical labor.
Capturing what exactly is displaced when it comes to automation is a bit more challenging. In part, this is because the distinctions I’m making between mechanization and automation on the one hand and automation and animation on the other are admittedly fuzzy. In fact, all three are often simply grouped together under the category of automation. This is a simpler move, but I’m concerned that we might not get a good grasp of the complex ways in which technologies interact with human action if we don’t parse things a bit more finely.
So let’s start by suggesting that automation, the stage at which machines operate without the need for constant human input and direction, displaces attention. When something is automated, I can pay much less attention to it, or perhaps, no attention at all. We might also say that automation displaces will or volition. When a process is automated, I don’t have to will its action.
Finally, animation– the stage at which machines not only act without direct human intervention, but also “learn” and begin to “make decisions” for themselves–displaces agency and judgment.
By noting what is displaced we can then ask whether the displaced element was an essential or inessential aspect of the good or end sought by the means, and so we might begin to arrive at some more humane conclusions about what ought to be automated.
I’ll leave things there for now, but more will be forthcoming. Right now I’ll leave you with a couple of questions I’ll be thinking about.
Also, coming back to Arendt, she laid out two sets of three categories that overlap in interesting ways with the three stages as I’m thinking of them. In her discussion of human doing, she identifies labor, work, and action. In her discussion of human thinking, she identifies thought, will, and judgment. How can her theorizing of these categories help us understand what’s at stake in drive to automate and animate?
More than likely, you’ve recently heard about Facebook’s experiment in the manipulation of user emotions. I know, Facebook as a whole is an experiment in the manipulation of user emotions. Fair enough, but this was a more pointed experimented that involved the manipulation of what user’s see in their News Feeds. Here is how the article in the Proceedings of the National Academy of Sciences summarized the significance of the findings:
“We show, via a massive (N = 689,003) experiment on Facebook, that emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness. We provide experimental evidence that emotional contagion occurs without direct interaction between people (exposure to a friend expressing an emotion is sufficient), and in the complete absence of nonverbal cues.”
Needless to say (well except to Facebook and the researchers involved), this massive psychological experiment raises all sorts of ethical questions and concerns. Here are some of the more helpful pieces I’ve found on the experiment and its implications:
- “Everything We Know About Facebook’s Secret Mood Manipulation Experiment” by Robinson Meyer
- “Social Media as Political Control: The Facebook Study, Acxiom, & NSA” by David Golumbia
- “The Empire Strikes Back” by Alan Jacobs (read the exchange in the comments too)
- “Facebook Does Mind Control” by Michael Gurstein
Update: Here are two more worth considering:
- “Data Science: What the Facebook Controversy is Really About” by Sara Watson
- “What Does the Facebook Experiment Teach Us?” by Danah Boyd
four six pieces should give you a good sense of the variety of issues involved in the whole situation, along with a host of links to other relevant material.
I don’t have too much to add except two quick observations. First, I was reminded, especially by Gurstein’s post, of Greg Ulmer’s characterization of the Internet as a “prothesis of the unconscious.” Ulmer means something like this: The Internet has become a repository of the countless ways that culture has imprinted itself upon us and shaped our identity. Prior to the advent of the Internet, most of those memories and experiences would be lost to us even while they may have continued to be a part of who we became. The Internet, however, allows us to access many, if not all, of these cultural traces bringing them to our conscious awareness and allowing us to think about them.
What Facebook’s experiment suggests rather strikingly is that such a prosthesis is, as we should have known, a two-way interface. It not only facilitates our extension into the world, it is also a means by which the world can take hold of us. As a prothesis of our unconscious, the Internet is not only an extension of our unconscious, it also permits the manipulation of the unconscious by external forces.
Secondly, I was reminded of Paul Virilio’s idea of the general or integral accident. Virilio has written extensively about technology, speed, and accidents. The accident is an expansive concept in his work. In his view, accidents are built-in to the nature of any new technology. As he has frequently put it, “When you invent the ship, you also invent the shipwreck; when you invent the plane you also invent the plane crash … Every technology carries its own negativity, which is invented at the same time as technical progress.”
The general or integral accident is made possible by complex technological systems. The accident made possible by a nuclear reactor or air traffic is obviously of a greater scale than that made possible by the invention of the hammer. Complex technological systems create the possibility of cascading accidents of immense scale and destructiveness. Information technologies also introduce the possibility of integral accidents. Virilio’s most common examples of these information accidents include the flash crashes on stock exchanges induced by electronic trading.
All of this is to say that Facebook’s experiment gives us a glimpse of what shape the social media accident might take. An interviewer alludes to this line from Virilio: “the synchronizing of collective emotions that leads to the administration of fear.” I’ve not been able to track down the original context, but it struck me as suggestive in light of this experiment.
Oh, and lastly, Facebook COO Sheryl Sandberg issued an apology of sorts. I’ll let Nick Carr tell you about it.
A follow-up to my last post is still forthcoming. In the meantime, here are a few links for your reading pleasure.
At The New Atlantis, Joshua Schulz writes about “Machine Grading and Moral Learning.” The essay is a critique of machine-based grading of student essays, but it ranges widely and deeply in its argument. Here’s an excerpt:
“The functional- and efficiency-centric view of technology, and the moral objections to it, have been around for a long time. Look to the tale of John Henry, the steel-driving man of American folklore who raced a tunnel-boring steam engine in a contest of efficiency, beating the machine but dying in the attempt. The moral of the tale is not, of course, that we will always be able to beat our machines in a fair contest. Rather, the contest is a tragic one, highlighting a cultural hamartia, namely, the belief that competing with the steam engine on its own terms is anything other than degrading.”
A post at Librarian Shipwreck asks, “Whose Afraid of General Ludd?” You may remember that Borg Complex symptoms include, “Uses the term Luddite a-historically and as a casual slur.” That observation is echoed here:
“Whenever the term ‘Luddite’ appears as an insult it acts less as a reflection of the motives of those being slurred and more as a reflection of the fears of the person delivering the insult. But far from undermining Luddism, all that these insults do is underscore the tremendous power that a critique of technology couched in ‘commonality’ can still command.”
Read the whole thing for a historically grounded look at the Luddites and their motives.
Relatedly, here is a video and transcript of an interview with the late Jacques Ellul posted at Second Nature Journal. His insights resonate still. Here are two from the interview:
“Technology also obliges us to live more and more quickly. Inner reflection is replaced by reflex. Reflection means that, after I have undergone an experience, I think about that experience. In the case of a reflex you know immediately what you must do in a certain situation. Without thinking. Technology requires us no longer to think about the things. If you are driving a car at 150 kilometers an hour and you think you’ll have an accident. Everything depends on reflexes. The only thing technology requires us is: Don’t think about it. Use your reflexes.
Technology will not tolerate any judgment being passed on it. Or rather: technologists do not easily tolerate people expressing an ethical or moral judgment on what they do. But the expression of ethical, moral and spiritual judgments is actually the highest freedom of mankind. So I am robbed of my highest freedom. So whatever I say about technology and the technologists themselves is of no importance to them. It won’t deter them from what they are doing. They are now set in their course. They are so conditioned.”
Keep that last paragraph in mind as you read this last story: “For One Baby, Life Begins with Genome Revealed.”
If you’re paying any attention at all to news coming out of the tech sector, it’s hard to go a day without coming across a story about a new robot, app, or tool that promises (or threatens) to do what we previously did for ourselves. Some of these tasks involve physical labor, but increasingly they involve cognitive, emotional, and even ethical labor. Thinking carefully about the implications of this trend is, in my view, one of the most important tasks before us.
Taking a comment from Adam Thierer on a recent post about “smart-homes” as my point of departure, I propose that we think about the trend described above as a three-step process aimed at the enchantment of the human-built world.
In Adam’s view, as I read his comment, the “smartness” of the “smart home” is simply an extension of the many ways that we have already automated household tasks over the course of the last 100 years or so. Moreover, to my claim that a “smart home” is “an active technological system that requires maintenance and regulation,” Adam commented, “Even in the days of mud huts and log cabins that was somewhat true.”¹
In my initial response to Adam, I suggested that while it is true that more primitive homes, huts and cabins if you will, involved technology and might even be considered technological systems if we press the semantic range of that phrase a bit, there were important discontinuities as well. To clarify that claim, I began by making some distinctions using home heating as an example.
For most of human history, if I wanted warmth in my home I would need to build and sustain a fire of some sort. I could, for example, build a fire in a fireplace or I could light one in a coal burning stove. This would require a good deal of effort and caution. In short, it required a significant amount of engagement on my part, physical and mental.
Then along came the furnace and central heating. I no longer needed to build and sustain a fire. I could simply flip a switch and a machine would generate the heat and disperse it throughout my home. But I would hesitate to call this an instance of automation. Instead, let’s call it mechanization. Central heating, machine heating if you will, mechanizes the work of providing heat. And, of course, with that mechanization comes far less engagement on my part. In fact, this initial step is probably the most obvious and striking point of discontinuity in the evolution of home heating.
Add a thermostat and I no longer need to actively monitor the temperature in order to keep my home comfortably warm. I can set the thermostat at a toasty 73 degrees and trust the system to do the work for me. Ease of use continues to advance as my degree of engagement diminishes. Now, I think, we can talk about automation. Of course, thermostats of varying degrees of sophistication are available. The simplest models allow you to set just one temperature and require you to manually change that setting if you want the temperature to adjust over the course of the day. More elaborate, digital thermostats allow you to program a series of temperature changes throughout the course of the day and for different days of the week. All of these, however, allow me to automate the functioning of the machine. But, we should note, altering these settings still required direct action on my part.
It seems that the next step in this progression is something like Nest, a thermostat that “learns” your preferences and takes over management of the temperature for you. Nest is illustrative of the “smart” trend that promises something more than simple automation. Tools like Nest take automation further along the path toward autonomous functionality by “learning” to regulate themselves. Nest can also be controlled with a smart phone. In other words, it can be networked; it can “talk” to other devices. It is, then, a potential component of the assemblage of technologies that together constitute the so-called Internet of Things, one manifestation of which is the “smart home.” At this point, I am thoroughly disengaged from the process of providing heat for my home. I don’t need to cut wood or start a fire. I don’t need to flip a switch. I don’t need to adjust controls. Without labor, attention, or decisions on my part, my home is comfortably heated.
I remain uncertain about what to call this last step. Mechanization, automation, and … what?
In my previous post, I couldn’t quite resist an allusion to the famous scene in Boris Karloff’s Frankenstein where Dr. Frankenstein shouts, “It’s alive. It’s alive!” The allusion suggested “animation” a name for the third step after mechanization and automation. That strikes me as a provocative and vivid word choice, but it also threatens to mystify more than it clarifies. I mean the term in a figurative sense, but it may be too easy to suppose that something more literal is intended. Nonetheless, throwing caution to the wind, I’m going to go with animation, at least for the time being. Blogging is nothing, if not provisional, right?
So then, we have three discernible stages–mechanization, automation, animation–in the technological enchantment of the human-built world. The technological enchantment of the human-built world is the unforeseen consequence of the disenchantment of the natural world described by sociologists of modernity, Max Weber being the most notable. These sociologists claimed that modernity entailed the rationalization of the world and the purging of mystery, but they were only partly right. It might be better to say that the world was not so much disenchanted as it was differently enchanted. This displacement and redistribution of enchantment may be just as important a factor in shaping modernity as the putative disenchantment of nature.
In an offhand, stream-of-consciousness aside, I ventured that the allure of the smart-home, and similar technologies, arose from a latent desire to re-enchant the world. I’m doubling-down on that hypothesis. Here’s the working thesis: the ongoing technological enchantment of the human-built world is a corollary of the disenchantment of the natural world. The first movement yields the second, and the two are interwoven. To call this process of technological animation an enchantment of the human-built world is not merely a figurative post-hoc gloss on what has actually happened. Rather, the work of enchantment has been woven into the process all along.
In support of this claim we might consider, first, the entanglement of technology and magic just as the process of disenchantment is taking off in the early-modern period as well as the pervasive and, secondly, the persistent presence of the religion of technology within the western technological project.
I’m going to leave it at that for now. In a subsequent post, I’ll bring Hannah Arendt’s discussion of labor, work, and action into the discussion to help us think about the trade-offs involved in this enchantment of the human-built world.
[Discussion continued here: More On Mechanization, Automation, Animation.]
¹ There is a methodological question lying beneath the surface of this exchange: how do we wisely weigh the relevant degrees of continuity and discontinuity with older technology as we think about new technology? This can be tricky. There can be a tendency to exaggerate either the continuity or the discontinuity. In both cases, there would be nothing at all to learn because either nothing has changed and thus nothing needs to be learned, or else everything has changed and nothing of use can be learned. In the most unhelpful cases, both exaggerations are simultaneously affirmed. To generate hype, proponents of a new technology breathlessly proclaim its revolutionary character while at the same disingenuously allaying criticism by insisting the new revolutionary technology is really just like any number of other technologies that preceded it.