Imagine you’re taking an online business class — the kind where you watch video lectures and then answer questions at the end. But this isn’t a normal class, and you’re not just watching the lectures: They’re watching you back. Every time the facial recognition system decides that you look bored, distracted, or tuned out, it makes a note. And after each lecture, it only asks you about content from those moments.
This isn’t a hypothetical system; it’s a real one deployed by a company called Nestor. And if you don’t like the sound of it, you’re not alone. Neither do the actual students.
When I asked the man behind the system, French inventor Marcel Saucet, how the students in these classes feel about being watched, he admitted that they didn’t like it. They felt violated and surveilled, he said, but he shrugged off any implication that it was his fault. “Everybody is doing this,” he told me. “It’s really early and shocking, but we cannot go against natural laws of evolution.”
As a reporter who covers technology and the future, I constantly hear variations of this line as technologists attempt to apply the theory Charles Darwin made famous in biology to their own work. I’m told that there is a progression of technology, a movement that is bigger than any individual inventor or CEO. They say they are simply caught in a tide, swept along in a current they cannot fight. They say it inevitably leads them to facial recognition (now even being deployed on children), smart speakers that record your intimate conversations, and doorbells that narc on your neighbors. They say we can’t blame these companies for the erosion of privacy or democracy or trust in public institutions — that was all going to happen sooner or later.
“When have we ever been able to keep the genie in the bottle?” they ask. Besides, they argue, people buy this stuff so they must want it. Companies are simply responding to “natural selection” by consumers. There is nobody to blame for this, they say. It’s as natural as gravity.
Perhaps no one states this belief more clearly than inventor and futurist Ray Kurzweil in his 2005 book The Singularity Is Near: “The ongoing acceleration of technology is the implication and inevitable result of what I call the law of accelerating returns, which describes the acceleration of the pace of and the exponential growth of the products of an evolutionary process.”
In fact, our world is shaped by humans who make decisions, and technology companies are no different.
To claim that these devices are the result of some kind of ever-improving natural process not only misunderstands how evolution works, but it also suggests that everything from biological weapons to fraudulent startups like Theranos to Juicero (the $400 machine that squeezed juice out of packets) are necessary and natural.
While these “innovations” range from the dangerous to the silly, they share a common thread: Nothing about them is “natural.” No natural process is creating a “smart” hairbrush or a “smart” flip flop or a “smart” condom. Or a Bluetooth-enabled toaster, a cryptocurrency from a photography company, or an internet-connected air freshener.
Evolution is a terrible metaphor for technology
Technologists’ desire to make a parallel to evolution is flawed at its very foundation. Evolution is driven by random mutation — mistakes, not plans. (And while some inventions may indeed be the result of mishaps, the decision of a company to patent, produce, and market those inventions is not.) Evolution doesn’t have meetings about the market, the environment, the customer base. Evolution doesn’t patent things or do focus groups. Evolution doesn’t spend millions of dollars lobbying Congress to ensure that its plans go unfettered.
In some situations, even if we can’t literally put a technological genie back in a bottle, we can artificially intervene to make sure the genie plays by specific rules.
There are clear laws about what companies can and can’t do in the realm of biological weapons. The FDA ensures drugs are tested for efficacy and safety before they can be sold. The USDA ensures new food research is done with care. We don’t let anybody frack or drill for oil or build nuclear power plants wherever they like. We don’t let just anybody make and sell cars or airplanes or guns.
So the assertion that technology companies can’t possibly be shaped or restrained with the public’s interest in mind is to argue that they are fundamentally different from any other industry. They’re not. (...)
This endless, punishing race in the name of “progress” is often what drives consumer behavior, too. Despite the “American dream” — security, safety, prosperity — being more and more out of reach for everyday Americans, the idea that it’s just around the corner drives people to purchase these products.
If you have the newest app, people think their lives will be easier, you’ll have more free time, more quality time. Commercials promise more backyard barbecues under sparklers and birthday surprise parties facilitated by internet-connected light bulbs.
And when we buy the products, tech companies take that as a green light to continue on their “inevitable” path, inching ever toward a world where Amazon knows exactly what you’re doing, thinking, feeling — perhaps even before you do. “It’s all a loop,” says Stark. “It’s weird. That’s what puts people in this bind. They think they should be able to have it all. They can’t, and technology is a kind of prophylactic to cope with this stuff.”
by Rose Eveleth, Vox | Read more:
Image: Zoë van Dijk
This isn’t a hypothetical system; it’s a real one deployed by a company called Nestor. And if you don’t like the sound of it, you’re not alone. Neither do the actual students.
When I asked the man behind the system, French inventor Marcel Saucet, how the students in these classes feel about being watched, he admitted that they didn’t like it. They felt violated and surveilled, he said, but he shrugged off any implication that it was his fault. “Everybody is doing this,” he told me. “It’s really early and shocking, but we cannot go against natural laws of evolution.”
As a reporter who covers technology and the future, I constantly hear variations of this line as technologists attempt to apply the theory Charles Darwin made famous in biology to their own work. I’m told that there is a progression of technology, a movement that is bigger than any individual inventor or CEO. They say they are simply caught in a tide, swept along in a current they cannot fight. They say it inevitably leads them to facial recognition (now even being deployed on children), smart speakers that record your intimate conversations, and doorbells that narc on your neighbors. They say we can’t blame these companies for the erosion of privacy or democracy or trust in public institutions — that was all going to happen sooner or later.“When have we ever been able to keep the genie in the bottle?” they ask. Besides, they argue, people buy this stuff so they must want it. Companies are simply responding to “natural selection” by consumers. There is nobody to blame for this, they say. It’s as natural as gravity.
Perhaps no one states this belief more clearly than inventor and futurist Ray Kurzweil in his 2005 book The Singularity Is Near: “The ongoing acceleration of technology is the implication and inevitable result of what I call the law of accelerating returns, which describes the acceleration of the pace of and the exponential growth of the products of an evolutionary process.”
In fact, our world is shaped by humans who make decisions, and technology companies are no different.
To claim that these devices are the result of some kind of ever-improving natural process not only misunderstands how evolution works, but it also suggests that everything from biological weapons to fraudulent startups like Theranos to Juicero (the $400 machine that squeezed juice out of packets) are necessary and natural.
While these “innovations” range from the dangerous to the silly, they share a common thread: Nothing about them is “natural.” No natural process is creating a “smart” hairbrush or a “smart” flip flop or a “smart” condom. Or a Bluetooth-enabled toaster, a cryptocurrency from a photography company, or an internet-connected air freshener.
Evolution is a terrible metaphor for technology
Technologists’ desire to make a parallel to evolution is flawed at its very foundation. Evolution is driven by random mutation — mistakes, not plans. (And while some inventions may indeed be the result of mishaps, the decision of a company to patent, produce, and market those inventions is not.) Evolution doesn’t have meetings about the market, the environment, the customer base. Evolution doesn’t patent things or do focus groups. Evolution doesn’t spend millions of dollars lobbying Congress to ensure that its plans go unfettered.
In some situations, even if we can’t literally put a technological genie back in a bottle, we can artificially intervene to make sure the genie plays by specific rules.
There are clear laws about what companies can and can’t do in the realm of biological weapons. The FDA ensures drugs are tested for efficacy and safety before they can be sold. The USDA ensures new food research is done with care. We don’t let anybody frack or drill for oil or build nuclear power plants wherever they like. We don’t let just anybody make and sell cars or airplanes or guns.
So the assertion that technology companies can’t possibly be shaped or restrained with the public’s interest in mind is to argue that they are fundamentally different from any other industry. They’re not. (...)
This endless, punishing race in the name of “progress” is often what drives consumer behavior, too. Despite the “American dream” — security, safety, prosperity — being more and more out of reach for everyday Americans, the idea that it’s just around the corner drives people to purchase these products.
If you have the newest app, people think their lives will be easier, you’ll have more free time, more quality time. Commercials promise more backyard barbecues under sparklers and birthday surprise parties facilitated by internet-connected light bulbs.
And when we buy the products, tech companies take that as a green light to continue on their “inevitable” path, inching ever toward a world where Amazon knows exactly what you’re doing, thinking, feeling — perhaps even before you do. “It’s all a loop,” says Stark. “It’s weird. That’s what puts people in this bind. They think they should be able to have it all. They can’t, and technology is a kind of prophylactic to cope with this stuff.”
by Rose Eveleth, Vox | Read more:
Image: Zoë van Dijk
