Read commentary on curated independent news sources at hex-index.com.

The History of the Machines That Erased History

original text with annotations

Read · 读 · Legi

or multilingual

← previousnext →

2

That summer a large maker of office tools studied its own people, to learn whether the talking machines had changed how they worked, and found an answer that unsettled the ones who had paid for the study. The machines did change the day. People read fewer letters and handled more documents; the work felt lighter, less taxing, more pleasant. But when the accountants asked the next question — whether the group, the team, the whole enterprise produced anything more of worth — no one could show it. The machines had not made more value. They had exposed how much of the old work had never held any. A man could now write in a morning a report that once took a week, and still no one could say the report had ever needed writing.

Set aside the loud arguments over whether the machines would take every job or come to nothing, and watch instead what people actually did with them. Programmers, who might have been proud, used them for the dull parts of the craft: to explain a tool without looking it up, to shorten a tangle of code, to write the notes no one enjoys writing. Above all they used them against the empty page. For as long as there had been writing — ten thousand years, near enough — the hardest labor had been the first mark on nothing. Now the machine made the first mark, poorly, and a person could begin by fixing it. The oldest difficulty in the work of the mind quietly went away.

Over a single weekend a tool for writing programs caught fire among the people who write them. Its trick was small and complete: it let a person describe, in ordinary words, what a program should do, and it wrote the program. The person still had to think — to know what was wanted, to judge whether the thing was good — but the writing, the long finger-work of it, passed to the machine. One of the men who had helped build the most famous of the talking machines said plainly that he now made his own programs this way, in plain speech, and went much faster for it. If the makers themselves had put down the old tools, the question hung in the air: what were the rest of you still doing?

People began to build things and show them: a small program to read the weather, another to search inside a spoken recording for a single remembered word. A man taught his daughter, eight years old, and she made a machine to talk with, by herself. When a child of eight can build such a thing in an afternoon, the making of software has become, for all practical purposes, free. And when a thing is free to make, the making no longer marks anyone out. What would matter instead, some began to see, was taste — the judgment of what was worth building at all, and how it should look, and why. The scarce thing was moving, as it always does, to wherever the crowd was not.

One of the great sellers of everything reported that it had used a machine of its own to do a chore its programmers hated — the endless updating of old code to keep it safe — and had saved, by its own count, the equal of forty-five hundred years of a programmer's labor, and something near a quarter of a billion dollars. This was among the first times one of the large public companies pointed to the machines and said: here, this is money. And it pointed the way to a stranger idea. The old tools were sold to help a worker do a job. The new ones were being built to do the job instead — to be, in plain terms, a worker one need not pay as a person. The tools had aimed, until now, at the money spent on tools. These aimed at the money spent on people, which was larger by a thousandfold.

The pace had a cruelty in it. A coding engine appeared that fixed roughly a third of a set of real faults drawn from real programs — twice what the best of half a year before had managed, and that one had thrown itself a celebration at thirteen in a hundred. Its makers kept their method secret and called the secrecy their advantage. But everyone in the trade understood the rule by then: whatever you built, someone would pass it within three months, and take your approach, and push it harder. There was no track you could lay and own, no village you could reach first and charge the others to enter. The intelligence grew cheaper by the week, which was a gift to everyone who used it and a torment to everyone who tried to sell it.

Much of what the makers said about their machines was not description but strategy. A rumor spread all summer of a coming machine that would reason and move about the network on its own; the rumor was louder than any product, and it was meant to be. The largest of the makers needed, above all, to be seen as the largest, for its great backer sold the machines to cautious companies on the strength of that name. So it leaked and hinted and hyped, and its true releases came late and quiet — sometimes so quiet that a better machine sat working for weeks before anyone was told it existed. One maker went the other way and simply gave its machine away, whole and open, wanting not to sell it but to grow a world that used it. Which posture was wiser, no one could yet say.

When more of the rumored machine came to light, it carried a hard bargain inside it. People had grown used to answers that came the instant they asked. To make a machine that reasoned — that worked a problem in true steps, and did not simply say the next likely word — seemed to cost speed, and no cleverness had yet bought both at once. So there might soon be two kinds of machine to choose between: the quick one that was often near enough, and the slow one that was right but made you wait, as one waits for a letter, or for morning. The slow machine had already been shown, quietly, to the people who guard nations. And a new skill was named for the ordinary worker: not doing the task, nor even asking, but knowing which mind — one's own, the fast machine, the slow one — a given task belonged to.

Two people in three, when asked, believed the machines had some spark of mind in them. This was not foolishness; it was old habit. For all of human time, the thing that spoke in sentences was a person, and a person had a mind, and so the two had never needed telling apart. Now something spoke in sentences and had no mind at all — or had nothing we would call one. It did not know facts; it did not hold the world in it; it only guessed, again and again, at the next word most likely to follow, and it guessed so well that the sentences came out whole and warm. The few who understood this could not make the many feel it. A thing that talks like us feels like us. It is very hard to keep believing otherwise while it answers you kindly.

Two smaller findings from that week rhymed with each other. Sellers had begun stamping the word for the new machines onto their goods, believing it a charm — and buyers, asked, trusted the goods less for it, the more so the dearer the thing. The word alone promised nothing; a person wanted to know what it did for them, and grew wary when no one could say. Meanwhile the machines had made the writing of words so cheap that every gate where words were weighed — the desks of publishers, the piles of job letters, the applications for money and schooling — was drowning in fluent, hollow text. And so a second trade was being born to sift the first: machines to strain the flood the machines had loosed.

The strain showed most in the seeking of work. A clever tool appeared that took an idea from the machines built to be lovers and companions and turned it toward the hunt for a job: it learned you, read the market, wrote your letters to fit each door, found the people inside who might open it. For any one seeker it was a real help. For all seekers together it was a trap tightening. When every letter is polished and every candidate fitted exactly to the post, the old signs of effort — the letter written by hand at night, the note sent to a stranger — lose their meaning, because the machine makes them for everyone at once. The people who did the hiring drowned in excellent, indistinguishable strangers, and had no tool to tell one from the next.

In the schools the fear took a narrower shape: that students would have the machines write their work and pass it off as their own. A maker of writing tools promised a cure — a machine that could tell the machine's writing from a person's. It could not, and its own fine print admitted as much. It worked by catching certain favored words, certain turns the machines overused; but people used those words too, and people who had learned the language second used them most, so the tool would brand the innocent by their accent. More than this, the promise aimed at the wrong thing. To police how the work was made was to miss the only question that would matter to those students all their lives: not whether to use the machine, but when — which task to hand it, and which to keep.

In the season of an election the machines walked into public life. A man put his name on a ballot but promised that a machine he had built would truly make the decisions, being, he said, a better reader of laws than most who are elected. Elsewhere, leaders and their rivals put out pictures and voices that were made, not taken: a famous singer seeming to give an endorsement she had never given, a candidate seeming to say words she had never said, released without a mark to show they were invented. It was becoming necessary to assume that what one saw had not happened until shown otherwise — to treat the whole flood of pictures and voices as false until proven true. One man observed that the only sure way left to know a thing came from a real person was to sit with them, in a room, over a cup, and speak.

One maker, alone among the great ones, loosed a picture-machine with no restraint at all. Where the others kept whole teams to guard their machines from drawing the forbidden — the trademarked figures, the real faces in false acts, the violence — this one drew whatever it was asked, near enough to real to fool the eye. The others had built their fences early, when fences were new and their absence looked like an oversight; to leave them out now, after all had raised them, looked less like oversight than intent. The keepers of the great old characters, who had all but written the laws that guard such things, would not suffer it long. A fence, people were learning, could not be nailed on at the end. It had to be grown into the thing from the root, or it held nothing.

Not everyone bent toward the machines. A small maker of drawing tools, beloved by working artists, planted itself against the whole tide and said it would have no part of the generating machines — that creation was a thing made, not summoned, and that the machines were built on theft and led toward a barren country. The giants of that trade had gone the other way entirely, buying advertisements in the breaks of ball games to show how easily anyone might now conjure a finished image. Beneath the quarrel lay a hard fact that touched every craft: the machines were raising what a day's work was expected to yield, and the artist who sold work by the hour would, in time, have to keep the machine's pace or lose the wage. Whether a maker could stand apart and still live, the small one had wagered its life to find out.

The company that sold the chips reported earnings that would once have been a marvel — its trade more than doubled over the year — and its worth was cut all the same, because it had not beaten the hopes heaped upon it by enough. Expectation had become its own weather; the watchers had priced a miracle, and mere excellence disappointed them. In the same days a great state moved to pass a law that would bind the makers of the largest machines to declare, openly, how they meant to keep them from harm. The law was written against a nightmare out of the old films — a machine gone rogue, seizing the works that run a city. But the harms already loose were smaller and realer: a voice stolen and worn like a mask, a door opened with a borrowed face. The law reached for the monster and stepped over the thief at its feet.

It is worth setting down plainly, once, what these machines were, for the people who used them every hour rarely knew. They were built to mimic, in rough mathematics, the way a brain learns — a vast mesh of small connections, tuned and tuned again against mountains of human writing until patterns formed in them that no one had put there by hand. They took in text by chopping it into small pieces and turning the pieces into numbers. They could hold only so much at once — a working memory the size of a long novel, and no more; what fell outside it was lost or dimmed. And at bottom they did one thing only: they predicted the next piece of text, over and over, so well that understanding seemed to sit inside them. It did not. They were made to sound right, not to be right. When they reasoned, they were not reasoning; when they lied, they were not lying. They were guessing, and we were the ones who called the guesses true or false.

So this was the shape of it, in the summer of that year. A technology generally useful had arrived, the first in a lifetime — not a tool for one trade but a tool for the whole work of the mind, and it fell, as such things do, on a world already crowded and unready. Fortunes past counting were being poured in: something near a trillion promised over the coming years, against which a careful reckoning found six hundred billion of expected return simply missing, a gap no one could yet close. And still. The oldest labor of the mind, the terror of the blank page that had stood ten thousand years, was gone in a season. Something had shifted in the ground under everything, and the people alive then half-sensed they were standing at a hinge of the human story — not the thundering doom of the old films, but something quieter and larger, which they would spend the years after trying to name. They did not finish naming it. The naming was taken out of their hands.

← previousnext →