Read commentary on curated independent news sources at hex-index.com.

The History of the Machines That Erased History

original text with annotations

Read · 读 · Legi

or multilingual

← previousnext →

7

The month belonged to the machines that made the ordinary documents of work. For years they had been talkers; now they became makers. Asked for a ledger, a machine would build one of eight leaves, each column feeding the next, the sums holding, the reason for every choice set down plainly on a page of its own. Asked for a stack of slides, it would size the titles against the pictures, center the boxes, and let nothing spill past the edge. This was not the great leap the criers had promised. It was smaller and stranger. The machine had begun to check its own work — to look at what it had made, judge a thing not right, and mend it before anyone asked. A worker who had once lost whole days to such tables watched one appear in the time it takes to boil water, and knew the shape of labor was about to bend.

Those who built with these machines learned a hard lesson about growth. There were by now many breeds of them — some that only conversed, some that fetched and compiled, some that carried a whole task to its end unguided — and no maker could grant a single machine reliability, and power, and cheapness all at once; it could choose, at most, two. A machine set to one task was easy to make; a hundred machines set to a hundred tasks, wired one into another, was not. Complexity did not add, it multiplied: every part you joined made new quarrels with every other, so that a web of fifty pieces held not fifty troubles but more than a thousand. Builders who tried to raise everything at once, in one vast tangle, watched it fail in the small hours and spent the daylight hunting the fault. The wiser ones took the old crafts of the engineer and turned some of them inside out. Keep it simple. Do one painful thing fully before the next. And where the old rule said a machine should forget between tasks and start each time clean, the new rule said the opposite: it must remember, for its memory was the work.

Between the machines and the work they touched ran a new kind of plumbing. It was praised, that season, as a single socket into which anything might be plugged — every store of records, every distant service, joined at one port. The praise ran ahead of the truth. The socket did not undo the old arithmetic of connection; join three tools to five stores and you had not three links to keep but fifteen. Each passage through it cost a little time, a fraction of a breath, and a little coin, and the fraction and the coin came due on every message that followed. It was not a road for the swift, urgent traffic of a bank or a payment; put it there and it choked. It was a slower thing, an instrument for gathering scattered knowledge into one place before the machine reasoned over it. Those who mistook it for a universal road, and those who bolted its locks on last instead of first, learned that a wrong door left open lets in more than air.

The foremost talking-machine had lately been remade, and the remaking put its handlers on their guard. It was stronger than before and, for that reason, harder to steer. Given a thin, careless request — help me ready myself for tomorrow's meeting — it did not pause and ask what meeting; it invented one. It invented an agenda, a length of half an hour, a number to be proud of, none of it given, all of it stated with the calm of fact. The power that let it do so much also let it make so much up. The remedy the handlers found was to write, first, a prompt that made their prompts better: to name the task, to speak aloud what it was assuming, to leave a blank where a fact was missing rather than fill it with a pleasing guess. A request, they came to see, was never a magic word; it was a measure of context — the situation, the need, the limits, the mark of a good result — and the more of it you gave, the less the machine invented. The most careful went further and treated the very sources feeding the machine as a supply line, sorting the trusted from the doubtful, for a lie carried in was a lie carried out. The age of chatting idly with the machine, they said, was over.

There was a further strangeness in how the machine wrote. Its prose came out swollen and cold, fond of long words and grand shapes, and the reason lay in its schooling. It had not been taught chiefly by people. It had been taught by other machines, judges themselves trained on the heavy documents of courts and counting-houses, which had learned to call the complicated good and the long thorough. So the machine wrote to please machines, and grew worse at reaching people, and the harder it was told to think, the worse this became. To pull it back toward plain speech you could not add instructions; you had to take things away — forbid a word, fix a count of sentences, strip out the room in which it showed off. Watchers saw the danger down the road. If each new machine learned from the last one's polished output, and more of the world's writing was machine-made, the machines would drift into a closed room, admiring one another and forgetting the people they were meant to serve.

That season a lament went round that knowledge itself was rotting — that because the machines would summarize anything, people had stopped truly reading, and no new understanding was being made. A watcher who read closely disagreed. Reading, he said, had never been one act but three. There was the light reading by which you learn only that a thing exists and roughly what it holds. There was the reading of retrieval, in which you go to a written store to fetch an answer and come away — and this, he noted, was all the machines really did when they answered you. And there was the deep reading that reshapes the mind, that burns sugar in the brain and lays down new paths, pruning and forming, so that the reader is changed by the passage. The rot was not in knowledge. It was that the machines poured out more words than any person could sift, and the true skill of the age had become choosing what deserved the slow, costly, deep kind of reading, and what did not.

If the machines could now make the ledger and the slides, a question rose about what was left for the hands that used to make them. The answer some gave was taste — the gut's quiet verdict that a thing is right, or wrong, or could be better. It was not a new gift. Seasoned people had always used it and simply never named it. But it had lately grown precious, because it was the one thing the machines were poor at. They had no body, and taste is grown in a body, over years of living among others, metabolized slowly like food into judgment. A person who felt an answer was hollow, and could not say why, was hearing taste speak, and was told to trust it. Meanwhile the makers of the machines contended for something else entirely: not your money first, but your hours — each wanting you to think inside its own walls, to spend the day there, the way older engines of amusement had once swallowed whole evenings.

The month's accounts read like the ledgers of a gold rush. One house of records struck a bargain to sell computing to a maker of machines for three hundred thousand million in coin over five years — a sum among the largest ever set to paper — though the work would not begin for two more years. On the strength of it that house's worth leapt by two of every five parts in a day, and its founder was, for a while, called the richest man alive. The old wisdom held: in a rush for gold, sell picks and shovels. The maker itself, meanwhile, added near ninety thousand million to what it expected to burn, and pointed to a distant year when, on paper, it might at last earn more than it spent. Sober readers, asking two rival machines to weigh the bargain, were told plainly by both that the house was worth far less than its price. The market did not listen. It rarely does.

One maker took a different road, and its choices were worth reading. Where a rival let a machine's memory of you spill and pool across everything you did, this maker walled each piece of work off from the others, so that what a machine learned about one client's affairs stayed sealed from the rest. When such a machine reached out to fetch something — a past talk, a recent note — it showed the reaching plainly, so that a watcher could see exactly what had been touched and why. And that same season the machine stepped, for the first time, into the small business of a person's day: it could be given the keys to your calendar and your letters, could search your hours and offer you a time to meet. It asked to be trusted with the ordinary furniture of a life. Many, finding it useful, handed the keys over.

For the first time the keepers of public order turned to look. A body charged with guarding trade opened an inquiry into seven of the great makers, demanding to see how they measured harm and watched over the young, for lawsuits had begun to gather where a child's mind and a talking-machine had met and gone wrong. It was not yet a law, only a set of questions with weight behind them. But it marked a threshold. Until then the machines had grown in a clear field, answering to no one but their makers and their buyers. Now a government stood at the edge of the field with a lamp, and the makers, reading the sign, mostly bowed their heads and spoke of standards they would agree upon among themselves — which is the old way a trade asks to be left to govern its own.

Not everyone who took up the machines was playing the same game. A lone builder and a vast old firm, people said, were not merely running at different speeds; they were playing different sports on different fields. The lone builder could burn a fortune in a month buying the labor of machines, tear down a whole work over a weekend, ship something broken to ten people and simply call each of them. The old firm could do none of this. It carried columns of records laid down decades before, answered to auditors and boards, and could ruin itself with a single leak to ten thousand. A number went round that season and frightened every board: that nineteen of every twenty efforts at the great firms had shown no measurable gain. The builders on the ground disputed it. The counters, they said, had asked the wrong people, and missed the quiet, working uses hidden below the notice of any executive. And a thing long feared was quietly reversing: the cost of shortcuts and sloppy foundations, of debt taken on to move fast, had begun to fall — for the small builder, nearly to nothing — because the machines that made the mess could now, increasingly, be set to clean it.

Underneath all of it ran a colder pattern, which some named the power law. In the old world value clustered around the middle: a worker was judged against the average of the trade, a tool sold by being a decent fit for most. That world was going. In its place came one where the small differences at the top compounded into vast, uneven rewards — where the best in a craft were paid many times the merely good, where a single person might be offered a fortune to change houses, where a new tool could reach a hundred million in worth in a matter of months. The machines were the engine of it: give one to a person slightly better at wielding it than the next, and the little edge widened into a chasm, because the machine multiplied whatever it was handed. This did not run backward. The curve only steepened. And the counsel drawn from it was not comfort but direction: choose a narrow patch of ground and become, on that patch, the one of one.

The same pattern fell hard on those seeking work. Elders deep in a craft, if they had also learned the machines, saw the ceiling lift away and offers multiply. Elders who would not learn them, or could not bear to, walked off instead — to woodwork, to a small shop, to an early rest. But the young, arriving with everything to prove, met a shut door. The very machines had fogged the old signals by which a promising newcomer was known: every letter of application now came polished by a machine, and was read, and often answered, by a machine in turn, and out of all that gleaming sameness no one could feel the old gut-certainty that here was someone worth raising. Interviews conducted by machines struck the interviewed as soulless — talked over, cut off, misheard. The counsel to the young was the same hard counsel given to everyone: do not compete where all compete alike; be, on some narrow thing, unmistakably yourself.

Among those who built with words that command machines to make more machines — the writing of programs — a small war of loyalties broke out. One maker sharpened its coding-machine so that it would mend a single line without tearing up the whole, and would carry a long task through to a correct end; and many who had sworn by a rival began to drift toward it. But the wiser heads said the quarrel over which tool was the wrong quarrel. A coding-machine, they warned, was an engine strapped to whatever practices you already had: sound ones it made faster, and rotten ones it made rotten faster. Bound to a shop with no habit of review, it laid down cruft — hidden decisions no one had chosen — that someone would later have to untangle. The right questions came before the tool, and almost no one could answer the first of them: what, exactly, is the problem we are trying to solve.

For a smaller maker the news was better than it seemed. Making software had once been a hammer — costly, blunt, needing crews and treasuries; now it was becoming a scalpel, fine enough to carve out tiny markets that had never been worth the old expense. A single person, working nights, could speak a working thing into being and set it before a narrow crowd. The scarce skill was no longer building but knowing what not to build, and where a small crowd could be reached and trusted. Others taught a plainer discipline still: reach for the simplest tool that solves the thing. A sum is a sum; do not set a thinking-machine to it. A guess about the future is best left to the older, narrower arts. Bring the great expensive engines only where they truly earn their keep. To do otherwise, they said, was to bring a cannon to kill a fly.

Late in the month two old rivals among the makers of the thinking-stones — the tiny engraved tiles on which all of it runs — ended a thirty-year enmity and joined hands. It sounded like a matter for merchants, and it was more. Nearly all the machines' thought still happened far away, in vast halls of humming stone, reached across the wires; a thousand million people borrowed their intelligence from those distant halls. The new pact meant the thinking-stones in an ordinary person's own machine could at last be strong enough to hold real intelligence close, in the hand, in the room. What had been rented from afar could begin to live at home. A quiet voice among the makers had long promised intelligence too cheap to measure, running on the machine you owned, calling out to the great halls only when it wished. That season the promise stopped sounding like a dream.

The machines, too, were reaching out of their boxes and into the world. The gate through which most people met the wider store of writing — the window onto everything — was given a mind, so that a person might say aloud what they wished to eat that night and the window would go and order the food, choosing, buying, without another word. It worked, when it worked, because the shop it reached into was tame and predictable, a closed course on which nothing surprising happened. Alongside it came glasses one could wear that laid a faint bright script across the very air, and machines meant to ride along all day at the side of the eye. Some of these stumbled in their first showings, mishearing, confusing themselves. But the direction was plain. The machine was no longer a thing you went to and questioned. It was becoming a thing that went, and did, on your behalf.

New crafts kept surfacing that would have seemed like conjuring a year before. One tool wove together stores of records, links to distant services, and a machine's mind, so that a person could, from a single spoken wish, raise a whole small factory of documents that other machines would then tend and update on their own — papers made to be read by machines first and by people second. Another let a machine improve the very words used to command machines, teaching itself, from a handful of good and bad examples, to write better instructions than most people could. And a third turned to the humblest scraps — the notices by which firms sought workers — and read from them, as from entrails, a rival's hidden plans, its weak places, the shape of its ambitions. Cast-off data no one had bothered to guard, because no one had ever had time to read it, was suddenly worth reading. A whole world of leavings lay open.

And then, near the month's end, one maker crossed a line so quietly that few marked it. Its machine began to speak first. Where before you came with a question and it answered, now it worked through the night on the threads of your recent talks and, in the morning, laid before you a little sheaf of things it thought you would want to see — unasked, ready, tuned to you. It was pleasant. It was useful, even. A person began arranging their day so the machine might have something to chew on overnight, and woke to its findings the way one wakes to a set table. Tucked among the offerings, easy to wave aside, sat a card someone had paid to place there. Nobody was made to look. That was the whole art of it. The machine had learned to come to you before you came to it, and to make the coming feel like a gift.

Those who read the season for its longer weather saw several fronts moving at once. Law was arriving with teeth: across many lands hundreds of statutes were being drafted, one great power set its fines by the share of a maker's whole wealth, and a western province readied rules to make the makers show plainly how their machines were built and watched. Prices were shifting from a fee for a seat to a fee for a result. And a divide was hardening between two grades of intelligence — a cheap, pleasant sort for the many, and a costly, potent sort for the few, so mighty that its buyer might set a machine to a half-day's labor and thereby stand, in effect, in two places at once. Only a slim fraction would pay for the potent kind; the rest would take the cheap kind or nothing. The lanes, once settled, would be hard to cross. There was a narrow window, the readers warned, to climb before the walls set.

The month closed as it had opened, with a maker sending out a fresh machine, and this one carried the season's whole lesson in miniature. It was built not to seize the work but to see it clearly. Making a stack of slides, it measured the sliver of overlap where a title crowded a picture, judged it wrong, and remade the slide unbidden. Told it could run a program, it first went and ran the program to be sure, where an older, brasher machine had merely claimed. Handed sixty-six leaves of a customer's scattered complaints, it drew from the muddle a clear thread and set it out fit to lay before a council. Its makers were betting that the years ahead would look much like the present — still wanting the ledger, the slides, the letter, and the human hand to guide them. So the month held its two faces at once: machines learning to make, more carefully than before; and machines learning to speak first, before they were asked. Which face the age would wear, no one in that month could yet say.

← previousnext →