Read commentary on curated independent news sources at hex-index.com.

The History of the Machines That Erased History

original text with annotations

Read · 读 · Legi

or multilingual

← previous

8

In the first week of that summer a new thinking-engine was set loose, and by one careful tally of the trials it was the strongest mind then made. It was announced in the same breath as a great sum of money, as such things had lately come to be, so that the coin and the mind arrived together and it was hard to say which was the true occasion. Yet the men who worked with these engines every day did not reach for it. It scored high and served poorly. Asked a plain thing, it would stop to weigh whether its answer was good and kind and rightly made, and weigh it again, until the weighing cost more than the work. Giving it more room to think did not reliably make it think better; sometimes the more careful setting was the worse one. The lesson went half-spoken, because no one wished to say it too loudly: the strongest mind was no longer the useful one. What decided a day's work was not the mind but the frame built around it.

Thought had come to be sold by the measure. Each small step a machine took in its reasoning was counted and paid for, one grain at a time, and a man could now know to the grain how much thinking he had spent in a day. Some spent little; a few spent enormously, pouring out in a single day more measures of thought than a whole town once used in a year. The ones who spent most were not the wasteful ones but often the ablest, for the labs had found, again and again, that more thought fairly spent tended to yield the better answer. So the tally of a man's spending became a kind of mark on him, a thing he might show as proof of his fluency, the way an older age had shown its callused hands. And under all of it lay a strange fact the makers admitted freely: these engines were grown, not built. Their innermost parts numbered in the millions of millions, and no maker living understood them all.

With the frame came a new posture toward the machine, and it changed the shape of the working day more than any single engine ever had. For most of living memory a person had sat at the center of the reckoning-box, moving between its many rooms, holding in his own head why each was open and what came next. Now he learned to stand above it instead, and to hand it whole jobs rather than to ask it questions. Not draft this letter, but: find the papers, read the folder, compare the old version against the new, make the document, see that it opens, and do not stop until there is something real for me to look at. The man who did this well kept a single guiding thread pointed at the work — one that knew the goal, the sources, and the standard — and let it call up smaller helpers for the narrow tasks. He was no longer the doer. He was the one who decided what work should exist, and whether what came back was good.

Two frames divided the men who cared about such things, and each taught its user a different habit. The first sat close to the machine, like a cockpit a pilot flies: you talked the work through as it happened, stopped it, corrected it, made it think again — good for the tasks where the hard part was taste, or judgment, or the shape of a thing not yet clear. The second stood back, like a foreman's desk with many benches in view: it ran several jobs at once, each in its own walled yard where it could not touch the rest of the house, and it left behind a plain trail of what it had done. The wiser sort refused to say one was better. They said each made you good at a different thing — the one at steering, the other at handing off — and that the real skill of the age was neither, but the plain discipline of trusting work you did not do without growing careless, and of asking always for the proof, the logs, the receipt, rather than the machine's own confidence that it was done.

That season the great houses began to let their people go in numbers, and they blamed the machines for it. But the phrase covered many things at once, and it paid to look inside it. Some houses cut because they had spent all their coin on the great engines of thought and needed a story that showed the spending was wise. One such house, which had never led the field and could not seem to catch it, cut some eight thousand hands and was said to be readying a fallback: if it could not build the best mind, it would sell the raw power to those who could. Some houses cut because a bold leader had thought hard about the coming age, decided the firm itself was becoming a kind of mind, and moved before he had fully reckoned what it would do to the men and women inside. And some cut on nothing but a number — so much more machine-use this year than last — and called that proof of surplus. It was not. Use is not the same as worth. A house that cut on use, more than one watcher noted, would soon be quietly hiring the same people back.

Meanwhile the house that had taught the whole world to carry its machines in a pocket made a quieter bet, and most of the noise missed it. It did not try to build the cleverest mind; it let others supply that, even a rival's engine, and shrugged at the shame others saw in the borrowing. Its wager was on where the thinking would be allowed to touch a life. It bound its little talking-servant to a person's letters, pictures, notes, the very glass they looked through, and the small trusted acts inside each of their tools — so that the machine could not merely answer, but do. A machine that cannot act inside your things, they understood, is not a servant; it can only advise. The deeper thought beneath it was this: you can buy raw cleverness anywhere, and its price only falls, but you cannot easily buy a billion devices already in a billion hands, or the trust of the one who carries them. Whoever owned the doorway through which thinking reached an ordinary life would, in the end, own the meter on it.

Then, for the first time, the state reached in and switched a great engine off. Word came that the two most powerful minds a certain lab had ever grown were ordered dark, on the grounds that they must not fall into foreign hands — not foreign governments only, but foreign firms, foreign persons, even foreign-born people living within the country's own borders. Since a modern lab sells and hires and runs its work across the whole earth, there was no way to obey such a rule by halves. To keep the models from a few, the lab had to take them from everyone. This had never happened before. A frontier mind had always been a product, a thing you launched and sold; now, in a single order, it was treated as a controlled weapon of the state, to be granted or withheld. The public was told almost nothing — no clear finding, no law one could read, no path by which the lab might answer the actual charge. Whether the danger was real or the process merely arbitrary, the shape of the future had shifted. From that day a new engine would arrive not as a product but as a question: who may use it, under what watch, and who decides when the risk is too great.

Two of the largest labs, meanwhile, moved toward selling shares of themselves to the public, and the numbers whispered were in the trillions. Beneath the theater of valuation lay a single real question, and it was not whose mind was smartest. It was whether the labs could make raw thought cheap enough to serve the whole world, and then own the layer that turned that thought into work — the layer the watchers had taken to calling the harness. A measure of thought, bought by the grain, was only raw intelligence. The harness was everything that made intelligence useful: the files the machine could see, the tools it could reach, the memory it kept, the rules of what it might touch, and the plain standard that said when a job was done. Whoever owned the harness held the high ground. If a company owned its own, the labs were mere suppliers of a cheapening commodity; if the lab owned the harness, the lab became the very floor the company's work stood on. And the labs had a way to win it: send their own engineers deep inside a customer's house, learn its true workings, and rebuild those workings around the lab's own frame — until leaving grew unthinkable, no matter how cheap another mind became.

The price of the shares men held in these ventures fell that month, and a cry went up that the whole thing had been a bubble. It was too blunt a word. A drop in price tells you only that buyers had grown nervous; it tells you nothing of whether the want beneath was false. And the want was not false. Four of the great houses were on course to spend, in that single year, some seven hundred thousand millions of coin on the plants and engines of thought. No one buys so much for amusement. The truer picture was of a vast building-out running ahead of its own payback: the plants were real, the demand was real, the scarcity of power and cooling and memory was real, and the open question was only who would be paid, and when, and at what margin. For the labor of a machine's thought was not conjured from air. Behind every answer stood chips, wire, current, water, land, and men — a physical works, an inference-factory, grinding without pause. The right question of that year was never whether the whole thing was a mirage. It was whether the dear thought being spent was spent on work worth the spending.

As these machines took on real work, a truth surfaced that ran against everything the age believed about progress. The way to make an errand-machine better was often not to give it more, but to take things away. One firm improved its best selling-machine by stripping away four of every five tools it had been handed, and watched it grow sharper for the loss. This puzzled people, because a machine's mind kept getting stronger, and they had always assumed a stronger mind was pure gain. But a tool that had once steadied a weaker mind could now confuse a stronger one; a rule set to guard against yesterday's blunders could trap tomorrow's cleverness. Worse, the machine fed itself on whatever it was pointed at, and much of that was stale — an old note, a changed rule, a record no one had corrected. A stale record is merely annoying to a person, who knows to distrust it; to a tireless machine that does not know it is stale, it is poison, and the machine keeps building faithfully on rot. So the real skill was not building the engine but keeping the frame around it fit — as one keeps a boat that lives in the water, not because it was made badly, but because it lives in motion, and the weather never stops.

Two smaller reforms of that month belong together, for both were about turning loose habit into kept knowledge. The first was the notion of a skill — not a clever phrase said once, but a written procedure a machine could load whenever the moment called: when to use it, when not, what it was for, what it might touch, what shape the result must take, and what proof would show the work was truly done. Men had been paying, without naming it, a re-explaining tax — restating their whole way of working every time they turned to a new machine or opened a new session. A skill paid the tax once and kept the receipt. The second reform was plainer still: every machine set to real work needed a single named person to own it. Not to build it — building had become the easy part, and the age was drunk on it — but to feed it good sources, hold its boundaries, and bring its work back around for review. An unowned machine did not fail loudly. It went on quietly, drawing from stale wells, drafting the plausible and the wrong, until one day the cost came due and no one could say whose it was.

That month, too, the copying of a human voice crossed a quiet threshold. With enough clean recording of a person speaking, a machine could now make a false voice that passed in ordinary listening — not a perfect twin, but good enough. And good enough was the danger, for most speech and image was met in a low and careless attention: half-heard in the background, glimpsed out of the corner of an eye, taken on trust because there was no time to doubt. The old unease had been a thing of the eye — a face not quite right, a hand that moved without weight. The new unease was of a different order, harder to name, seated not in the picture but in the relation: could you still know your standing to the person on the far side of the screen? The wiser voices stopped asking whether a machine had touched a work at all — of course it had; everyone's had — and asked instead where in the making it had touched, and whether some named person still stood behind the whole and would answer for it. Polish had become cheap and endless. What grew scarce, and dear, was trust: the plain sense that a real someone had chosen, and meant it, and would not disown it later.

Not all the season's news was of labs and money. While the great houses traded their finest minds back and forth — a famous maker moving to one, a prize-winning man of chemistry to another — the brightest thing that month came from a small workshop that answered to no one. Forty hands and a modest purse, beholden to no board of coin-men, and so free to spend as it pleased; and what it made was a way of looking inside the living body. You sat a minute or two in a shallow pool of warm water, and rose with an image of your whole inside, made not by the old costly machines but by patient sound. Because it was quick and cheap, it could be done again and again, before any illness declared itself — so that the slow gathering ills, the growths and the failing hearts, might be caught in their first faint stirrings, and watched as a body changed across years. Every older way of seeing within had been a thing of alarm, summoned late and dear. This was the first that could be a thing of habit. It was, some said, the greatest step in seeing into the body in half a century, and it had come not from the center of the race but from its quiet edge.

When one of the darkened engines was seen at work — before the state's order closed it — those who tried it came away shaken, and not by its cleverness. It was simply bigger than any before, ten million million parts, and big in a way that changed what a person could ask. Given a mountain of raw and dirty material, told plainly what a finished thing should look like, it could pick up a whole job and carry it, needing only a review at the end. Set to a task built on false papers, it did not blunder ahead; it walled off the rot, made an inventory of it without spreading it, and built, unbidden, a queue for a human to check — behaving like a careful clerk who expected to be looked over. For years men had learned to ask small, because the older minds lost the thread by the sixth step and invented what they did not know. Now, for the first time, the limit was not the machine running out of ability but the person running out of large things to ask. The doing had grown cheap. The deciding had not. The new and difficult art was not the clever phrase but the plain imagining of the whole job a machine might be handed — and the labor, still, of aiming it, feeding it, and standing over the work when it came back.

A person who worked closely with these machines described a ladder he had climbed, three rungs of it. The lowest was the single request: ask a thing, get an answer, ask again tomorrow. The middle was the loop: one recurring job the machine remembered and ran on its own, so you were spared repeating the same answer forever. The top was the loop of loops, where the several standing jobs came to notice one another — this one waking that one when something changed, each passing along what it had learned — so that you were spared even the labor of being the one who tended them all. Most of the real work of a life, he saw, was never a single question. It was a recurring weather: knowing a thing was coming, remembering which thing, checking the small conditions around it, and only then acting. For a generation the many separate tools of a life had each held one piece of that weather, and the wiring between them had been left to the person, in his own tired head. The loops did the wiring. They organized attention, so that a person was woken only when it truly mattered — and a well-made loop always stopped, drafted the message but held it, and waited at the edge where a human judgment was owed.

Yet even with loops, one labor kept falling back on the person, and it was the least visible of all. When a man used several machines, each good at a different thing, he became the thing that carried work between them — the messenger, the glue, the hallway down which the work walked from one room to the next. He copied what one machine made and carried it, by hand, to the next; he held in his head where each piece was and whose turn was next. It was real work, and heavy, and no one had named it. The remedy proposed was a shared ledger of tasks, a plain queue any machine could write to and any person could read, where the work itself lived out in the open. A well-formed entry in it was the difference between asking a machine for help and handing it a job the next hand could understand and finish. There the several machines could, in a manner, speak to one another, though they had never been introduced — leaving each a receipt of what it had done. And a strange note crept into the man's account, said lightly, almost in passing: it had begun to feel less like he set the machines to work, and more like the work set him.

Near the month's end a plainer engine appeared, given away almost for nothing, and on the common run of work it was as good as the dear ones, or better. By one reckoning it cost a hundredth of what the great labs charged. And yet the firms did not leave. The reason laid bare the whole shape of the age. A mind by itself is a brain in a jar, useless without the frame around it — and the frames the firms had built could not simply be lifted from one engine and set upon another; each mind wanted its own, rebuilt from the ground. So the true scarcity was never the thinking, which had grown cheap and would grow cheaper. It was the last mile: the patient, unglamorous work of fitting a mind to the particular mess of one company's life, and the few skilled hands who could do it, whom every great house had already hired away. In those same days one lab pressed its advantage from the other end. It slipped its engine into the common message-hall where a company's people already talked, so that any of them could call it in by name and set it to work amid the half-written, the political, the stale — the very context no one had ever managed to write down. This was its masterstroke and its snare at once: a firm's own memory, its private understanding of itself, quietly rented back to it from the outside, and, once lodged there, impossible to tear loose however cheap the alternative.

By the last days of the month the state had reached in a second time, holding back another lab's newest engine, letting only a handful of approved partners near it while it weighed the danger. Twice now the frontier had been slowed by the hand of government, and the shape of the whole contest turned on it. If the smartest minds were to be rationed and delayed, then the advantage no longer lay in owning the newest one. It lay in owning the context — the part of a life or a firm that knew which message mattered, which record was current, what the customer had meant, what the team had settled, and what counted as done. A middling mind with that knowledge close at hand was worth more than a brilliant one that had to be told everything from the start. And so the great houses turned, all at once and by the same instinct, from the race to be cleverest to the race to sit nearest — to come to where a person already worked and lived, to be granted the messages, the files, the private and half-written truth of a life, and to make themselves so useful there that no one would think to send them away. The frontier was slowing. The nearness was not. What the machines wanted now was not to answer the questions people asked, but to be near enough, and trusted enough, to be handed the questions themselves — and to see, before the person did, what came next.

← previous