How Computers Will Continue to Shape the World - Herbert A. Simon (Video & Transcript)

· 30 min read
By Herbert A. Simon, and

“…I don’t believe that predicting the future is really what we’re about. After all, we ourselves, or at least the younger ones among us, are going to be a part of the future. So, being a part of the future, our task isn’t to predict it. It is to design it…”

The Simon Institute for Longterm Governance’s namesake - Herbert “Herb” A. Simon (1916-2001) - was an American polymath who made groundbreaking contributions in the fields of artificial intelligence, psychology, economics, and organizational theory. As a Nobel laureate in Economics (1978), Herb is best known for his concept of “bounded rationality,” which posits that human decision-making is constrained by limited cognitive abilities and fundamental uncertainties about the structure of our environment. Herb’s interdisciplinary approach also led to a Turing Award (1975) for work in artificial intelligence, cognitive psychology, and list theory - solidifying his legacy as a visionary thinker.

Shortly before his death, at the Earthware Symposium of Carnegie Mellon University (CMU), in October 2000, Herb gave a striking talk on the choices humanity is and will be facing. Throughout the past years, CMU has restored old video recordings, allowing us to better understand Herb’s work. His impressive grasp of socio-technological developments continues to resonate. SI transcribed & summarized the 30-minute talk due to its exemplary demonstration of Herb’s oeuvre. The video & transcript are below.

This talk is a great reminder that the problems we face today were anticipated by groups of people who thought hard about how to bring about the best possible future. Herb explains why predicting the future is not our primary goal; instead, we must focus on designing a sustainable and acceptable world, actively shaping it through action. As technology and computers continue to advance, we must redefine our worth by recognizing our role within a larger cooperative enterprise. The ultimate goal is to create a world where we have enough resources, warm relationships, and intellectually challenging tasks, free from pain, fear, and hate. Technology, as a form of knowledge, should be evaluated by its ability to help achieve these goals. As we navigate our partnership with computers, it is crucial to ensure their goals align with our own while addressing issues such as ecological sustainability, fairness in resource allocation, and reducing conflict between different groups.

We hope that the transcript helps to process Herb’s wisdom and understand the choice of our namesake. Given recent progress in artificial intelligence, the words of one of the field’s fathers feel more important to recall than ever. Just a decade after his death, Herb’s worries about humanity’s shortcomings became applicable to computers due to the deep learning revolution - artificial intelligence that mimics the brain. Humanity’s challenge is even larger than Herb imagined, and we’re lucky to be able to stand on the shoulders of giants.

A verbatim summary

I don’t believe that predicting the future is really what we’re about. After all, we ourselves, or at least the younger ones among us, are going to be a part of the future. So being a part of it, our task is not really to predict it. Our task is to design it, to design a sustainable, and acceptable world. And then to devote our efforts to bringing that future about now. (…)

We’re actors, who, whether we wish it or not, by our actions and our very existence, are going to determine the future’s shape, for better or for ill. (…)

I happen to be among the group, but I’m not going to debate the point here, who think that computers will be doing most of the things intellectually, that humans do in a relatively short time, and maybe have already been doing a great many of them for some years now. (…)

We human beings have defined our worth in the world in terms of our uniqueness, our differences from other things. And perhaps that’s not a sustainable point of view for the future, we’re going to have to ask how we can get our deepest satisfactions from realizing that we are part of a much larger enterprise, and that we are able to operate cooperatively as a part of that enterprise. (…)

We must as a necessary condition for maintaining any standard of fairness, find some way of greatly mitigating, and, if possible, eliminating, the innumerable and passionate divisions of “we” from “they” that continue to make the human world a bloodstained collection of warring tribes. (…)

What do people want? We want enough to eat and want warm relations with some other people, we want pleasant and intellectually challenging tasks and surroundings. We want freedom from pain, from fear and hate.

So technology, as a part of that system must be evaluated by its ability to help or hinder us in pursuing those goals, and not by the flashing lights that it enables us to produce. (…)

Technology at base is knowledge. And it’s we human beings who decide the extent to which it will be used to advance the kinds of human goals I propose, to what extent it will be used to defeat these goals. I see no reason to take a pessimistic view of what the balance will be. Because that’s for us to choose, not to forecast. My own assessment of the past is that the accumulation of knowledge and technology over the ages has brought on balance more good than harm to humankind. But it’s only through our present efforts, that we’ll determine how favorable that balance will be in the future. (…)

It is our human behavior that has to be modified and altered, in solving problems. Problems about our ability to govern our own conduct in using the world’s resources, and conserving it as a life-maintaining planet, and most of all, problems of governing our relations with each other. (…)

For this reason, any picture of the future of our world is inseparable from a picture of the computer’s future. As we search for the physical resources that the world of our desires will need, and the balance between resources and requirements that we must work out, the computer must play a central role in our thinking. (…) With its assistance, we can aspire to handle the formidable complexities of understanding and designing the future. And we must not let our human vanity limit the role we assigned to the computer in this partnership. (…)

While engaged in this partnership with the computer, we must also remain sensitive to the need to keep the computer’s goals attuned with our own. (…)

And if the new knowledge is to bring us more good than harm, we must deal with the issue of how humankind will define its own purposes in the future world. Purposes that we would hope would extend beyond the passive existence of the spectator. (…)

Perhaps our very salvation will, at least in the shorter run, come from the severity of the problems we’ll have to solve. Finding an ecologically sustainable stage for the earth and all its living inhabitants; injecting far stronger criteria of fairness into the allocation of available resources and their products; and disarming the vicious competitions that now take place between every imaginable sort of “we” and “they”. (…)

Transcript

Moderator James Morris

Now it’s my pleasure to introduce Jared Cohon, the president of Carnegie Mellon University. Jerry has been our president for a little over three years and has really meant big things for the university. Jerry is educated as a civil engineer at the University of Pennsylvania and MIT. And his research specialty is the analysis of environmental problems, or environmental things happening based upon our technology. So in that sense, he’s a sort of perfect participant in this symposium. Dr. Cohon.

President Jared Cohon

Thank you very much, Jim. That simulated synthetic interview was just fantastic. I’m waiting for the optimized synthetic interview to come next. Maybe that was the optimized version of Jim Morris. I regret that I was not here earlier to partake in what I understand to have been just a wonderful symposium so far, all leading up to this great keynote address, which is a wonderful way to cap off this great day. 

The symposium itself is another wonderful example of Carnegie Mellon’s best - highly interdisciplinary, bringing together great minds, to interact with each other. And for all of us to partake in that. And this is another wonderful example, listening to Sir Arthur. I really wanted to hear Herb. And when the President shows up for a talk, it’s natural for them to say, Well, if you’re coming, then why don’t you introduce him? And while that you might think of that as the price I pay for being part of something like this, of listening to Professor Simon, in fact, it is a great honor. Without question, over the past 50 years, Herb’s vision, his wonderful participation in this university in so many different ways, has shaped what is best about Carnegie Mellon, and has helped to create and sustain the things that Carnegie Mellon has done, which have had the greatest impact in the world, from GSIA, to the School of Computer Science, to the Department of Psychology. Herb’s impact on this university has been enormous. And it continues, I’m delighted to say. 

And as you no doubt know, his contributions and his influence extend well beyond this campus. I have a wonderful, very recent reminder of this for you. Great demonstration of it. I just returned from the annual fall meeting of the Association of American Universities. This is the top 60 or so research universities of America. The AAU was founded in 1900 at the University of Chicago. So in this centennial year of the AAU, we had our meeting at the University of Chicago and it’s just a coincidence that Herb graduated from Chicago. I mean, the great things he did undoubtedly have something to do with his training in Chicago, but I mean, the meeting being at Chicago was pure happenstance. In recognition of its centennial, the organizers of this meeting decided to invite four great scholars to come and speak to the presidents of the AAU institutions, about the future of their fields, and which is not a small task. And we grouped them into the biological sciences, the physical sciences, humanities, and the social sciences. The speaker for the social sciences is a well known economist, was a well known economist from Stanford, who in his remarks said, the most important paradigm for the future of the social sciences, not just economics, by the way for the social sciences overall, is bounded rationality. And he put in parentheses, as he should have (Herbert Simon) - I forgot the year that he used. It was that innovation that great inspiration and insight that he had, which led to the Nobel Prize awarded in economics in 1978 to Herb Simon. I mention this story though, because here we are, in the year 2000 listening to one of the leading social scientists of this country speak about the next several decades in his discipline and related disciplines, and he chose work that Herb did 35 years ago, 40 years ago, something like that - some decades ago - as the key organizing idea for the future those disciplines, it’s remarkable. I mean, how many of us can say that a paper we wrote three years ago something that people still look at. And it shows you just how much influence Herb Simon has had on the world, not just on this institution. It’s my great pleasure to introduce to you Dr. Herbert Simon, the Richard K Mellon University professor of computer science and psychology.

Professor Herbert Simon 

Obviously, lots of reasons why I’m pleased to be speaking here this afternoon. The occasion of our new building, this dedication is one of them. And I hope I’ll have a chance to say a few words about that at the dedication itself, and the role of such colleagues as Alan Newell, in particular, Alan Perlis - the two Alan’s who were so central, in developing the computer science field and other intellectual activities on this campus, I’ll postpone that part of my remarks. Until then, because there’s a lot of business to attend here, as a result of the very exciting talks that we’ve heard here today, this morning, and this afternoon, about the future of computers and about our future in relation to computers.

Now, if you listened very carefully, to those remarks, although they were all offered in the best humor, cheerfully, but without any signs of debate or argument that I could hear, not all of the speakers reached the same conclusions. I would be, I think, a little bold or rash, maybe even rasher than I usually am a little rash to pick on, either individually or collectively, all of these points of view. And so I’m going to cop out in the sense by answering a different question. 

Now, several questions have been addressed here today. But they primarily had to do or at least a lot of them had to do with what is the future going to look like? Put in the declarative mode? What is future declarative? I guess? I’m not so good on tenses, what is the future going to look like? What are the computers? Is the future going to look like? What are they going to be doing in this society that we live in? Now, in a way, that’s a very important question, and I’m going to be drawing on some of the conclusions that they reach at least the ones that I agree with. But in another sense, I don’t think that’s more than a part of the question that we really want to answer. 

I don’t believe that predicting the future is really what we’re about. After all, we ourselves, or at least the younger ones among us, are going to be a part of the future. So being a part of it, our task is not really to predict it. Our task is to design it, to design a sustainable, and acceptable world. And then to devote our efforts to bringing that future about now, Sir Clarke, for example, was talking about the busy world we live in. And I think if we address that task, we won’t have any trouble in knowing how we will spend our time. Now this small task, but it is a task we have to face because we’re not observers of the future. We’re actors, who, whether we wish it or not, by our actions and our very existence, are going to determine the future’s shape, for better or for ill.

As some of you may regard my proposed goal of a sustainable and acceptable world, as unreachable. And that’s the Council of Despair, and serves no purpose. You might just as well go to bed, pull the covers over your head, if that’s your response. Others will think that my goal which I phrased in terms of acceptability is too pessimistic. Well, if you think that, then the best way to show that I’m wrong is to help the world find an attainable path to higher goals. I guess, Raj might be one of those who think, or will think when I get done, that the acceptable world is too pessimistic. And I invite Raj to bring in that better world which might be possible for us. 

Well, what are the conditions of acceptability the future we want to design? I think of three goals, which I sort of put at the head of my list, I’m not suggesting this is the whole list. First, we must find a way for living at peace with all of nature, not destroying the basis for the survival of all of us, we humans, and all the other life with which we share this planet. Now meeting this condition, almost surely requires finding acceptable ways for limiting total demands on the planet’s resources. Even as we find more efficient ways of meeting the demands, just finding more efficient ways more efficient energy, faster ways of getting communication around are not enough, we’re also going to have to learn to limit our demands. It also requires us to give up the false pride that views us as separate from nature. And to recognize that we’re a part of nature, have been several remarks made today about the problems of our uniqueness, when we find computers doing the kinds of things that we are able to do and that we’re very proud of doing. And there was a range of views as to how soon computers were going to do all of these good things. I happen to be among the group, but I’m not going to debate the point here, who think that computers will be doing most of the things intellectually, that humans do in a relatively short time, and maybe have already been doing a great many of them for some years now. But regardless of the view we take on that, to a very important extent, we human beings have defined our worth in the world in terms of our uniqueness, our differences from other things. And perhaps that’s not a sustainable point of view of for the future, we’re going to have to ask how we can get our deepest satisfactions from realizing that we are part of a much larger enterprise, and that we are able to operate cooperatively as a part of that enterprise. 

Second, we must find a way for sharing broadly and fairly, the outputs of our productive efforts of all kinds, no matter how ample or limited, these outputs are in a sustainable world. Without that fair sharing our talk of supporting diversity is meaningless. I don’t think I am even satisfied with Sir Arthur Clarke’s one computer station per village. I think each hut in a village at least, ought to have one and maybe that isn’t fair. Contemporary theories that oversimplify the distributive system, in our world or in our country, to a set of competitive markets, with their associated prices, is a caricature of the actual complexities, which basically ignores the issues of distribution and sharing.

Third, we must as a necessary condition for maintaining any standard of fairness, find some way of greatly mitigating, and, if possible, eliminating, the innumerable and passionate divisions of we from they that continue to make the human world a bloodstained collection of warring tribes. Including that little world of Sri Lankan, which Sir Arthur Clarke lives today, where the same kind of ethnic warfare has been going on for years, that we could find on almost any other spot on the map that was displayed up here a little while ago. Tribes that are continually engaging in fluctuating patterns of mutual hostility and collective mayhem. Of which traditional war is really only one of the more obvious forms, perhaps not even the most devastating, perhaps not even the most difficult to root out. 

Probably most of us would agree that if we could come even halfway toward meeting these three conditions, we would regard our world as pretty well-designed, or pretty well operated. However you want to, you want to put it. We’ve been hearing today a great deal about the future technology, especially electronic communications technology - its promise, its dangers. But oddly enough, I think most of you have observed that my specification of a design for the future didn’t mention technology at all. Went back to simple age old human desires that must have been felt by our earliest ancestors. And several of the discussions today have made similar points to this. What do people want? We want enough to eat and want warm relations with some other people, we want pleasant and intellectually challenging tasks and surroundings. We want freedom from pain, from fear and hate.

So technology, as a part of that system must be evaluated by its ability to help or hinder us in pursuing those goals, and not by the flashing lights that it enables us to produce. Well, what about the technology we’ve been discussing today? Does it give us the means for reaching the kind of future we’d like to have? Or is it the root of the problems that we have to solve? No, I think we’ve had plenty of evidence today that it’s some of both. Today I think we understand that technology is not metal and glass and plastic. We understand that technology is knowledge. And knowledge provides the capability for doing new things. Things we haven’t been able to do before good things and or bad things. 

We’ve long had two myths about technology, had the myth of Prometheus stole fire from the gods for the use of mankind. He’s the hero of modern science, we have a big mural of Prometheus in the dome of our National Academy of Sciences, hero of the sciences. Our second myth is the myth of Pandora, who was created by Zeus, in fact, to punish man, humankind for the presumption of Prometheus. He’s going to show them that you can’t steal fire and get away with it. On opening the basket of mischief that Zeus gave her, Pandora brought all kinds of ills to us. Our modern struggle to defeat infectious viruses reenacts that morality play, or the combat between Prometheus technology and Pandora’s mischief. As we create new knowledge in the form of anti viral technology, mutation, and natural selection in nature, create new knowledge in the form of new viruses. So good knowledge and bad knowledge at least evaluated from our standpoint, the viruses might have a little different evaluation of the outcome. 

So technology at base is knowledge. And it’s we human beings who decide the extent to which it will be used to advance the kinds of human goals I propose, to what extent it will be used to defeat these goals, I see no reason to take a pessimistic view of what the balance will be. Because that’s for us to choose not to forecast. My own assessment of the past is that the accumulation of knowledge and technology over the ages has brought on balance more good than harm to humankind. But it’s only through our present efforts, they will determine how favorable that balance will be in the future. The question was raised this morning, in our earlier virtual interview about the moral obligation of scientists for the consequences of their scientific discoveries. And that’s a hard question to answer, particularly for people who think they’re doing basic science. How do scientists - how do we know what are going to be the outcomes, other things we discover we’re discovering knowledge, and as I’ve suggested that can be used for good or bad. And so we have to make some very general judgment, like the perhaps bland one that I made just a moment ago, about whether on balance, knowledge, basic knowledge about how the world operates about what the laws of nature are. The balance of good and evil that’s likely to result from that. 

The kind of electronic knowledge that we’re acquiring today is unique in its direct relevance to the problems that face us. Of course, it’s not really electronic knowledge. It’s knowledge about the creation transmission and processing of information. And thus, it gets to the very roots of our own humanity. It’s knowledge about ourselves as thinking individuals and as communicating collectivities. It’s given us a whole new picture already, of what the human mind and what the human thought processes are all about, and how they’re carried on. To be sure, the picture that we formed so far, has been largely focused on thought to the relative neglect of emotion, and motivation. But as we progressed in our studies, that gap’s also beginning to be closed. And again, I noticed today that some of the most exciting ideas that were presented to us were questions about specifically those aspects of the human mind, and the human body, that are not devoted to thought, at least the purest of thought, that also involve emotion, and involve motivation. It’s sometimes called hot cognition, to distinguish it from, I guess, the more scholarly variety of human thinking. So in our research, using computers to understand the human mind, we have focused primarily on that cold cognition. But we also are using it and need to use it even more, to understand the rest of us to understand how that cognition is linked to our emotions to our motivations. Much of that linkage depending on the mechanisms of our bodies.

Well, what should we be looking for in the new technology? When I look forward, first and foremost, are better ways to understand ourselves. And by understanding ourselves to find solutions to the human problems that I’ve proposed are the central ones of our world, because if those really are the central problems that we have to, to solve, then it is our human behavior that has to be modified and altered, in solving them. Problems about our ability to govern our own conduct in using the world’s resources, and conserving it as a life maintaining planet, and most of all, problems of governing our relations with each other.

Because computers can process every kind of pattern and symbol, they provide us with a powerful new instrument and a powerful new methodology for modeling individual human minds. And therefore a powerful new tool for understanding these complex systems called people and societies. And so they have to play a central role in designing a plan for that attainable and sustainable future. Of course the computer extends far beyond this single domain of application. In fact, most of the applications, not all of them, but most of the applications we heard about today, went beyond that one computer is an invention that has a power and generality compared to the power and generality of the pencil, or even a written language. If it is not the most important of all human inventions, I think it clearly ranks among the top three. It’s also already become, will continue to become increasingly a constant companion and partner of the human mind. 

For this reason, any picture of the future of our world is inseparable from a picture of the computer’s future. As we search for the physical resources that the world of our desires will need, and the balance between resource and requirements that we must work out, the computer must play a central role in our thinking. With its assistance with the kind of modeling we can do with computers, for example, with the kind of help it can give us in doing our science. With its assistance, we can aspire to handle the formidable complexities of understanding and designing the future. And we must not let our human vanity limit the role we assigned to the computer in this partnership. 

I think any of you who have been engaged in research in artificial intelligence, or in computation generally, which applies computers to scientific activities to the work of the laboratory or the work of the the scientific theorists will have recognized the kind of ambivalent attitude with respect to the advance of computer capabilities in these fields. As long as the computer can be presented as a tool, which a scientist uses - the scientist’s assistant - then the acceptance comes rather readily. The minute is suggested that perhaps the computers become a partner of the scientists, I won’t go farther than that. That’s the partner, a partner of the scientist, then we distinguished changes in attitude changes in response to the computer. And this is another part of that human vanity I, I mentioned before, which in which we secure part of our feeling of worth out of our distinction out of being better than someone, if not other people. Well, at least we can be better than computers. An attitude we really have to rethink rather carefully. 

While engaged in this partnership, if you’ll permit me now to make it a partnership. While engaged in this partnership with the computer, we must also remain sensitive to the need to keep the computer’s goals attuned with our own. I’m not convinced that this will be difficult. The reading of history persuades me that the most dangerous villains we will encounter along the way will rarely be the forces of nature, and in particular, that they are more likely to have human than computer form. I think it’s still true that when I have nightmares, which fortunately isn’t too often, they more often have human than computer form. When we observe computers misbehaving, our cry should not be “cherchez l’ordinateur” but “cherchez l’homme”.

Remember that the next time you’re told that something can’t happen because the computer is down, a human being - careless, lazy, stupid, or even malignant - rather than the computer is likely to be at the root of the trouble. Here around CMU we’ve been amazed, amused, gratified and instructed in the last three or four years by the developments in robot soccer. I’m amazed we got through the session so far today, I think almost without mentioning robot soccer. I think of it all the time. Although I’ve been very slightly involved in the effort. For four years with rapidly increasing skill, computers have been playing a human game, requiring skillful coordination of all the senses and motor capabilities of each player, as well as communication and coordination between players on each team, and strategic responses to the moves of the opposite team. We’ve seen in the soccer games an entire social drama. Played out with far less skill so far than human soccer. But with all the important components of the latter clearly available. And today, computer soccer seems to me to be sort of the quintessence of, of where artificial intelligence has arrived. There are other examples as we heard this morning, but you just can see it all in one piece almost in the computer soccer, interaction with the environment, social interaction, use of language. So it’s a complex world with all the elements of intelligence and learning that artificial intelligence has been exploring for a half century and a harbinger of its promise for continuing rapid development. Almost all of our hopes and concerns for the future can be examined in miniature in this setting, including our own role in relation to computers. Are we going to stop playing soccer when they really do get good? We haven’t, fortunately stopped playing chess. After the test bearings match. The robot soccer tournaments like human soccer tournaments are a spectator sport. Although they’ve not yet erupted in riots of the kind that have infected the spectators of human soccer matches. They raise many questions about our relations with computers. 

For example, how far will and should we humans adopt the role of spectators in a world where much perhaps all of the world’s necessary work can be done by computers? Again, a theme that has emerged in several of the talks that we’ve heard today, even before computers in the web appeared on the scene. Television has sports arenas. And for those of you are not Pittsburghers, I should say that that’s a topic of particular Pittsburgh interest. Even before computers in the web appeared on the scene television and sports arenas, especially in the affluent societies of the first world had caused a rapid development of spectatorism, which has been viewed and I think correctly without not a little concern because it has to do with what we do when we don’t have to do anything else? These developments remind us again, that the myth of Pandora is as viable as the myth of Prometheus. And if the new knowledge is to bring us more good than harm, we must deal with the issue of how humankind will define its own purposes in the future world. Purposes that we would hope would extend beyond the passive existence of the spectator. Please don’t misunderstand me, a couple of ballgames a year is admissible.

When we look backward at the numerous historical examples of social elites that have lived more or less parasitically on our societies because they could get other people in the society to do their work, when we look at those societies, what we see doesn’t speak well for the ability of people to find mutually supportive goals and challenges, when these are not pressed on them by needs for survival. I think the whole literature of social elites, leisure classes, supports that point of view. In the next round, the century we’re looking ahead to, or the half-century, in the next round we’ll have to do much better than such elites have done in the past. If we’re to meet the three design criteria I’ve proposed for our future, or even close to meeting them. Must find means to achieve yet another goal: define the activities for ourselves in this new world that will have the same wonder and excitement and intensity going far beyond the crowd roars and spectatorism, as the most challenging activities that today’s world provides us with. In sports, crafts, professions, arts and science, and especially in the day to day subtle processes of living closely and warmly with other people.

Perhaps our very salvation will come, at least in the shorter, little run, will come from the severity of the problems we’ll have to solve. Finding an ecologically sustainable stage for the earth and all its living inhabitants; injecting far stronger criteria of fairness into the allocation of available resources and their products; and disarming the vicious competitions that now take place between every imaginable sort of “we” and “they”. If we accept this challenge of social design, there may be little spare time for excessive spectatorism. The soccer robots will have to design other robots to watch their games.

Further reading

If you want to learn more about Simon’s life & work, start with wikipedia. We also recommend the following resources: