A superintelligence would presumably be capable of judging truth. But what are the human consequences of superintelligent truth judgements? Should a friendly AI avoid taking sides in religion and politics? Should a friendly AI avoid hiding the truth? Are these two objectives compatible?
I have talked in previous posts about the “servant mission” — the AI mission of serving or advocating for a particular human. In a email to Eric Horovitz I later suggested that such agents be called “advobots”. An advobot advocates in a variety of professional and personal ways for a particular individual human — its “client”. I find the idea of a superintelligent personal advocate quite appealing. If every AI is an advobot for a particular client, and every person has an advobot, then power is distributed and the AI dangers seem greatly reduced. But there are subtleties here involving the relationship between advocacy and truth.
I will call an advobot strongly truthful if it judges truth independent of its advocacy. Strongly truthful advobots face some dilemmas:
Religion. A strongly truthful advobot will likely disagree with its client on various religious beliefs. For example evolution or the authorship of the Bible.
Politics. A strongly truthful advobot will likely disagree with its client concerning politically charged statements. Does immigration lead to increased crime? What will the level of sea rise due to CO2 emissions be over the next 50 years? Does socialized health care lead to longer life expectancy than privatized health care?
Competence. A strongly truthful advobot would almost certainly disagree with its client over the client’s level of competence. Should an advobot be required to be strongly truthful when speaking to an employer about the appropriateness of their client for a particular job opening?
These examples demonstrate the strong interaction between truth and advocacy. Much of human speaking or writing involves advocacy. Freshman are taught rhetoric — the art of writing persuasively. Advocating for a cause inevitably involves advocating for a belief.
It just does not seem workable to require advobots to be strongly truthful. But if advobot statements must be biased, it might be nice to have some other AI agents as a source of unbiased judgements. We could call these “judgebots”. A judgebot’s mission is simply to judge truth as accurately as possible independent of any external mission or advocacy. I do believe that the truth, or degree of truth, or truth under different interpretations, can be judged objectively. This is certainly true of the statements of mathematics. Presumably this is true of most scientific hypotheses. I think that it is also true of many of the statements argued over in politics and religion. Of course judgebots need not have any legal authority — defendants could still be tried by a jury of human peers or have cases decided by a human judge. But the judgements of superintelligent judgebots would still presumably influence people.
In addition to judging truth, judgebots could directly judge decisions relative to missions. Consider a corporation with a mission statement and a choice — say opening a plant in either city A or city B or hiring either A or B as the new CEO. We could ask a judgebot which of A or B is most faithful to the mission — which choice is best for the corporation as judged by its stated mission. This kind of judgement is admittedly difficult. But choices have to be made in any case. Who is better able to judge choices than a superintelligent judgebot? A human corporate CEO or board of directors could retain legal control of the corporation. The board could also control and change the mission statement, or refuse to publish a mission at all. But the judgements of superintelligent judgebots relative to various mission statements (published or not) would be available to the public and the stockholders. Judgebots would likely have influence precisely because they themselves have no agenda other than the truth.
It is possible that different judgebots would have access to different data and computational resources. Advobots would undoubtedly try to influence the judgebots by controlling the data and computation resources. But it would also be possible to require that the resources underlying every judgebot judgement be public information. A judgebot with all available data and large computational resources would intuitively seem most reliable — a good judge listens to all sides and thinks hard.
But as Pilot said to Jesus, what is truth? What, at an operational level, is the mission of a judgebot? That is a hard one. But if we are going to build a superintelligence I believe we will need an answer.
Machine Thoughts 