Grounding

This was originally posted Saturday July 27, 2013.

There has been some interest lately in combining vision and language for the purpose of “grounding” language.  Although I work on vision and am very interested in language I am not very  interested in combing these areas in this way.  I often describe myself as being anti-grounding.

First for some background.  What is grounding?  I usually describe it as the relationship between language — what we say — and reality.  How are sentences related to the actual world?  It might seem natural to think that vision is a primary source of information about the actual world and a natural thing to consider when discussing grounding.

When I think about language I like to go to google news and look at the first few sentences of the lead article.  At the moment of this writing the opening sentence of the lead story is the following.

The Muslim Brotherhood in Egypt says dozens of its supporters were killed overnight in clashes with security forces.

How do we conceptualize reality such that this sentence is grounded in reality?  What is “the Muslim Brotherhood”? What is Egypt? What is a supporter? What is death?  What is a clash?  What are security forces?

The Physical Fallacy. Many people naturally assume that reality is physical — when we use the term “reality” we are talking about physical material, its arrangement in space, its temperature, motion and so on.  It then seems natural to provide a meaning for language in terms of physical states (material arrangements and so on).  Most scientifically sophisticated people are materialists — the world and its people are believed to be implemented in physics.  However, materialism is usually irrelevant. For example, when I buy a house someone does a title search — they go to the registry of deeds and look up documents tracing the ownership history of the house.  Yes, it is true that the documents are physical pieces of paper.  But the physicality of the paper seems irrelevant.  I will use the term “physical fallacy” for the belief that grounding should be understood as a relationship between language and positions of material in space.  When I say that I am anti-grounding I mean, in particular, that I reject this physical sense of grounding.

Reality as a Database.  I think that much of reality should be understood to be facts in a socially agreed upon database.  Part of this is codified in law — who owns what, who is married to whom, who is employed by whom, how much money do I have, what insurance policies do I have and what do these policies guarantee?

A huge part of reality is who said what.  Speech acts are important.  Acts like promising, ordering, greeting, warning, inviting and congratulating are performed simply by saying. Such acts change reality — the act creates the consequential fact (in reality) that the act was performed.

Other parts of reality are closer to being physical but still, I would argue, not appropriately viewed in terms of arrangements of material.  I have four children.  I live in Illinois.  I drive a 2000 Honda Civic. I would argue that when we think about these facts the computation is more likely to be symbolic than to involve simulations of physical arrangements of material.  It seems that the real physics behind “having four children” is largely irrelevant when hearing or thinking about the consequences of such a fact.  I would even so far to say that the “the cup is in the cupboard” is, for many purposes, treated the same way as “your answer is in the email” — both can be processed at a symbolic level and even have the same consequences such as “to get your answer, open your email”.  Lakoff might say that the email example is a metaphoric use of a physical notion of containment.  But I would argue that the meaning (grounding?) of containment is directly understood at an abstract (non-physical) level. Object-oriented programming seems relevant here.  Think of a container class in C++.

My view is that reality itself — even those parts that are not just social convention — is best viewed as some kind of database.  In one sense this is not saying much since databases are information-theoretically universal.  I could have said “reality is a bit string”. Anything, including physical arrangements of material (finitely represented), and including sensory perceptions and motor actions, can be represented in either a database or in a bit string.  But the database representation of reality, including percepts and actions, seems particularly useful for understanding the relationship between language and reality.

Entities and Truth Conditions.  If reality is a database, what are its entities and what are its relations? What is “The Muslim Brotherhood”?  I would argue that it is useful to model the entities of reality as uninterpreted tokens such as “entity23963”.  The database can include facts such as “entity23963 is an organization” and “entity23963 is named `The Muslim Brotherhood’ “. We are then left with the reference problem — what entity in reality does a given utterance of a given noun phrase refer to? The reference problem seems related to Tarskian semantics where expressions (phrases) are mapped to their referent.In addition to assigning referents (meanings) to mentions, it would be nice to be able to assign truth values to statements.  When is a statement like “she had a choice between x and y” true in reality?

 

 

I will stop for this post but of course semantics is a big topic. What about the large literature on tense, aspect, and modality?  What about type-logical semantics, the typed lambda calculus and categorial grammar?  What about counterfactuals, the situation calculus, Wittgenstein’s public language hypothesis?  What about language learning? What about vector semantics?  Is mathematics a part of reality?  Why do mathematicians use Platonic language and do mathematical entities such as the vector space R^n really exist?  There is a lot of room for blogging here.