Thomas Howard
University Of Rochester
The ability for humans and robots to accurately and effectively perform collaborative tasks is highly dependent on the efficiency of bi-directional human-robot interaction. Language is an important mode of communication for collaborative human-robot teams because it does not require line of sight and is capable of communicating high-level constraints and objectives of a task, low-level actions that a robot or human must perform, or observations that may inform a teammate’s model of the environment. Efficient algorithms and models for grounded language interaction are particularly important for space exploration because computational resources are limited, operations are often interdependent, and interactions may be intermittent. Recent models for efficient natural language understanding and generation based on approximate probabilistic inference exploit conditional independence assumptions across constituents of the symbolic and linguistic representations and learn from corpora of examples that map language to symbols in the context of the perceived environment. While such work has improved the efficiency of grounded language communication, the ability to effectively communicate how the algorithm reached the final distribution remains a significant and important challenge. A more advanced model for natural language communication should be able to explain why inference failed to ground a particular concept or provide bounds on the metric and semantic state of the world for which a symbol will remain valid. This is particularly important for verifiable natural language understanding, since verifiable execution of the synthetized state machine is based on the formal representation of the problem generated by probabilistic inference. This research will investigate new models and algorithms for natural language understanding and generation for explainable and verifiable human-robot teaming that provide new mechanisms for interacting with natural language symbol grounding models and enable more efficient and effective collaborative task execution.