Keynote Speech

 

 

Integration of Sensory and Language Data

 

Leonid I. Perlovsky

Air Force Research Laboratory

 

 

 

Abstract

Tremendous amount of data and knowledge is available to information systems today.  It includes sensory signals and images, language data streams, data bases of sensory and text data, knowledge in forms of mathematical models, exact laws of nature and uncertain expert intuitions.  System builders need computational techniques to extract useful actionable information from these data and knowledge.  Integration of knowledge and data in virtually any application requires multi-agent algorithms.  An agent is a human, machine, device or software code; agents are significantly autonomous and goal-oriented, perform various functions, and communicate with other agents.  An agent is equipped with sensors or collects data, receives communications, extracts information using existing knowledge, integrates this information into producing new knowledge, sends communications; these functions of agents embody the concept of life and intelligence.  Agents supporting collaborative technologies require adaptive man-machine and machine-machine interfaces, knowledge and data access, understanding of language, understanding of situations and environment, combining knowledge and data from diverse sources and disciplines, decision making.  This implies knowledge management, ability to make decisions in heterogeneous environment, with inaccurate data, uncertain knowledge and intuitions, information exchange, in other words, abilities for thinking and language.  

 

Computational techniques for thinking and language are far from matching human abilities.  I summarize the working of the mind and language emphasizing possible computational approaches.  This includes concepts, understanding, thinking, emotions, instincts, adaptation and learning, behavior, language ability, signs and symbols.  I discuss behavior of integration of signals and knowledge with emphasis on fusion of language and thinking in the mind.  The talk briefly reviews the history of the development of computational approaches to intelligence including pattern recognition, artificial intelligence, neural networks, knowledge and model based systems, evolutionary computation, hetero-hierarchical organization, modeling field theory (MFT, developed by the author) and computational linguistics.  Advantages and disadvantages of various approaches are compared.  I analyze dynamic logic, underlying MFT, and compare it with formal and fuzzy logic, underlying most of algorithms and neural networks.  Whereas in formal logic exact knowledge leads to exact conclusions, and in fuzzy logic fuzzy knowledge leads to fuzzy conclusions, in dynamic logic, much like in human mind, fuzzy knowledge leads to exact conclusions.  MFT is described in some details emphasizing possible approaches to integration of thinking and language.  

 

 

Short Bio:

Dr. Leonid Perlovsky is Technical Advisor at the Air Force Research Laboratory/SNHE.  Previously, from 1985 to 1999, he served as Chief Scientist at Nichols Research, leading the corporate research in information science, intelligent systems, sensor fusion, and algorithm development.  He participated as a principal in commercial startups developing tools for text understanding, biotechnology, and financial predictions.  He published about 50 papers in refereed scientific journals and about 100 papers in conferences, organized academic/engineering conferences, delivered invited and plenary talks and authored a book "Neural Networks and Intellect: model-based concepts", Oxford University Press, 2001.  He also served as professor at Novosibirsk University and New York University.