Skip to main content

Machine Learning, Reasoning and Intelligence

The Machine Learning, Reasoning and Intelligence program focuses on developing the science base and efficient computational methods for building versatile intelligent agents (cyber and physical) that can perform various tasks with minimal human supervision. In addition, they should be able to collaborate seamlessly with teams of humans and other agents in environments that are unstructured, open, complex and dynamically changing.

Even though there is no universally accepted definition of intelligence, or artificial intelligence, there are certain capabilities an intelligent agent must possess the ability to:

  • Learn, improve and adapt
  • Understand the environment
  • Plan to achieve its goals

This requires world-scale knowledge and reasoning ability. There have been significant advances in building highly intelligent agents that can play challenging games with well-specified rules.

This program focuses on developing agents that are “street smart”, namely, agents that can make good (or optimal) decisions and survive to achieve their goals in dynamic environments where rules are not clear and other actors in the scene do not take turn.

Although the field of artificial intelligence is broad, the goal of this program is to advance the fundamentals of AI with a primary focus on understanding visual data, and perception and planning for single and teams of agents. Application-driven research in in this area is supported by the automated image understanding thrust under the Computational Methods for Decision Making program.

This program addresses naval applications that involve the use of intelligent autonomous agents. Such agents serve as force multipliers by rapidly understanding sensor data streams and turning that into decision aids. This program is basic research, therefore while direct linkages to naval problems are not necessary, the potential for future applicability should exist.

Research Concentration Areas

  • Building blocks of machine intelligence – develop methods for:
    • Building knowledge bases from diverse sources
    • Learning complex concepts and tasks from annotated and unlabeled examples, instructions, and demonstrations
    • Reasoning with uncertain and qualitative information, as well as self-assessment
    • Planning in large-scale domains in information architectures that seamlessly integrate knowledge-bases, learning, reasoning, and planning for decision-making
  • Teams of agents and humans:
    • Develop computational methods for building decentralized collaborating teams of autonomous agents, in particular agents that are fairly capable in terms of sensing, communication and computational resource
    • Develop computational models of human decision-making and behavior for use by agents
    • Develop mathematical theories of swarm control, particularly engineered swarms with desired behaviors
  • Visual scene understanding – develop theory and algorithms for 4D scene understanding from images/video; recognition of scene type, objects, activities, and events; and inference of intentions:
    • For autonomous agent perception
    • For understanding surveillance imagery
    • For semantic search of visual databases
    • For succinct natural language descriptions of images and video

Research Challenges and Opportunities

  • Building knowledge bases, machine learning, reasoning, planning, and architectures for seamless integration of these modules.
  • Decentralized perception and planning for cooperative teams of autonomous agents.
  • Computational models of human behavior and decision-making for use by autonomous agents.
  • Scene understanding from visual data and other modalities, object recognition, activity recognition, event recognition, inferring intensions.

How to Submit

For detailed application and submission information for this research topic, please refer to our broad agency announcement (BAA) No. N00014-22-S-B001.

Contracts: All white papers and full proposals for contracts must be submitted through FedConnect; instructions are included in the BAA.

Grants: All white papers for grants must be submitted through FedConnect, and full proposals for grants must be submitted through; instructions are included in the BAA.


Kamgar-Parsi, Behzad Dr.
Program Officer
Code 311