Deep Active Inference with Generative Actions and Diversity-Based Action Choice

EasyChair Preprint 14808, version 1

Abstract

The literature of Deep Active Inference (DAIF), implementing the generative, biologically inspired Active Inference framework (AIF) with the Deep Learning approach, often makes use of a hidden state transition model to generate current hidden states. It also usually leverages the Monte Carlo family methods to choose the agent’s next action that minimizes the Expected Free En- ergy (EFE). The action identification typically uses either a stochastic sampling process or a learning of sampled actions by a ’habit’ model. In this work, we explore an approach based on the learning and generation of actions as a result of hidden state transitions. The corresponding generative model, along with the variational form of the Free Energy (FE) and the EFE, are formulated for an environment represented as a POMDP, and the DAIF model architecture is also presented. We also suggest a novel approach for the action choice: the generated action minimizing the EFE is chosen based on the diversity of the expected risk relatively to that of its originating action set. The AIF agent is also equipped with top-down, selective, context-dependent attention mechanisms to control its behavior. Experiments have been conducted by addressing the continuous versions of Mountain Car and Inverted Pendulum problems. The results show the ability of the agent to learn and solve both problems with promising performance, requiring noticeable changes only on high-level attention parameters.

Keyphrases: Action Policy, Active Inference, Deep Active Inference, deep learning, generative model, planning, top-down attention

@booklet{EasyChair:14808,
  author    = {Yacine Benabderrahmane and Godefroy Clair and Jérémy Dufourmantelle and Claire Ky},
  title     = {Deep Active Inference with Generative Actions and Diversity-Based Action Choice},
  howpublished = {EasyChair Preprint 14808},
  year      = {EasyChair, 2024}}