Overview

The core interface provides access to the biomimetic core that runs across the embedded processors P1 and P2. It is summarised in the diagram below.

This page describes the signal streams available on the core interface, it does not describe the behaviour of the core in any detail; for this, see the page Biomimetic Core.

Upstream

State

ROS stream specification
stream descriptionsynchronous state data
topic namecore/state
message typemiro_msgs/core_state

Core state is provided synchronously at the hi-rate (50Hz), with a time_usec timestamp. Individual fields are detailed in the table below.

Affect
emotionMIRO's emotion state (as valence, arousal).
moodMIRO's mood state (as valence, arousal).
sleepMIRO's sleep state (as wakefulness, pressure).
 
Action
priorityPriority of each action sub-system implemented in the biomimetic core (see miro_P2.h for a list of these).
 
Spatial
priority_peakLocation of peak in three attentional priority maps, one for each visual field and one for an aural field. Each peak is located in a space referenced to MIRO's head, with an azimuth and elevation, a peak height (i.e. peak priority) and an approximate peak size (which indicates something of the attentional target size in the visual field). Data is fresh if t_complete changes.
audio_eventLocation (referenced to MIRO's head) and magnitude of the latest audio event that was detected. All audio events that pass the detection threshold are placed into this stream, even those that are too weak to drive the attentional priority maps appreciably. Event is fresh if t_complete changes.
 
Branch
selectionSelection state at output of action selection system (GPR Basal Ganglia model).
disinhibitionDisinhibition state at output of action selection system.

MIRO maintains an affective state which comprises valence (negative to positive) and arousal (low to high), both mapped onto the interval [0, 1]. Immediate events cause changes to the emotion state, which returns over time to the mood state. The mood state, meanwhile, approaches slowly over time the emotion state, so that prolonged emotional excursions drive mood, in time. Alongside, homeostatic pressure builds whenever MIRO is awake, until wakefulness begins to fall and MIRO falls asleep; MIRO will wake again when homeostatic pressure has fallen off again.

At all times, action sub-systems built into the biomimetic core are competing for control of MIRO. Each indicates how strongly it requires control by raising its priority signal. A higher control system can choose to disinhibit any one of these systems by sending a non-zero signal on the corresponding line of the disinhibition signal in the downstream core/control message. If an action sub-system is disinhibited, it will execute its action, taking control of MIRO through the platform signal streams.

The spatial attention system, meanwhile, is monitoring the video and audio streams from the eye cameras and ear microphones for salient events. As each video or audio frame is processed, the most salient location in the corresponding frame is updated, and reported as the priority_peak for that stream. Internally, these priority peaks are key input data to action sub-systems that are concerned with implementing overt spatial attention (such as the orienting action). All detected audio events on the microphone streams are reported upstream, also, in the audio_event signal.

If required, a simple implementation of the GPR (Gurney, Prescott and Redgrave) Basal Ganglia model for action selection can be enabled to run on-board, connecting the upstream priority signal with the downstream disinhibition signal, and closing the behavioural loop. It is this loop closure that constitutes the "demonstration" mode that is used to illustrate the overall operation of the biomimetic core. Developers interested in developing the control architecture can alternatively implement this loop closure in their own software.

Priority

ROS stream specification
stream descriptionframes from spatial priority processor
topic nameplatform/pril, platform/prir, platform/priw
message typesensor_msgs/Image

Priority map frames are delivered as they are computed, only if the signal SPATIAL_SEND_PRIORITY is set in core/config. Since computation of these frames follows camera frame acquisition, these frames arrive shortly after the arrival of each camera frame.

The three streams are (l)eft camera, (r)ight camera, and (w)ide-field (generated from aural sensing stream). The wide-field stream is computed after the left camera stream.

Downstream

Control

ROS stream specification
stream descriptionsynchronous control data
topic namecore/control
message typemiro_msgs/core_control

Downstream control signals can be processed at the hi-rate (50Hz). However, if they are delivered more slowly, they are held over (for up to one second) to maintain a stable control signal to the platform. It is a choice for the higher-level controller, therefore, at what rate to deliver this stream—a rate of 10Hz, for instance, may be adequate for some applications.

If hi-rate control messages are being sent on both Platform Interface and Core Interface, both messages should set flags FLAG_SYNC_PLATFORM and FLAG_SYNC_CORE. In this case, the messages will be assembled and sent down to the platform in the same control packet, ensuring synchronicity of the two control streams.
Action
disinhibitionDisinihibition signal for each of the on-board action sub-systems.
priority_scaleA scaling factor applied to the computed priority from each action sub-system before it is processed further.
priority_offsetAn arithmetic offset applied to the computed priority from each action sub-system before it is processed further (scale is applied first, followed by offset).
 
Affect
mood_drive_targetTarget for adjustment of mood state.
mood_drive_gammaRate for adjustment of mood state (0 has no effect, -1 means "infinite" i.e. set state to target).
emotion_drive_targetTarget for adjustment of emotion state.
emotion_drive_gammaRate for adjustment of emotion state (0 has no effect, -1 means "infinite" i.e. set state to target).
sleep_drive_targetTarget for adjustment of sleep state.
sleep_drive_gammaRate for adjustment of sleep state (0 has no effect, -1 means "infinite" i.e. set state to target).
 
Motor
body_pushUp to four body pushes can be applied to the kinematic motor model to drive MIRO's motion.

Config

ROS stream specification
stream descriptionconfiguration data
topic namecore/config
message typemiro_msgs/core_config

There are two facets to the core configuration data.

The first is the signals, which are distributed amongst the various modules of the core. These are mostly binary switches, with some event signals scattered around, and many may usefully be changed at run-time. Signals do things such as enable modules, or individual module functions; a few turn on debugging facilities. Since the biomimetic core is entirely disabled by default, a specific signal set will have to be provided if the biomimetic core is to be used.

The second is the model_pars (model parameters) which set includes all the configurable parameters of the core, with the exception of the signals. It is unlikely to be useful to change these at run-time—indeed, it may not prove necessary to change them at all. If different parameters are needed for a particular application, they can be set once when MIRO is turned on. If required, they can be persisted also across power cycles, by passing the flag MIRO_P2_MODEL_PARS_PERSIST when they are sent.

Signals
P2B_W_signalsSignals for the branch module.
P2U_W_affect_signalsSignals for the affect sub-module of the upper module.
P2U_W_express_signalsSignals for the express sub-module of the upper module.
P2U_W_action_signalsSignals for the action sub-module of the upper module.
P2U_W_body_signalsSignals for the body sub-module of the upper module.
P2L_W_signalsSignals for the lower module.
P2S_W_signalsSignals for the spatial module.
 
Model parameters
model_parsAn object of type MIRO_P2_MODEL_PARS, see miro_P2_model_pars.h for details. The flag MIRO_P2_MODEL_PARS_UPDATE must be set or the object is ignored. If the flag MIRO_P2_MODEL_PARS_PERSIST is set, the object is stored for use on subsequent instantiations of miro_bridge; if the flag MIRO_P2_MODEL_PARS_CLEAR is set, any such stored object is removed.