Spinnaker Camera

Cameras on Aeon are typically triggered using synchronised trigger events emitted by a Harp CameraController (Gen2) device. Prior to adding cameras to the system, one would usually set this device up in the workflow first.

Nodes

SpinnakerVideoSource

The SpinnakerVideoSource (Aeon.Video) node establishes a connection and configuration of a specific Spinnaker camera.

Inputs

None

Outputs

Stream of Harp.Timestamped<Aeon.Acquisition.VideoDataFrame> carrying video data from the specified camera.

Properties

Camera settings

Property name	Description
Binning	Set the size of the binning area of the sensor
ExposureTime	Set the exposure time per frame. If the trigger frequency is too high to accommodate the exposure time here, it will be overridden
Gain	Gain on the sensor
SerialNumber	Serial number of the camera to acquire frames from

Subjects

Events (outputs) and commands (inputs) from the node are published to shared Subjects, the names of which are configured in the properties of the node. The output Subjects are then accessible in the Bonsai editor’s toolbox and are useable elsewhere in the workflow using the same names.

Device event subjects

Subject name	Type	Description
FrameEvents	Harp.Timestamped<Aeon.Acquisition.VideoDataFrame>	Stream of camera frame events, ideally with one frame per trigger event. This stream is also output directly from the node

Device command subjects

Subject name	Type	Description
TriggerFrequency	double	Stores frequency of the incoming trigger. The value of this Subject is set in the CameraController
TriggerSource	Harp.Timestamped<Harp.CameraControllerGen2.CameraEvents>	Set the Subject that will act as a source for trigger events to this camera. This is generated by the CameraController

Usage

To place a camera source, first create a GroupWorkflow with an appropriate name for the camera, e.g. “CameraTop”. Inside, place a SpinnakerVideoSource (Aeon.Video) node, externalise all properties, and connect it to the WorkflowOutput. Configure the properties and subject names according to your requirements.

DroppedFrames

The DroppedFrames (Extensions) node monitors a sequence of FrameEvents from a specified Subject, e.g. “CameraTop”. Specifically, it monitors the “FrameID” to ensure frames arrive consecutively.

Inputs

None

Outputs

In the event that a frame is dropped, the node outputs a DynamicClass containing the “Name” of the camera and the “Timestamp” and “FrameID” of the dropped frame.

Properties

Subjects

Events and commands from the camera are collected from and published to Subjects. Here you set the names used for these Subjects to identify events and triggers for this specific camera. Each of these Subjects is published and becomes accessible in the Bonsai editor’s toolbox anywhere in the workflow using its name.

Device input subjects

Subject name	Description
StreamEvents	Set the name of the events Subject output from a camera stream to be monitored

StreamTimeout

The StreamTimeout (Extensions) node monitors a sequence of FrameEvents from a specified Subject, e.g. CameraTop. Specifically, it monitors the actual time between receiving frames to ensure frames arrive within a given DueTime property.

Inputs

None

Outputs

In the event that no new frames arrive in time, the node outputs a DynamicClass containing the “Name” of the camera and the “Timestamp” and “FrameID” of the dropped frame.

To avoid false alarms when cameras are not active, subscription to the StreamTimeout workflow is controlled using the same triggers that activate and stop the cameras through the CameraController.

Properties

Subjects

Events and commands from the node are published to shared Subjects, the names of which are configured in the properties of the node. These Subjects then become accessible in the Bonsai editor’s toolbox and are useable elsewhere in the workflow using the same names.

Device event subjects

Subject name	Description
StreamEvents	Set the name of the events Subject output from the camera stream to be monitored
DueTime	Set the maximum time between values to trigger a timeout event

GUI

TBC

Logging

All frame events of a SpinnakerVideoSource (Aeon.Video) node are logged using a LogVideo (Aeon.Video) node.

This node saves the video itself in dedicated .avi files encoded with the FMP4 codec. Harp and hardware timestamps and the frame counter recorded at the camera are logged in corresponding .csv files with the following columns:

Data schema

Column name	Type	Description
time	F64	Harp timestamp for this frame
hw_counter	I64	Camera hardware frame counter. Should be consecutive
hw_time	I64	Hardware (camera) timestamp for this frame

The .csv data files are matched to the .avi movies, i.e. the first row of the data file correspond to the first frame of the movie.

State persistence

Not required for state recovery.

Alerts

Camera streams are monitored for stream timeouts and dropped frames, which usually result from power outages or connection issues. The StreamTimeout and DroppedFrames nodes detect these failure events and their outputs can be formatted as alert strings, which are then sent to the “EnvironmentAlertMessages” and “AlertLogs” Subjects to send and log alerts, respectively.

In order to monitor multiple camera streams simultaneously, multiple instances of the DroppedFrames and StreamTimeout nodes can be merged before they are passed to the Alert Subjects. This can be achieved by placing the nodes together in a GroupWorkflow and using Merge on the results before outputting them to the WorkflowOutput. Below is an example of how multiple DroppedFrames nodes are combined in a GroupWorkflow named “Dropped Frames Monitor”.

Similarly, multiple StreamTimeout nodes can be combined in another GroupWorkflow named “Stream Timeout Monitor”.

Both “Dropped Frames Monitor” and “Stream Timeout Monitor” can then be used to monitor all camera streams simultaneously by configuring the alerts.