(target-module-camera)= # Spinnaker Camera Cameras on Aeon are typically triggered using synchronised trigger events emitted by a Harp [CameraController (Gen2)](https://github.com/harp-tech/device.cameracontrollergen2) device. Prior to adding cameras to the system, one would usually [set this device up](target-module-camera-controller) in the workflow first. ## Nodes (target-node-spinnakervideosource)= ### SpinnakerVideoSource The `SpinnakerVideoSource (Aeon.Video)` node establishes a connection and configuration of a specific Spinnaker camera. #### Inputs None #### Outputs Stream of `Harp.Timestamped` carrying video data from the specified camera. #### Properties ##### Camera settings | Property name | Description | |--------------------|-------------------------------------------------------------------------------------------------------| | **Binning** | Set the size of the binning area of the sensor | | **ExposureTime** | Set the exposure time per frame. If the trigger frequency is too high to accommodate the exposure time here, it will be overridden | | **Gain** | Gain on the sensor | | **SerialNumber** | Serial number of the camera to acquire frames from | ##### Subjects Events (outputs) and commands (inputs) from the node are published to shared `Subject`s, the names of which are configured in the properties of the node. The output `Subject`s are then accessible in the Bonsai editor's toolbox and are useable elsewhere in the workflow using the same names. ###### Device event subjects | Subject name | Type | Description | |-----------------|----------|----------------------------------------------------------------------| | **FrameEvents** | `Harp.Timestamped` | Stream of camera frame events, ideally with one frame per trigger event. This stream is also output directly from the node | ###### Device command subjects | Subject name | Type | Description | |----------------------|----------|--------------------------------------------------------------| | **TriggerFrequency** | `double` | Stores frequency of the incoming trigger. The value of this `Subject` is set in the [CameraController](target-module-camera-controller) | | **TriggerSource** | `Harp.Timestamped` |Set the `Subject` that will act as a source for trigger events to this camera. This is generated by the [CameraController](target-module-camera-controller) | #### Usage To place a camera source, first create a `GroupWorkflow` with an appropriate name for the camera, e.g. "CameraTop". Inside, place a `SpinnakerVideoSource (Aeon.Video)` node, externalise all properties, and connect it to the `WorkflowOutput`. Configure the properties and subject names according to your requirements. :::workflow ![Aeon.Acquisition.Camera](../../workflows/camera.bonsai) ::: ### DroppedFrames The `DroppedFrames (Extensions)` node monitors a sequence of [FrameEvents](#device-event-subjects) from a specified `Subject`, e.g. "CameraTop". Specifically, it monitors the "FrameID" to ensure frames arrive consecutively. #### Inputs None #### Outputs In the event that a frame is dropped, the node outputs a `DynamicClass` containing the "Name" of the camera and the "Timestamp" and "FrameID" of the dropped frame. :::workflow ![DroppedFrames](../../workflows/droppedFrames.bonsai) ::: #### Properties ##### Subjects Events and commands from the camera are collected from and published to `Subject`s. Here you set the names used for these `Subject`s to identify events and triggers for this specific camera. Each of these `Subject`s is published and becomes accessible in the Bonsai editor's toolbox anywhere in the workflow using its name. ###### Device input subjects | Subject name | Description | |----------------------|-------------------------------------------------------------------------------------| | **StreamEvents** | Set the name of the events `Subject` output from a camera stream to be monitored | ### StreamTimeout The `StreamTimeout (Extensions)` node monitors a sequence of [FrameEvents](#device-event-subjects) from a specified `Subject`, e.g. `CameraTop`. Specifically, it monitors the actual time between receiving frames to ensure frames arrive within a given `DueTime` property. #### Inputs None #### Outputs In the event that no new frames arrive in time, the node outputs a `DynamicClass` containing the "Name" of the camera and the "Timestamp" and "FrameID" of the dropped frame. :::workflow ![StreamTimeout](../../workflows/streamTimeout.bonsai) ::: To avoid false alarms when cameras are not active, subscription to the `StreamTimeout` workflow is controlled using the same triggers that activate and stop the cameras through the [`CameraController`](target-module-camera-controller). #### Properties ##### Subjects Events and commands from the node are published to shared `Subject`s, the names of which are configured in the properties of the node. These `Subject`s then become accessible in the Bonsai editor's toolbox and are useable elsewhere in the workflow using the same names. ###### Device event subjects | Subject name | Description | |--|--| | **StreamEvents** | Set the name of the events `Subject` output from the camera stream to be monitored | **DueTime** | Set the maximum time between values to trigger a timeout event | ## GUI TBC ## Logging All frame events of a [`SpinnakerVideoSource (Aeon.Video)`](#spinnakervideosource) node are logged using a [`LogVideo` (`Aeon.Video`)](target-node-logvideo) node. :::workflow ![Aeon.Video.LogVideo](../../workflows/logVideo.bonsai) ::: This node saves the video itself in dedicated `.avi` files encoded with the **FMP4 codec**. Harp and hardware timestamps and the frame counter recorded at the camera are logged in corresponding `.csv` files with the following columns: **Data schema** | Column name | Type | Description | |-------------------------|------|------------------------------------------------------| | **time** | F64 | Harp timestamp for this frame | | **hw_counter** | I64 | Camera hardware frame counter. Should be consecutive | | **hw_time** | I64 | Hardware (camera) timestamp for this frame | The `.csv` data files are matched to the `.avi` movies, i.e. the first row of the data file correspond to the first frame of the movie. ## State persistence Not required for state recovery. ## Alerts Camera streams are monitored for stream timeouts and dropped frames, which usually result from power outages or connection issues. The [`StreamTimeout`](#streamtimeout) and [`DroppedFrames`](#droppedframes) nodes detect these failure events and their outputs can be formatted as alert strings, which are then sent to the "EnvironmentAlertMessages" and "AlertLogs" `Subject`s to [send](target-node-sendalert) and [log](target-node-formatlogmessage) alerts, respectively. :::workflow ![CameraAlerts](../../workflows/cameraAlerts.bonsai) ::: In order to monitor multiple camera streams simultaneously, multiple instances of the `DroppedFrames` and `StreamTimeout` nodes can be merged before they are passed to the Alert `Subject`s. This can be achieved by placing the nodes together in a `GroupWorkflow` and using `Merge` on the results before outputting them to the `WorkflowOutput`. Below is an example of how multiple `DroppedFrames` nodes are combined in a `GroupWorkflow` named "Dropped Frames Monitor". :::workflow ![mergeDroppedFrames](../../workflows/mergeDroppedFrames.bonsai) ::: Similarly, multiple `StreamTimeout` nodes can be combined in another `GroupWorkflow` named "Stream Timeout Monitor". :::workflow ![mergeStreamTimeout](../../workflows/mergeStreamTimeout.bonsai) ::: Both "Dropped Frames Monitor" and "Stream Timeout Monitor" can then be used to monitor all camera streams simultaneously by configuring the [alerts](target-module-alerts). :::workflow ![mergeCameraAlerts](../../workflows/mergeCameraAlerts.bonsai) :::