BioImageOperation (BIO) - Tracking step by step guide

Here is a step by step example of image tracking, using the BioImageOperation software including BIO script (see below). This software allows real-time processing (on standard consumer hardware) directly from HD video source or video files. For this example we use a public source: ants walking on concrete with varying lighting (video used from this public source - full video available here).

This is what the source video looks like

orig

 

First, detection is applied on each raw video image, to determine the objects of interest

detect

 

Next, we use contour detection to determine clusters (from OpenCV), then apply a custom high performance tracking algorithm (including direction & 'collision' detection)

track

 

An alternative way of visualising the tracking shows the history for each ant

track2

 

Additionally, we can analyse common paths over time

paths

 

The basic script used for basic tracking (using default / automatic parameter values):

 

OpenVideo("ants_in_concrete.mov")

{

  Grayscale()

  5:background = UpdateBackground()

  DifferenceAbs(background)

  Threshold()

 

  CreateClusters()

  CreateTracks()

 

  DrawClusters()

  ShowImage()

}

 

ant64

The latest release of Bio Image Operation (BIO) is available here.

BIO is only realsed as 64 bit, requiring a 64 bit version of Windows.

Download BIO version 1.4.341.7

Download BIO script instructions

 

Version 1.4.341.7:

- Added GPL license information

 

Version 1.4.341.6:

- Further improved automatically finding optimal tracking parameters

- Added 'Check for updates' functionality

- Minor bug fixes

 

Version 1.4.341.5:

- Significantly improved automatically finding optimal tracking parameters

- Draw logarithmic color legend based on input parameters

- Many improvements and bug fixes and updates to the help/manual

 

Version 1.4.341.4:

- Implemented directly writing to video file using H264 encoding

 

Version 1.4.341.3:

- Significant improvement in overall performance

 

Version 1.4.341.2 (OpenCV 3.4.1):

- Implemented all track draw functions including many options

- Improved script argument checking and help

- Uses OpenCV 3.4.1

 

Version 1.4.34.1 (OpenCV 3.4) - This is a major rewrite from C#/Emgu to C++/OpenCV. These are the key points:

- Using OpenCV directly - with full library support and using the latest version

- Real-time processing of HD video source (on standard consumer hardware) due to significant performance improvements

- Direct memory management with superior memory footprint

- Improved error handling and argument checking

- Built-in dynamically-generated script help

- Improved cluster detection: using connected components for accurate cluster sizes

- Very efficient custom image display, for minimal impact on main image processing loop

- Improved installer with pre-requisites detection and download (using WiX)

- Re-branded artwork

 

Flexible Script-based Image Operation

Bio Image Operation or 'BIO', is a next generation Image processing tool focussing on biological applications, balancing ease of use with desired flexibility required for research.

This tool has been developed in collaboration with biologists, using extensive captured images.

The solution balancing both the need for research purposes and flexibility required for this, and desired ease of use is realised in a script based user interface.

The tool uses the widely used OpenCV for many of it's image operations, with an efficient tracking algorithm allowing real time processing (upto HD @ 50 fps).

 

Automated tracking example: Tracking from public source: ants walking on concrete with varying lighting (video used from this public source):

tracking detail

(View full video here)

 

User Interface

bio

 

Visualisation example

Visualising tracks simply using image combination (Source material from Tom Wenseleers and colleagues - University of Leuven)

combine short

 

 

The initial idea was to create a simple agent based model, using very naive agents. Many models appear to invest on agents with considerable intelligence and abbilities. My goal was to make a model to reproduce agents that are very naive and even error prone, where the model strength results from the collective rather than the individuals. The first step of building an agent based model was to identify key model parameters. However, after realising the basic paramaters for common ants were not published at all, an initial step would be to determine these parameters from existing source material.

This led to image processing for agent tracking. The first step was producing image recognition from scratch - none of the existing tools available appeared to be able to reasonably import video formats of my source material. The source material was generously provided by expert biologists.

Image: low resolution source capture

original

Dynamic median background subtraction & normalised

normalised

 The main features of the first implementation:

- Class implementation allows image type operations: 'Image = Original - Background'

- Advanced operations such as normalisation

- Export as floating point RGB or grayscale image with optimised lossless compression (FTF format)

- Parallel processing to improve performance

 

Image tracking

 The image tracking was also developed from scratch.

tracking

The resulting software was able to perform tracking based on very low resolution and frequency source material (i.e. less than 10 pixels per subject)

- Using variable intensity level (as opposed to simple binary detection commonly used in tracking)

- Accurate and sensitive feature identification

- Tracking with velocity prediction

- Detecting and accumulating common paths (age / usage) using logarithmic colour scale

- Visualisation of features, tracking, common paths

- Export as floating point RGB or grayscale image with optimised lossless compression (FTF format)

- Parallel processing to improve performance