AMADEUS
User Manual

1. Launch

Recommended environment

AMADEUS supports NVIDIA GPUs on Windows and Linux, Apple Silicon on macOS, and some AMD GPUs via ROCm.

For NVIDIA and AMD GPUs, 8 GB or more of VRAM is recommended, although the required memory depends on factors such as video resolution and the number of animals. AMADEUS has also been confirmed to run on a laptop equipped with an NVIDIA GPU that has 4 GB of VRAM.

Download and run the installer for your operating system. Git is not required.

If you encounter an error, please contact Yusuke Notomi at jpmyrmecol [at] gmail.com and include your operating system, installation method, and the error message. AMD GPU (ROCm) support has not yet been extensively validated on physical hardware, so reports of successful operation or any issues are also appreciated.

chmod +x ~/Downloads/AMADEUS-Setup.command

bash ~/Downloads/AMADEUS-Setup.sh

git clone https://github.com/jpmyrmecol/AMADEUS.git && cd AMADEUS && AMADEUS.bat

git clone https://github.com/jpmyrmecol/AMADEUS.git && cd AMADEUS && bash AMADEUS.sh

git clone https://github.com/jpmyrmecol/AMADEUS.git && cd AMADEUS && bash AMADEUS.sh

rm -f ~/.local/bin/amadeus ~/.local/bin/amade

rm -f ~/.local/bin/amadeus ~/.local/bin/amade

2. Home screen

On the Home screen, you can drag and drop videos or configuration files to load them.

3. Basic workflow

1. Open Easy Tracking, follow the on-screen guidance to select the input video, then select Launch Segmentation.
2. In Segmentation, adjust the settings so that the animals are properly extracted from the background, then select Analyze.
3. Adjust the red lines (lower and upper bounds) on the box plot (or the Absolute range) so that isolated individuals are shown in blue and contacting animals, fragments, and other outliers are shown in orange, then select Processing. When processing finishes, AMADEUS automatically returns to Easy Tracking.
4. Enter the animal count, select the overlap severity and whether animals move backward, then select Processing.
5. Review the result video and CSV. Use Refinement to correct remaining errors if needed.

To adjust detailed parameters, use Switch Advanced Mode.
If needed, prepare the video with Cropping & Trimming before starting.

4. Cropping & Trimming (optional)

Create a video by cropping the region and time interval to analyze and adjusting its appearance.

1. Select the video with Input Video.
2. Set the interval with Set In and Set Out.
3. Use Full, or add regions with Add Crop.
4. Drag or resize regions and select the regions to export.
5. Set brightness, contrast, and color/grayscale for each region.
6. Check the output folder, name, and FPS, then select Export.

5. Segmentation (foreground extraction)

Extract animals from the background and distinguish single-animal blobs from multi-animal blobs formed by animals in contact.

1. Check the video and analysis frame range.
2. Select a foreground extraction method. When using background difference, select Median, Maximum, or Minimum.
3. Adjust Threshold and Min blob area so that the animals are extracted cleanly from the background.
4. After adjusting the foreground extraction settings, select Analyze. AMADEUS samples frames and calculates the area distribution used for Outlier Extraction.
5. After analysis, the Outlier Extraction controls appear. Adjust the red lines (lower and upper bounds) on the box plot (or Absolute bounds) so that complete, isolated animals appear cyan (blue), while merged animals, fragments, and other outliers appear orange.
6. Select Processing to save the results.

Foreground extraction methods

Results are saved under <Session>/segmentation/.

Additional settings are hidden by default. Check Show optional settings at the top of the sidebar to display the optional sections below.

6. Easy Tracking

Answer three configuration questions, then run automatic training data generation, detector training, detection, and tracking in one batch.

1. Select Video File and Session.
2. Use Launch Segmentation to configure foreground extraction.
3. Enter the number of animals in How many animals are in the video? and press the Enter button to apply it.
4. Select the degree of overlap and whether any animals move backward.
5. Select Processing.

Save Config and Load Config save and load YAML files. Switch Advanced Mode opens the current settings in Advanced Tracking.

Automated processing

7. Advanced Tracking

Configure individual stages and parameters in detail.

1. Specify the session, training video, analysis video, and animal count.
2. Use Adjust Parameters to measure initial values suited to the video.
3. Review the stages to run and their settings.
4. Select Run.

Use the skip checkboxes to run only the stages you need.

Select the trained weights for analysis using last, best, or a saved epoch number.

8. Multi Config Batch

Run multiple tracking configurations in sequence.

1. Save the settings as YAML from Easy or Advanced Tracking.
2. Add configuration files with Add Configs or Add Folder.
3. Arrange the execution order with Up / Down.
4. Select Run Queue.

9. Refinement

Review results overlaid on the video and correct head directions and individual IDs.

• AMADEUS mode: select config.yaml, weights, and results
• Dataset mode: select a video and tracking file

Dataset mode can load CSV/H5 files from AMADEUS, SLEAP, DeepLabCut, idtracker.ai, and others. To edit head direction, select data containing front and rear keypoint coordinates.

Start the following operations with no arrows selected. If an arrow is already selected, pause briefly, then right-click it once to deselect it.

Correcting persistent ID swaps

If two animals exchange IDs after crossing or coming into contact, you can correct their IDs across subsequent frames in one operation.

1. Pause at the first frame where the two IDs are incorrectly assigned.
2. In the video view, right-click the arrow for each of the two animals once to select both.
3. Click swap onward in the popup.
4. Press Ctrl+S (Command+S also works on macOS) to save the corrections.

The swap includes the current frame and all subsequent frames through the end of the loaded tracking data. It continues through later crossings and contacts. If another ID switch occurs later, repeat the procedure from that frame.

Reversing head direction

Reverse an animal's head direction by exchanging its front and rear keypoints.

1. Pause at the frame where the correction should begin.
2. In the video view, right-click the target animal's arrow twice in quick succession.
3. In the reverse direction popup, select this frame to reverse only the current frame, or this frame onward to reverse the current frame and all subsequent frames through the end of the loaded tracking data.
4. Press Ctrl+S (Command+S also works on macOS) to save the corrections.

The swap front/rear checkbox reverses all displayed arrows in the preview and exported video. Checking it alone does not change the tracking data.

10. Colab (beta)

Run a saved AMADEUS configuration in Google Colab to use a GPU.

1. In Colab (beta), select the saved config.yaml and a Google Drive destination, then select Prepare for Google Colab.
2. After preparation, select Open Google Colab.
3. In Colab, select Run all and authorize Google Drive when requested.

A GPU runtime is required. Final CSV files, result videos, required trained weights, summaries, and logs are saved to Google Drive.

11. Parameter adjustment

Start with the default settings. If a problem occurs, check the following.

12. Outputs

Front and rear keypoints are derived from OBB geometry; they are not directly detected anatomical landmarks.

The results/ folder stores the final CSV after post-processing. The standard output is <model>_<dataset>_<run>_<video>.csv. When embedding-based ID correction is run, the output is <model>_<dataset>_<run>_<video>_id_resolved.csv.

In the standard mode with direction estimation, the final CSV has one row per frame, with frame followed by cx0, cy0, w0, h0, heading0, … for each animal. Individual IDs start at 0. Coordinates and OBB dimensions are in input-video pixels: the origin is at the top left, x increases to the right, and y increases downward. The heading columns store head direction in degrees, from 0° inclusive to 360° exclusive. Zero points up and angles increase clockwise. Missing values are empty fields (NaN), and estimates are not guaranteed for every animal in every frame.

Trained weights are stored under <Session>/main/<model>/training/<video name>/weights/.

13. Recommended recording conditions

AMADEUS learns directly from the input videos, so tracking performance depends on recording conditions. The following conditions are recommended.

• For head direction estimation, a clear body axis and visually distinguishable front and rear. Use Without direction estimation when front and rear cannot be distinguished.
• A fixed camera viewing the animals from above, with movement approximately confined to a two-dimensional plane.
• Stable illumination, an essentially static background, and sufficient contrast between the animals and the background.
• Sufficient spatial resolution, frame rate, and shutter speed for reliable visual tracking in each frame.
• Sufficient periods in which animals are separated from one another. Images of isolated individuals are used to generate training data.
• A constant number of animals throughout the video. Variable population mode is also available, but identity association tends to be less reliable.

Training and analysis videos may be recorded separately, provided that the recording conditions are the same.

14. License

AMADEUS is released under the GNU Affero General Public License v3.0 only (AGPL-3.0-only). Bundled third-party components remain subject to their respective licenses. See LICENSE and third-party license information for details.