English Manual · 1.0.0
AMADEUS
User Manual
A guide to using AMADEUS.
1. Launch
Download and run the installer for your operating system. Git is not required. On macOS, a compatible Python installation is required as described below.
AMADEUS has not yet been extensively tested on macOS or Linux. If you encounter an error, please contact Yusuke Notomi at jpmyrmecol [at] gmail.com and include your operating system, installation method, and the error message.
The installer downloads AMADEUS and prepares the required environment, then opens the Home screen. An internet connection is required. Windows and Linux launchers also prepare Python automatically.
How to open the downloaded file
Windows: double-click AMADEUS-Setup.bat. For subsequent launches, you can also use the AMADEUS desktop shortcut.
macOS: open AMADEUS-Setup.command. A downloaded file may not have permission to run. If needed, type chmod +x in Terminal, drag the downloaded file into the Terminal window, press Enter, and open the file again. If macOS blocks opening it, check System Settings → Privacy & Security.
Linux: allow execution as a program in the file properties, then choose to run AMADEUS-Setup.sh in a terminal. Names and actions vary by desktop environment. A graphical desktop or WSLg is required.
On macOS, install a native Apple Silicon version of Python 3.10–3.12 with Tcl/Tk 8.6 before launching AMADEUS. Tcl/Tk 9 is not supported by the current GUI dependencies. To select another installation, set AMADEUS_PYTHON to the full path of its Python executable. To use a separate environment, set AMADEUS_VENV to its directory. See the macOS setup instructions.
On macOS and Linux, run the same downloaded launcher for subsequent sessions. It uses the existing AMADEUS installation.
Install using Git (Windows, macOS, Linux)
Windows
With Git installed, open Command Prompt (cmd) in your preferred installation directory and run this one line (Windows 10 / 11). The directory must not already contain an AMADEUS folder.
git clone https://github.com/jpmyrmecol/AMADEUS.git && cd AMADEUS && AMADEUS.batThe launcher automatically prepares uv, Python, and the required packages, then opens the home screen. Initial setup requires an internet connection.
For subsequent launches, open a new Command Prompt and run the following. Keep the created AMADEUS folder in place.
amadeusmacOS (Apple Silicon)
With Git installed, run the following command in a terminal in the directory where you want to install AMADEUS.
git clone https://github.com/jpmyrmecol/AMADEUS.git && cd AMADEUS && bash AMADEUS.shThe supplied environment targets Apple Silicon. Intel Macs are not supported by the bundled PyTorch 2.7.1 configuration. Launch without Rosetta.
Open AMADEUS.command. If permission to run the file is denied, run chmod +x AMADEUS.command AMADEUS.sh in Terminal in the AMADEUS folder.
AMADEUS uses MPS when available and CPU otherwise. MPS and the full workflow on actual macOS hardware have not been validated.
To use amadeus / amade from Terminal, add ~/.local/bin to PATH. See the instructions shown at startup for setup. Read any error messages in Terminal.
Linux (Ubuntu / WSL2)
With Git installed, run the following command in a terminal in the directory where you want to install AMADEUS.
git clone https://github.com/jpmyrmecol/AMADEUS.git && cd AMADEUS && bash AMADEUS.shRun bash AMADEUS.sh in the AMADEUS folder. A GUI display environment, such as a desktop session or WSLg, is required.
If errors related to Tk, OpenCV, or shared libraries occur, check the missing component reported in the terminal. CUDA is used with a supported NVIDIA GPU; otherwise CPU is used.
To launch with amadeus or amade, add ~/.local/bin to PATH. Linux x86_64 CPU environment setup has been verified, but the full workflow with real video and Linux ARM hardware have not been validated.
Install from a downloaded ZIP
To install without Git, open the GitHub repository and select “Code → Download ZIP”.
- Extract the AMADEUS ZIP archive.
- Start the launcher for your operating system. The same launcher can be used after the initial setup.
- After the initial setup is complete, the home screen opens.
| OS | Launcher |
|---|---|
| Windows 10 / 11 | Double-click AMADEUS.bat |
| macOS (Apple Silicon) | Open AMADEUS.command |
| Ubuntu / WSL2 | Run bash AMADEUS.sh |
2. Home screen
| Screen | Purpose |
|---|---|
| Easy Tracking | Run the standard tracking workflow with only a few inputs |
| Cropping & Trimming | Preprocess video regions, time ranges, brightness, and related settings |
| Segmentation | Separate animals from the background and define the analysis region |
| Advanced Tracking | Configure individual stages and parameters in detail |
| Multi Config Batch | Run multiple configuration files in sequence |
| Refinement | Review and correct positions, head directions, and individual IDs |
| Colab (beta) | Prepare the files needed to run training and tracking on a Google Colab GPU |
3. Basic workflow
- Prepare the video with Cropping & Trimming, if needed.
- Open Easy Tracking and select a video and session folder. The session folder stores the configuration and results.
- Select Launch Segmentation and configure foreground segmentation so that complete, isolated animals are retained.
- Specify the animal count, overlap severity, and whether the animals move backward. Select Processing to run training and tracking.
- Review the result video and CSV. Use Refinement to correct remaining errors if needed.
Use Advanced Tracking instead of Easy Tracking to configure individual stages and parameters.
4. Cropping & Trimming (optional)
Create a video by cropping the region and time interval to analyze and adjusting its appearance.
- Select the video with Input Video.
- Set the interval with Set In and Set Out.
- Use Full, or add regions with Add Crop.
- Drag or resize regions and select the regions to export.
- Set brightness, contrast, and color/grayscale for each region.
- Check the output folder, name, and FPS, then select Export.
5. Segmentation (foreground extraction)
Extract animals from the background and distinguish single-animal blobs from multi-animal blobs formed by animals in contact.
- Check the video and analysis frame range.
- Select a foreground extraction method. When using background difference, select Median, Maximum, or Minimum.
- Adjust Threshold and Minimum Blob Area.
- Select Analyze.
- Adjust the blob area bounds so that complete, isolated animals appear cyan (blue), while merged animals, fragments, and other outliers appear orange.
- If segmentation misses animals or parts of animals, use Additional Outlier to mark these regions with an additional threshold. Adjust it until the missed regions appear orange and are excluded from training samples.
- Select Processing to save the results.
Foreground extraction methods
| Mode | Suitable video |
|---|---|
| Background difference | Animals differ consistently from a static background |
| Dark region | Animals are darker than the background |
| Bright region | Animals are brighter than the background |
| Dark + background difference | Combine darkness with background difference |
| Bright + background difference | Combine brightness with background difference |
Results are saved under <Session>/segmentation/.
Result OBB Import (Beta)
This is a recently added beta feature and remains incomplete. Check the resulting classification carefully before use.
Blob area alone may not distinguish a large individual from two animals in contact. If you have already tracked the same video, you can import the result OBBs to help retain large single-animal blobs.
Open Result OBB Import (beta) in the sidebar, select Use imported result OBB, and choose a CSV with Reference. Use a file from <Session>/results/ containing frame and the cx, cy, w, h, and heading columns for each ID. The coordinates must correspond to the same video, even if the CSV comes from another project.
Each imported OBB is assigned to one blob. This is the blob containing the OBB center, or the blob with the greatest overlap if the center lies on the background. An outlier blob is reclassified as a single-animal blob only when it has exactly one assigned OBB and that OBB covers at least Min OBB coverage of the blob area (default 0.7). Blobs already within the area bounds, Additional Outlier regions, and blobs with zero or multiple assigned OBBs retain their existing classification.
The coverage threshold helps reject merged blobs when tracking missed one of the animals. It cannot guarantee that every merged blob is rejected. Hover over blobs to inspect their OBB counts and coverage, and choose the threshold using your video.
Frame offset is the frame number in the result CSV minus the video frame index, for cases such as tracking a video that was trimmed afterwards (default 0). With Show imported OBB enabled, the imported OBBs are drawn in magenta on the preview and on the exported labeled video, so misalignment is visible. The result-single count in the preview title reports how many blobs were taken out of the outlier group, and hovering over a blob shows its OBB count and measured coverage.
The settings are stored in segmentation_gui_config.json. To stop using an imported result, select Clear imported result or clear the checkbox.
6. Easy Tracking
Answer three configuration questions, then run automatic training data generation, detector training, detection, and tracking in one batch.
- Select Video File and Session.
- Use Launch Segmentation to configure foreground extraction.
- Enter the number of animals in How many animals are in the video? and press the Enter button to apply it.
- Select the degree of overlap and whether any animals move backward.
- Select Processing.
| Input | Choices | Meaning |
|---|---|---|
| How many animals are in the video? | Positive integer | Number of animals to track |
| Use paste augmentation when animals = 1 | On / Off | Whether to use synthesized images for training even with one animal |
| How severe is the overlap? | Very heavy / Heavy / Light | Severity of visual overlap. Select Very heavy when almost complete occlusion is expected. This adds identity verification and correction using contrastive learning and increases processing time |
| Do the animals ever move backward? | Yes / No | Yes adds a preliminary detector to identify and exclude potentially incorrect direction class labels caused by backward movement |
| Delete tmp files | On / Off | Whether to delete temporary files after completion |
Save Config and Load Config save and load YAML files. Switch Advanced Mode opens the current settings in Advanced Tracking.
Automated processing
| Stage | Operation |
|---|---|
| Initial Tracking | Associate single-animal blobs across frames and determine initial tracking parameters |
| Direction Estimation | Assign one of eight direction classes to each retained OBB using movement direction |
| Refine Blobs | Identify and exclude potentially incorrect direction class labels when backward movement is expected |
| Clustered / Mixed Paste | Synthesize interaction images by copy-pasting single-animal blobs |
| Crop Images / Create Dataset | Crop images and prepare training and validation datasets |
| Training | Train a YOLO OBB detector on the generated dataset |
| Detection / ID Tracking / ID Correction | Detect OBBs and direction classes, associate detections over time, and refine tracks using geometric and temporal information |
| Create Video | Create a video with the tracking results overlaid |
Without direction estimation [Beta]
This is a beta feature. Check the tracking results carefully before use.
Enable Without direction estimation in Easy or Advanced Tracking when head direction is not needed or front and rear cannot be distinguished reliably. It is off by default.
This mode estimates OBBs and tracks identities without assigning direction classes. OBB rotation is retained as geometric information, but no front or rear is assigned. Stationary animals and animals with short trajectories are not excluded solely because movement cannot establish head direction.
Outputs are stored separately under <Session>/without_direction_estimation/. Final CSV files contain frame, cx0, cy0, w0, h0, … and no heading columns. Use the training data and weights generated for this mode.
Variable population [Beta]
This is a beta feature. Check the tracking results carefully before use.
Use this mode when the number of animals changes during a video, such as when animals enter or leave the field of view. It is off by default. In Easy Tracking, enabling it disables animal-count entry. In Advanced Tracking, enable it under Main → Basic Settings.
Newly appearing animals receive new IDs, while IDs that remain missing are eventually retired. The same animal may receive a new ID after re-entry. False detections or association failures can also create extra IDs, so this mode is not intended to provide exact population estimates.
Variable-mode results are saved in the variable/ folder within the tracking output directory.
7. Advanced Tracking
For a changing population, enable Variable population in Main → Basic Settings.
Configure individual stages and parameters in detail.
- Specify the session, training video, analysis video, and animal count.
- Use Adjust Parameters to measure initial values suited to the video.
- Review the stages to run and their settings.
- Select Run.
| Section | Main settings |
|---|---|
| Basic | Videos, session, animal count, image size, frame range |
| Initial Tracking | Extraction of single-animal blobs and association across frames |
| Create single animal images | Tracklets, direction class assignment, and filtering of potentially incorrect labels |
| Create with crossing | Synthesis of interaction images |
| Create dataset | Training image count, cropping, validation data |
| Training | Epochs, batch size, device, learning rate |
| Analysis | Detection, staged association, and refinement, including optional identity correction using contrastive learning |
| Create Video | Output range, FPS, overlays |
Random noise on animals [Beta]
Add small patches of background over animals in synthetic training images to simulate occlusion by dirt on the lens or arena cover. Patches are added after copy-paste augmentation. This option is available only in Advanced Tracking and is off by default.
Noise Size (% of body length) sets the patch size as a percentage of body length (default 10%). Max Noise per Animal sets the maximum number of patches per animal (default 10). The number is chosen randomly from zero to this maximum. Patch shapes are random. OBB labels and masks are unchanged.
Use the skip checkboxes to run only the stages you need.
Analysis weights can be specified as last, best, or a saved epoch number.
8. Multi Config Batch
Run multiple tracking configurations in sequence.
- Save the settings as YAML from Easy or Advanced Tracking.
- Add configuration files with Add Configs or Add Folder.
- Arrange the execution order with Up / Down.
- Select Run Queue.
9. Refinement
Review results overlaid on the video and correct head directions and individual IDs.
- AMADEUS mode: select
config.yaml, weights, and results - Dataset mode: select a video and tracking file
Dataset mode can load CSV/H5 files from AMADEUS, SLEAP, DeepLabCut, idtracker.ai, and others. To edit head direction, select data containing front and rear keypoint coordinates.
| Action | Effect |
|---|---|
| Drag an arrow endpoint | Move the front or rear keypoint |
| Drag the whole arrow | Move the animal position |
| Right-click | Reverse the head direction or swap IDs between two animals |
| Mouse wheel | Zoom in / out |
| Drag the background | Pan the view |
| Left / right keys | Move to the previous / next frame |
| Up / down keys | Change the selected animal |
| Space | Play / pause |
| Ctrl+S (Command+S also works on macOS) | Save |
10. Colab (beta)
Run a saved AMADEUS configuration in Google Colab to use a GPU.
- In Colab (beta), select the saved
config.yamland a Google Drive destination, then select Prepare for Google Colab. - After preparation, select Open Google Colab.
- In Colab, select Run all and authorize Google Drive when requested.
A GPU runtime is required. Final CSV files, result videos, required trained weights, summaries, and logs are saved to Google Drive.
11. Parameter adjustment
Guidance on adjusting the main parameters will be added. The default settings are generally sufficient.
| Symptom | Check first |
|---|---|
| Out of memory | Reduce the training batch size in Advanced Tracking if automatic adjustment does not resolve the error |
12. Outputs
| Location | Contents |
|---|---|
<Session>/config.yaml |
Run settings |
<Session>/segmentation/ |
Background images and Segmentation settings |
<Session>/model/ |
Downloaded pretrained weights |
<Session>/main/<model>/ |
Training data, trained weights, tracking results, result videos |
<Session>/results/ |
Final CSV |
<Session>/log.txt, time.csv |
Logs and processing times |
The results/ folder stores CSV files with a suffix identifying the processing stage. tracked denotes the output of staged association before refinement. filled denotes output after standard refinement. id_resolved includes identity correction using contrastive learning and subsequent refinement.
In the standard mode with direction estimation, the final CSV has one row per frame, with frame followed by cx0, cy0, w0, h0, heading0, … for each animal. Individual IDs start at 0. Coordinates and OBB dimensions are in input-video pixels: the origin is at the top left, x increases to the right, and y increases downward. The heading columns store head direction in degrees, from 0° inclusive to 360° exclusive. Zero points up and angles increase clockwise. Missing values are empty fields (NaN), and estimates are not guaranteed for every animal in every frame.
Trained weights are stored under <Session>/main/<model>/training/<video name>/weights/.
13. Recommended recording conditions
AMADEUS learns directly from the input videos, so tracking performance depends on recording conditions. The following conditions are recommended.
- A sufficiently elongated body shape to define a body axis, with visually distinguishable front and rear.
- A fixed camera viewing the animals from above, with movement approximately confined to a two-dimensional plane.
- Stable illumination, an essentially static background, and sufficient contrast between the animals and the background.
- Sufficient spatial resolution, frame rate, and shutter speed for reliable visual tracking in each frame.
- Sufficient periods in which animals are separated from one another. Images of isolated individuals are used to generate training data.
Training and analysis videos may be recorded separately, provided that the recording conditions are the same.
14. License
GNU Affero General Public License v3.0 only (AGPL-3.0-only). See LICENSE and third-party license information.