Dr Aaron Smiles

Creative technologist and researcher with an EngD in computer vision. I prototype with generative video, 3D and sound, and I measure what the models keep and lose.

StillDrawn, my first benchmark, is pre-registered and checked against human raters: it measures whether an inked comic or manga panel still looks drawn once an image-to-video model animates it, and its data and code are public. I also release open-source tools, most recently loopcutter, which cuts sample-accurate loops for live DJ sets.

I hold an EngD in Computer Science from Queen Mary University of London, where I was part of The Centre for Advanced Robotics @ Queen Mary (ARQ) and the Human Augmentation & Interactive Robotics (HAIR) team. My research used stereo vision to detect and range transparent objects underwater, and produced an open dataset. My supervisors were Ildar Farkhatdinov and Changjae Oh.

I'm currently COO at Nuvelen, where we model how a forest grows and stores carbon, in effect a digital twin of each site that simulates years of growth from species growth curves, climate and soil data, checked against sensors in the ground. It is the question I bring to video and world models: how far can a simulated world drift from the real one before it stops being useful?

From 2011 to 2017 I ran a software startup building VR audio-visual production tools for the Oculus Rift DK1, where I worked on visual spatial sound rendering in virtual environments.

I also hold an MSc in Digital Media from Newcastle University, where I was part of Culture Lab and the parachuting club, and an Executive MBA from Quantic School of Business and Technology.

Email  /  Google Scholar  /  GitHub  /  LinkedIn

Datasets

StillDrawn: A Benchmark Dataset for Inked Line Art under Image-to-Video Generation

The data behind the StillDrawn benchmark: 29 inked comic and manga panels controlled for content and ink treatment, plus 5 ground-value variants, each with its line-art layer and generation recipe; all 373 clips generated from them by six image-to-video engines under a pre-registered protocol, with a per-clip manifest and metric outputs; and the frame strips and pseudonymised human ratings behind the paper’s findings. Data under CC BY 4.0, code under MIT.

UW-TransStereo: Underwater Stereo Vision Dataset for Transparent Object Detection & Ranging

This stereo vision dataset addresses the challenge of detecting transparent underwater debris. The collection includes 25 paired stereo vision recordings totalling over 9,000 stereo frame pairs across air and four aquatic environments. Coverage encompasses three bottle types tested under various conditions (freshwater, freshwater with enhancement, saltwater, saltwater with suspended pellets). The dataset provides over 5,000 labeled measurements, per-frame detections, and dimension estimates (height x width x depth). A standardized evaluation protocol uses MAE, RMSE, R-squared, and bias metrics. The system features real-time YOLO-based object detection (YOLOv5/YOLOv8), 3D depth integration with ZED stereo cameras, object dimension measurement capabilities, multi-point distance sampling, real-time Excel export with timestamps, screenshot capture with distance markers, SVO recording support, and OpenGL 3D visualization.

Publications

My research interests include AI, computer vision, the evaluation of generative video, robotics, and environmental engineering.

StillDrawn: Measuring the Stability of Inked Line Art under Image-to-Video Generation

A benchmark for one narrow question: when an image-to-video model animates an inked comic or manga panel, does the result still look drawn? StillDrawn pairs a 29-panel dataset (each panel with its line-art layer and generation recipe) with five automatic line-stability metrics and a pre-registered study of five video engines, two open-weights and three hosted. The pre-registered hypothesis returned null and is reported as registered. The failure taxonomy that emerged instead includes modes that stay in ink to a human eye while the metrics flag them, and modes the metrics barely see. A six-rater human check supports the metrics on content preservation; the absolute ink-register judgement proved weaker. An optical-flow control shows that engine rankings largely track how much each engine moves. The dataset, recipes and metrics are public on Zenodo and GitHub.

Sensing and Monitoring Canopy Growth: A Multi-Modal Survey of State-of-the-Art Technologies for Plant, Tree, and Forest Ecosystems

Prior surveys of canopy remote sensing organise the field by sensing modality. We instead organise it around the observation chain that connects sensing to ecological outcome: structure (height, biomass, leaf area), function (photosynthesis, water status, stress), and growth (temporal increment, accumulation, flux). We argue that integrated multi-scale growth monitoring has become feasible only since 2025: ESA Biomass (April 2025) is the first civilian P-band SAR satellite, penetrating tropical canopies to trunk level; NASA–ISRO NISAR (July 2025) adds dual L+S coverage every 6–12 days; GEDI resumed operations in 2024; and remote-sensing foundation models (Prithvi, TerraMind, SkySense) matured in parallel. Across eight technology domains — spaceborne lidar, SAR, optical multispectral, SIF, UAV/airborne, TLS, hyperspectral, in-situ IoT, and foundation models — each section identifies an explicit Integration Gap: the obstacle preventing its outputs from combining with adjacent stages. Headline results: GEDI–Landsat fusion correlates at r ≈ 0.84 with airborne lidar; P-band SAR yields direct biomass in tropical forests; the SIF–GPP relationship is mediated by canopy structure, demanding joint structural–functional fusion; pre-symptomatic wheat stem rust is detected 4 days after inoculation with F1 0.86–0.94; and recent foundation-model surveys catalogue dozens of remote-sensing foundation models, most still optical-only. We map each technology to its chain stage, name the gaps the 2025 convergence has not closed, and identify ten future directions led by physics-constrained multi-modal foundation models.

Descriptor: Underwater Stereo Vision Dataset for Transparent Object Detection & Ranging (UW-TransStereo)

Robust detection and ranging of transparent debris in water is challenging due to refraction, low contrast, backscatter, and turbidity. We present a paired, multi-environment stereo dataset acquired with a stereo camera, comprising 25 recordings totalling over 9,000 stereo frame pairs of three bottle targets (two transparent plastic bottles with filled/unfilled variants and a brown glass medicine bottle) across air and four underwater conditions (freshwater, freshwater with enhancement, saltwater, and saltwater with suspended pellets), along with analysis-ready detector spreadsheets and scripts. To promote reproducible use, we provide a standard evaluation protocol with well-defined accuracy measures (MAE, RMSE, R², and mean signed error) and report the proportion of frames with valid estimates, including vertical region-of-interest sampling. We include a lightweight vision enhancement variant, “basicEnhance”, to illustrate how pre-processing can be benchmarked against the same protocol without retraining models. The release is intended as a neutral testbed for benchmarking underwater transparent-object perception and for stress-testing stereo ranging under refractive media. We release all raw stereo streams, detector outputs, qualitative screenshots, and analysis scripts.

Exploring Stereo-Vision-based Object Detection and Ranging in Underwater Environments

Stereo vision offers dense depth without emitting light or sound, making it attractive for teleoperation and underwater robotics. Yet transparent and refractive objects, especially plastic bottles, remain problematic due to low texture, specularities, and medium-dependent refraction. This thesis examines how far commodity stereo pipelines can be pushed in two connected contexts: providing distance cues for an augmented telepresence interface, and detecting and ranging transparent bottles underwater. The first part implements a stereo-informed mixed reality teleoperation system combining a stereo display, a haptic device, and a Unity-based manipulator model. Underwater experiments using a custom tank and recalibrated stereo rig show that opaque and textured objects remain largely tractable, whereas water-filled transparent bottles frequently yield invalid or rear-surface depths. The second part introduces UW-TransStereo, a controlled multi-environment underwater stereo dataset of 25 paired recordings (over 9,000 stereo frame pairs) across air and four underwater conditions, with a baseline detection-and-ranging benchmark. Across environments, performance degrades systematically from air to saltwater with suspended particles, with water-filled smooth bottles driving most failures. The results show that commodity stereo can deliver useful telepresence cues in air and provide structured insight underwater, but refractive targets remain a fundamental challenge.

A Preliminary Study on Underwater Transparent Objects Detection with Stereo Vision: Air vs Freshwater

This paper investigates how underwater conditions affect the detection and distance estimation of transparent plastic bottles using a stereo computer vision pipeline. We created a multi-environment dataset of stereo recordings with per-frame range logs in air and freshwater across common bottle types (empty and filled, smooth and textured) and marked distances. Compared with air, fresh water environment consistently increased errors and reduced reliability: the root mean squared distance error for the filled transparent bottle rose from 38 mm to 696 mm, and detection accuracy for textured empty bottles fell from 100% to 74.4%. A simple image enhancement increased the share of valid object-distance estimates from 41.6% to 50.8% but also raised the root mean squared distance error from 146.2 mm to 209.3 mm, revealing a trade-off between stability and precision. Selecting more stable viewing regions and filtering unreliable detections improved robustness (for example, reducing object-level error for the filled transparent bottle in air from 445 mm to 13 mm), but closing the gap between air and water will require methods that explicitly account for refraction and underwater imaging effects.

Robotic Perception of Underwater Plastic Bottles for Augmented Telepresence

Not enough robots are used in the fight against ocean plastics. Remotely-operated vehicles (ROVs), controlled via telepresence, can collect ocean waste, but underwater depth perception is challenging, especially for transparent objects like plastic bottles on 2D screens, where augmented visuals could assist. This study investigates stereo vision technology for underwater depth estimation, focusing on transparent objects. We recalibrated a stereo camera for underwater applications, testing depth accuracy on air- and water-filled plastic bottles under controlled lighting. Results show that recalibration improves depth perception for opaque objects but remains limited for transparent materials, particularly water-filled bottles. Our approach and performance analysis highlight the challenges of accurate depth mapping for submerged plastics, underscoring a need for advanced methods to achieve reliable augmented visuals.

Implementation of a Stereo Vision System for a Mixed Reality Robot Teleoperation Simulator

This paper presents the preliminary work on a stereo vision system designed for a mixed reality-based simulator dedicated to robotic telemanipulation. The simulator encompasses a 3D visual display, stereo cameras, a desktop haptic interface, and a virtual model of a remote robotic manipulator. The integration of the stereo vision system enables accurate distance measurement in the remote environment and precise visual alignment between the cameras’ captured scene and the graphical representation of the virtual robot model. This paper delves into the technical aspects of the developed stereo system and shares the outcomes of its preliminary evaluation.

Projects and design work

My projects, films and design work are moving to studionephele.com, the new home of my studio, Studio Nephele. It is under construction. First up is StillDrawn Explorer, an interactive look at what video models do to hand-drawn line art, with its thesis and film. Its rating mode runs on blindpanel, an open kit for blinded human rating of generated video (film).

Design and source code from Jon Barron's website adapted using Leonid Keselman's Jekyll fork.