Face and species recognition

Recognition and correct classification of visually acquired objects is important for the survival of all creatures. In biological terms, classification errors almost always lead to loss of fitness (e.g. incorrect classification of another animal as a competitor may lead to costly agonistic behaviour, and the incorrect classification of a predator may lead to death). Evolution should thus strongly favour animals that are able to learn to correctly recognise and classify objects in the world around them. 

Understanding how patterns and faces are processed by the visual system and brain is also important if we want to understand the causes of systematic classification deficits in humans, such as prosopagnosia (inability to recognise faces). We are using a comparative approach, combining animal behaviour with machine learning and computer vision methodologies.

We are investigating face, pattern and object recognition abilities in lower vertebrates (fish, with no visual cortex), so as to understand the underlying mechanisms. We have already found evidence for parallel processing for colour and luminance information in fish, which is similar to what we know about human visual processing along the ventral and dorsal streams in the cortex and have discovered that fish are able to discriminate fish as well as human faces to a high degree of accuracy. 

During his Masters project in my lab, Rainer Obergrussberger developed custom software which allowed us to capture, extract and manipulate the UV facial patterns of reef fish.  

Dr. Siebeck’s training at the Max Planck Institute for biological Cybernetics in Tubingen, Germany on scanning and manipulating human heads for the creation of a human face database inspired the creation of a similar fish face database. Through that we discovered that the facial pattern of each fish (Pomacentrus amboinensis) is as unique as a fingerprint. This finding shows just how sophisticated the secret language of fish might be and also that these fish are a great model system for studying visual processing of complex patterns including faces.

Ambon damselfishAmbon damselfish Ambon damselfish

Ambon damselfish as it appears to human eyes. B/w: UV patterns of Ambon damselfish made visible by photographing them through a UV-pass filter. 

Collaborators and students

Visual learning

We have developed training and testing methods derived from human psychophysics for wild reef fish. We are using this methodology to test the visual, but also the cognitive abilities of various fish species (reef fish, Archerfish). The work has demonstrated the impressive ability of reef fish to learn (within days of capture on the reef) and perform visual discrimination tasks (e.g. colours, shapes). 

Training fish Training fish

Amira Parker training fish at the Lizard Island Research Station

We were able to show that wild reef fish are not only capable of associative learning (a rewarded stimulus is associated with a food reward), but also show anticipatory behaviour (the rewarded stimulus aids as a signal that a food reward will be available in a different location). This kind of behaviour had previously only been found in humans and other mammals. Birds and cuttlefish, on the other hand, show what is regarded as less advanced behaviour and tend to treat the stimulus as if it were the food reward itself.

Collaborators and students

Secret communication

The work on UV vision was inspired by the finding that many birds display ultraviolet markings invisible to the human eye. Findings of this type brought some researchers to the realisation that the entire literature in the area of animal colouration was flawed due to its implicit assumption that the scientist’s (human) eye was a suitable surrogate for that of the animal under investigation. Conclusions that had been reached about sexual selection in birds had to be re-evaluated once it became clear that birds can see UV light and that UV signals are important cues in mate choice. When we started working on UV vision in fish, it had just been described for goldfish and first reports mentioned the presence of UV colours in various fish, suggesting the exciting possibility that, similar to birds, UV might play an important role in the lives of fishes. 

Visual acuity tests Visual acuity tests

Ambon damselfish performing visual acuity tests in UV (340-390nm) and human visible (400-700nm) spectra. 

For UV signals to reach the retina, they have to be transmitted by the ocular media (cornea, lens and humors), which means that an assessment of the transparency of the ocular media would reveal whether there is at least the potential for UV vision in an animal. We demonstrated that around half of 400 reef fish species (adults and larvae) have the potential for UV vision while the other half have gone to extremes to prevent UV from entering their eyes. An important finding that resulted from this analysis was that predatory fish have UV-blocking ocular media and are thus unable to see the UV signals of their potential prey. UV signals therefore belong to a ‘secret communication channel’ that allows prey fish to be conspicuous to each other without attracting the attention of their predators.

The facial patterns found on some damselfish can only be seen by animals with UV vision. Many fish predators have UV-blocking ocular media and are thus UV blind. The facial UV patterns found on Ambon damselfish code for individuals and species identity and may also contain information about fitness and hierarchical status etc. 

DNA damage

Environmental ultraviolet radiation has positive effects (secret communication) as well as negative consequences (DNA damage). What are the protection mechanisms preventing DNA damage? 

Ultraviolet radiation (UVR) is known to damage the DNA of any exposed organisms, especially in the tropics where levels of UVR are among the highest on Earth. Protection from this radiation is therefore vital. Three protection mechanisms are under investigation: 1) Fish secrete chemical compounds (Mycosporine-like Amino Acids, MAAs) into their mucus that act as natural sunscreens. 2) Internal repair mechanisms specifically target UVR-induced DNA damage. 3) Behavioural avoidance is possible if deleterious levels of UV radiation can be detected. The degree of protection fish receive from each mechanism, the net damage under current UV exposure levels as well as whether fish are able to up-regulate their defences are unknown and need to be answered in times of a changing climate. Of particular interest to our lab is how fish, which use UV for communication and thus have reduced natural sunscreens in their mucus (UV-absorbing sunscreens would render UV-reflective patterns invisible) survive in an environment rich in damaging UV.

DNA damage
This graph shows DNA damage (64-PP and CPD lesions) detected in the skin of a settlement stage reef fish and UV radiation over the course of a day.
DNA damage
This graph shows DNA damage (64-PP and CPD lesions) detected in the skin of a settlement stage reef fish and UV radiation over the course of a day.
This is the subject of our study on the effects of UV exposure on reef fish larvae – a recently settled Pomacentrus amboinensis.

Collaborators and students

Larval fish orientation

The larval stages of most marine fishes spend days to weeks in the pelagic environment, where they must find food and avoid predators in order to survive. Some fish only spend part of their life history in the pelagic environment before returning to their adult habitat, for example, a coral reef. Orientation in ocean currents is challenging for tiny fish larvae, yet is essential for finding adult habitat and completing their life cycles. Fish larvae are good at swimming and orientating, but little is known about the senses (other than hearing) used for this.

In collaboration with Jeff Leis and Claire Paris, we are investigating the use of visual cues for orientation in larval reef fish. We are combining anatomical studies of the visual system with behavioural experiments (in the laboratory and in situ) to test the ability of reef fish larvae to detect and use visual cues, such as sun position and polarisation.

Collaborators and students

  • Jeff Leis
  • Claire Paris
  • Amanda Hay
  • Jack O’Connor


FishNet is part of the XL Catlin Seaview Survey, a global expedition documenting the world’s coral reefs in HD 360 deg imagery. In November 2015, nearly 1000km of reefs have been documented in 25 countries. Main focus of the survey is to capture the condition of the benthic community on the world’s coral reefs to create a baseline record against which future changes can be mapped. The imagery is available to the public on the Global Reef Record . The benthic community composition is analysed with the help of CoralNet an online repository and resource for benthic image analysis.

With FishNet we aim to quantify fish populations as seen in the images on the Global Reef Record. FishNet is currently been developed in collaboration with Prof. Franz’s laboratory at the Institute for Optical Systems, HTWG Konstanz in Germany. The idea is to use computer vision and machine learning to automatically extract fish numbers, identity and possibly even biomass. At present this project is in the development phase which relies heavily on the manual annotation of fish found in the images. In order to speed up the process of building the automated system, we have included the project in a massive open online course (MOOC) by Edx (Tropix 101x).

The plan is to also have a separate online tool available for people interested in identifying fish to help out with the project. This should be happening in the near future. 

Collaborators and students