Conserving the Unseen:
Data Gaps in the Atlas of Living Australia
Shandiya Balasubramaniam
Dax Kellie

I acknowledge the Traditional Owners of the lands on which we meet, the Kaurna people, as well as the Traditional Owners of the lands on which I live and work, the Wurundjeri people of the Kulin Nation. I pay my respects to their Elders past and present. I recognise the spiritual and cultural significance of land, water, and all that is in the environment to Traditional Owners, and their continuing connection to Country.

Lamprima imberbis Carter, 1926

Angela Bannon, South Australian Museum

This is Lamprima imberbis, and it belongs to a group of beetles commonly referred to as golden stag beetles, but this species doesn’t have a common name. It’s known from a single specimen that was collected almost 100 years ago. This specimen is a male and was collected from New South Wales, the female is unknown. No information was included about its abundance or ecology, and despite more recent attempts to search for this species, it’s never been recorded again. It is not listed as threatened, and has not had any conservation assessment. It could be extant but endangered, or it may be extinct. It is one species out of many examples I could give that illustrates a widespread and largely unresolved conservation management challenge: how do we address the conservation of poorly known species?

How do we address the conservation of poorly known species?

Open data infrastructures like GBIF and the ALA are invaluable sources of aggregated data that can be used for developing biodiversity indicators to inform conservation assessments and environemental policy, but they’re not immune to gaps and biases. Because the patterns that we see in biodiversity datasets are a reflection of where, when, and what we’ve chosen to observe and record, rather than being an exact copy of the biological world.

Our knowledge of biodiversity is spatially uneven…

Our knowledge of biodiversity is spatially uneven. This map shows relative species richness across the country based on data from the ALA, and you can see some clear patterns: there are fewer records as you move inland away from the coast, and the darkest grid cells correspond to capital cities. This is not unique to Australia - globally, the density of biodiversity records tends to be higher as you get closer to roads and population centres, compared to more remote areas. Sampling in remote areas is typically more expensive, and low population density means relatively fewer records get submitted by the public. Although this is a global trend, it’s particularly pronounced in Australia because the continent is so large and sparsely populated, and population centres are unevenly dispersed.

…and contains temporal gaps

Similarly, observations are unevenly distributed across time. These are counts of species represented by preserved specimens across the decades. And that spatial bias comes through again, but you can also see that there are frid cells where what’s been recorded fluctuates over time. This temporal variability in monitoring can have quite profound impacts on our understanding of species distributions. A large number of Australian birds are nomadic and have inconstant distributions year to year, with irregular movements that might be missed without consistent monitoring. Another thing to consider is that the distributions of many species changed following European colonisation, and many species are continuing to undergo rapid declines. And what this means is that distributions based on records from even just 1 - 2 decades ago may be unreliable evidence of their continuing occurrence.

“Endless forms most beautiful and most wonderful”

Darwin described the biological world as endless forms most beautiful and most wonderful constantly evolving, but it won’t be a surprise to anyone in this room that there are taxonomic gaps in biodiversity datasets.

Is the ALA mostly birds?

In the ALA, which holds over 150 million occurrence records, those endless forms look like birds. Almost 60% of records are of birds. Magpies alone make up 1.5% of the Atlas.

Is the ALA mostly birds? Yes.

In terms of coverage, the average number of records per species is over 100,000 for birds. When we look at invertebrates, it’s fewer than 200 records per species. What we record tends to be correlated with perceived charisma - birds and mammals are recorded more than invertebrates, plants, and fungi. Even among vertebrates, marsupials are recorded more than rodents and bats. For groups with good coverage, like a lot of bird species, we can model distributions and track changes in abundance over time. But for a lot of other groups, like invertebrates, we have much sparser information.

How many insects are effectively invisible?

If we take insects as an example, the amount of information we have varies enormously by order. The dark green squares show the percentage of species with more than 10 records, and the lighter green squares show the percentage with at least 1 record. For charismatic, easy-to-spot groups like dragonflies, we have at least some data for most species. But compare that with beetles: they’re the most species-rich insect group, with over 25,000 species in Australia, and we know almost nothing about over two thirds of them. Entire chunks of the insect world are basically invisible in national biodiversity analyses, and even more so in conservation decisions.

How can targeted efforts fill gaps in the biodiversity record?

Visualising these spatial, temporal, and taxonomic gaps, it’s clear that our picture of biodiversity is shaped by patterns of human effort, but it also gives us some idea of where we can direct our efforts to make the biggest difference in filling the gaps.

Queensland’s Christmas Beetle Plague (1935, December 25).
Sydney Mail (NSW : 1912 - 1938), p. 47

One example of these targeted efforts is the Christmas Beetle Count. Christmas beetles tend to make themselves known in midsummer, often spectacularly, by swarming around lights. In the early 1900s they were said to be so numerous in Sydney that branches of eucalypts would be bent under the weight of beetles.

Queensland’s Christmas Beetle Plague (1935, December 25).
Sydney Mail (NSW : 1912 - 1938), p. 47

This newspaper article from 1935 describes the Qld canefields being overrun with beetles, and describes how “beetling” could be very lucrative - good worked could average 10 pounds night, and it took 200 beetles to make a pound. But those annual swarms have disappeared, suggesting that there may have been a decline. But like many invertebrates, Christmas beetles haven’t been the focus of long-term monitoring, so there aren’t consistent records to tell us if these are natural population fluctuations.

Christmas Beetle Count

So Invertebrates Australia and the Australian Museum launched the Christmas Beetle Count, which encourages people across the country to record and upload sightings of these beetles, and when we look at the data that’s flowed into the ALA, the impact of the project is pretty clear. There’s a sharp rise in records that aligns with the launch of the project in 2022

Christmas beetles and friends

But what’s also interesting is that there’s been a concurrent rise in records for three other species, which are commonly misidentified as Christmas beetles. So there’s been this spillover effect of inadvertently increasing records for a suite of other non-target beetle species. This is especially significant for the Argentinean scarab, which is an invasive species and was first recorded here around the 1950s. These numbers suggest that the species could be far more common than we might have thought based on records from even just 5 years ago.

Michelle McFarlane, Museums Victoria

The flip side of going out and collecting new records, is making currently available records more accessible. There’s a wealth of data in collections institutions like museums and herbaria, which are in undigitised forms.

To address this, the ALA runs a program called the Australian Biodiversity Data Mobilisation Program, which provides funding to projects that improve access to biodiversity data through the ALA. So it’s about making data that we already have, more accessible and discoverable.

Australian Biodiversity Data Mobilisation Program:
Click Beetles, Museums Victoria (2024)

Lucy McMillan, Museums Victoria

One project that was funded through this program is the Museums Victoria Click Beetle project in 2024, where staff digitised over 6,000 click beetle records and uploaded them to the ALA. Prior to the project, the ALA had just over 11,000 records from 63 genera and 322 species of click beetles based on preserved specimens. This project added over 6000 new occurrence records and 1 genus and 60 species that had not been represented in the ALA before.

Australian Biodiversity Data Mobilisation Program:
Click Beetles, Museums Victoria (2024)

It’s estimated that there are about 800 species in this group, so we’re still lacking information on more than half the species in this family, but this one project alone was able to increase our species-level data by 18%.

Australian Biodiversity Data Mobilisation Program:
Click Beetles, Museums Victoria (2024)

This dataset is particularly valuable because differentiation to species level is very difficult for this group, it often requires close examination of small body structures so records based on preserved specimens are likely to be more reliable or better able to be identified to a lower taxonomic rank. This becomes very clear if you look at the breakdown of records based on record type - a lot more of the preserved specimens are identified down to species level, whereas this proportion is lower for human observations.

Sharing data is hard…

Munroe, R (2019) How To: Absurd Scientific Advice for Common Real-World Problems

Research data is the other type of biodiversity data that doesn’t always see the light of day. It sits on a hard drive or in the supplementary table of a paper, and is generally inaccessible. And that’s because sharing data is hard - if you want to submit it to infrastructures like GBIF or the ALA, it needs to align with a data standard called Darwin Core. But this isn’t a format that most people are familiar with, so it can be difficult to know where to start.

galaxias makes it easier

{galaxias} streamlines the process of converting biodiversity data into Darwin Core Archives, allowing users to more easily submit data to infrastructures such as the ALA or GBIF

We built galaxias to help with this - it’s a package in R and Python that steps you through the process of converting data into the right format, putting everything together in the right place, and bundling it all up.

Darwin Core Archive

An archive is a .zip file containing three things:

Living Atlases require data to be submitted as a Darwin Core Archive, which is essentially a zip file containing three things: the dataset, metadata, and a schema.

Darwin Core Archive

An archive is a .zip file containing three things:

galaxias has functions that will scan your dataset and give you a suggested workflow to meet the minimum requirements. This might involve renaming columns to match standard names, or adding new columns to include particular types of information. These are easily done in existing R and Python workflows.

Darwin Core Archive

An archive is a .zip file containing three things:

You can write a metadata statement in plain text, and galaxias will convert a markdown file to EML format.

Darwin Core Archive

An archive is a .zip file containing three things:

Once those are done, galaxias automatically builds a schema file and zips everything up into a Darwin Core Archive. galaxias handles all the file conversions and file management through this process, so everything’s always in the right place. Our hope is that this will encourage more researchers to make their data accessible. If you’d like to know more about galaxias, please come and talk to us, Dax and I are at the ALA booth where we have a poster that goes into more detail. And if you’d like to test out galaxias with your own dataset, we’d love to chat about it so please stop by.

Shandiya Balasubramaniam
Decision Support Program Lead
Atlas of Living Australia
shandiya.balasubramaniam@csiro.au
shandiya.bsky.social
shandiya

Slides:

shandiya.quarto.pub/esa2025