Continental-scale biodiversity data assessment using the Atlas of Living Australia Shandiya Balasubramaniam
@ShandiyaB
VicBioCon2023

I acknowledge the Traditional Owners of the lands on which I live and work, the Wurundjeri People of the Kulin Nation, and pay my respects to Elders past and present. I recognise the spiritual and cultural significance of land, water, and all that is in the environment to Traditional Owners, and their continuing connection to Country.

The Atlas of Living Australia was launched in 2010 and is now the most comprehensive dataset on Australian biodiversity, with over 100 million records sourced from more than 800 data providers, which includes collections like iNaturalist, BirdLife, and the Victorian Biodiversity Atlas. So it aggregates data from natural history collections, government departments, researchers, community groups, and individuals, and makes it all available online for anyone to access.

{galah}

The first project I wanted to introduce is the {galah} package, which acts as an interface to the biodiversity data in the ALA. The majority of these data consist of species occurrence records…

2020-09-23

…so, evidence of a taxon in a particular time and place. It’s possible to get these records through the website, but one of the main advantages of using {galah} is that it makes the query reproducible. It’s also implemented in both R and Python- the R version currently has more functionality but a fully-fledged Python release is going to be happening very soon.

Counts of records

• R
• Python

library(galah)

galah_call() |>
  atlas_counts()

# A tibble: 1 × 1
      count
      <int>
1 113225978

import galah
galah.atlas_counts()

I’ll run through some examples using the R syntax, but the Python code is here in this tab, and these slides are available online.

So, start by loading the package, and every query begins with galah_call(), which tells the API what we want. A lot of the syntax is similar to using dplyr, so if you’re familiar with the tidyverse this will be quite straightforward. The next line specifies what we want - here, atlas_counts() will retrieve counts of records. Voila, we’re told that there are just over 113 million records in the ALA. But we don’t always want everything, everywhere, all at once. So here are some functions that allow us to construct more specific queries

Identify taxa

• R
• Python

library(galah)

galah_call() |>
  galah_identify("pardalotus") |>
  atlas_counts()

# A tibble: 1 × 1
    count
    <int>
1 1019663

import galah
galah.atlas_counts(taxa="pardalotus")

galah_identify() lets us specify one or more taxa - here, I’ve asked for the genus pardalotus. Over a million pardalote records

Filter records

• R
• Python

library(galah)

galah_call() |>
  galah_identify("pardalotus") |>
  galah_filter(year >= 2020, 
               stateProvince == "Tasmania") |> 
  atlas_counts()

# A tibble: 1 × 1
  count
  <int>
1  7016

import galah
galah.atlas_counts(
    taxa="pardalotus",
    filters=["year >= 2020", "stateProvince == Tasmania"]
)

galah_filter() applies filters to different fields - here, I’ve specified records from 2020 onwards, from Tasmania. Down to 7000 records. But maybe you want to know if there are different species in there, or if the counts change over the years…

Group counts

• R
• Python

library(galah)

galah_call() |>
  galah_identify("pardalotus") |>
  galah_filter(year >= 2020, 
               stateProvince == "Tasmania") |> 
  galah_group_by(year, species) |>
  atlas_counts()

# A tibble: 10 × 3
   species                 year  count
   <chr>                   <chr> <int>
 1 Pardalotus punctatus    2021   2321
 2 Pardalotus punctatus    2020   1381
 3 Pardalotus punctatus    2022     23
 4 Pardalotus striatus     2021   1694
 5 Pardalotus striatus     2020   1315
 6 Pardalotus striatus     2022     19
 7 Pardalotus striatus     2023      1
 8 Pardalotus quadragintus 2021    133
 9 Pardalotus quadragintus 2020    123
10 Pardalotus quadragintus 2022      6

import galah
galah.atlas_counts(
    taxa="pardalotus",
    filters=["year >= 2020", "stateProvince == Tasmania"],
    group_by=["year","species"]
)

group_by() lets you group counts by the fields you specify - here, year and species. We can see that there are 3 species, and there’s quite a difference in the number of records.

Download occurrences

• R
• Python

library(galah)
galah_config(email = "shandiya.b@gmail.com")

galah_call() |>
  galah_identify("pardalotus") |>
  galah_filter(year >= 2020, 
               stateProvince == "Tasmania") |> 
  atlas_occurrences()

# A tibble: 7,016 × 8
   decimal…¹ decim…² eventDate           scien…³ taxon…⁴ recor…⁵ dataR…⁶ occur…⁷
       <dbl>   <dbl> <dttm>              <chr>   <chr>   <chr>   <chr>   <chr>  
 1     -43.6    147. 2021-02-09 13:00:00 Pardal… https:… 5cb452… eBird … PRESENT
 2     -43.6    147. 2021-02-09 13:00:00 Pardal… https:… 0308ea… eBird … PRESENT
 3     -43.6    147. 2020-12-17 13:00:00 Pardal… https:… 586de7… eBird … PRESENT
 4     -43.6    147. 2021-01-19 13:00:00 Pardal… https:… 3ce6c9… eBird … PRESENT
 5     -43.6    147. 2020-12-16 13:00:00 Pardal… https:… d7ce89… eBird … PRESENT
 6     -43.6    147. 2022-03-26 23:59:00 Pardal… https:… 859aae… iNatur… PRESENT
 7     -43.6    147. 2020-12-16 13:00:00 Pardal… https:… cc741f… eBird … PRESENT
 8     -43.6    147. 2020-12-16 13:00:00 Pardal… https:… 08e3d1… eBird … PRESENT
 9     -43.5    147. 2021-02-08 13:00:00 Pardal… https:… 38f948… eBird … PRESENT
10     -43.5    147. 2020-12-15 13:00:00 Pardal… https:… a2c7b1… eBird … PRESENT
# … with 7,006 more rows, and abbreviated variable names ¹​decimalLatitude,
#   ²​decimalLongitude, ³​scientificName, ⁴​taxonConceptID, ⁵​recordID,
#   ⁶​dataResourceName, ⁷​occurrenceStatus

import galah
galah.galah_config(email="shandiya.b@gmail.com")
galah.atlas_occurrences(
    taxa="pardalotus",
    filters=["year >= 2020", "stateProvince == Tasmania"]
)

To download these records and have a look at them in more detail, we can switch from atlas_counts() to atlas_occurrences(). Downloading records also requires that we provide the email we used to register with the ALA. This is a bit truncated, but we can see that the download consists of 7000+ rows and 8 columns.

Select fields

• R
• Python

library(galah)
galah_config(email = "shandiya.b@gmail.com")

galah_call() |>
  galah_identify("pardalotus") |>
  galah_filter(year >= 2020, 
               stateProvince == "Tasmania") |> 
  galah_select(species, 
               year, 
               dataResourceName) |> 
  atlas_occurrences()

# A tibble: 7,016 × 3
   species               year dataResourceName
   <chr>                <dbl> <chr>           
 1 Pardalotus punctatus  2021 eBird Australia 
 2 Pardalotus punctatus  2021 eBird Australia 
 3 Pardalotus punctatus  2021 eBird Australia 
 4 Pardalotus punctatus  2021 eBird Australia 
 5 Pardalotus punctatus  2020 eBird Australia 
 6 Pardalotus punctatus  2021 eBird Australia 
 7 Pardalotus punctatus  2021 eBird Australia 
 8 Pardalotus punctatus  2021 eBird Australia 
 9 Pardalotus punctatus  2021 eBird Australia 
10 Pardalotus punctatus  2021 eBird Australia 
# … with 7,006 more rows

import galah
galah.galah_config(email="shandiya.b@gmail.com")
galah.atlas_occurrences(
    taxa="pardalotus",
    filters=["year >= 2020", "stateProvince == Tasmania"],
    fields=["species","year","decimalLatitude","decimalLongitude"]
)

We might not want all the columns that are included in the default download, or we might want some different ones; we can specify this with galah_select() - here, we have records with information about species, year, and the data provider.

Geolocate

• R
• Python

library(galah)
library(sf)

melb <- st_read("data/metro_region.shp", 
                quiet = TRUE) |> 
  st_simplify(dTolerance = 1000)

galah_call() |> 
  galah_identify("pardalotus") |> 
  galah_geolocate(melb) |> 
  atlas_counts()

# A tibble: 1 × 1
  count
  <int>
1 93217

Watch this space!

{galah} also lets you search within a bespoke location; with galah_geolocate, you can provide a shapefile, well known text string, bounding box, or simple features object. In this example here, we load the {sf} package, then read in a shapefile of the melbourne metro region. Then it’s’ just a case of providing this as an argument to galah_geolocate, and we have 93 thousand pardalotes ever recorded in Melbourne.

galah.ala.org.au

This is just some of the functionality of the package, there’s lots of other cool things that you can do with it, like apply data quality profiles, or query other living atlases, and that’s all documented on the website.

ALA labs

The second project I’d like to draw your attention to is ALA labs…

…which is a resource for learning how to do things with data from the Atlas. The idea is to showcase and share tools for open, reproducible workflows

For example, Fonti and Margot wrote an article on convex and alpha hulls, which can be good options for visualising species distributions where there are very few existing observations. Recently, this method was used to estimate the impact of the 2020 bushfires on invertebrates in southern Australia. In this post, they walk though the process of downloading records of data-deficient species from the Atlas, and the steps involved in predicting species distributions using convex and alpha hulls, which can be a useful metric for conservation planning.

As you can see here, the science team at the Atlas have written articles describing a range of data visualisations and ecological analyses, and we’re continually adding to these. And if there’s something you think we should write about, let us know!

EcoAssets

The third project I’m introducing today is EcoAssets, which is a little bit different in terms of scale. EcoAssets brings together 3 research infrastructures…

…the Atlas of Living Australia, the Terrestrial Ecosystem Research Network, and the Integrated Marine Observing System, to provide data in forms that support environmental reporting, such as the State of the Environment Report.

TERN measures terrestrial ecosystem attributes at representative locations and produces datasets at a national scale, and IMOS collects data on all aspects of the marine environment, with extensive monitoring activities in coastal and open waters.

Historically, data from these activities have certainly fed into environemental reporting. Notably, the 2016 State of the Environement report referenced particular datasets from these 3 organisations, but it was difficult to access the data directly, and there was no opportunity down the track for researchers to evaluate and review the conclusions that were reached using these data

EcoAssets addresses this by delivering standardised and comparable data that can be versioned, so we can look at how things are changing over time. Records from these organisations are filtered and aggregated to create summary datasets, essentially counts of species and records since 1900, that have been faceted across areas of interest, like protected areas and bioregions, and according to whether species are of conservation concern, or classed as introduced or invasive.

ibraRegion	yearStart	yearEnd	griisStatus	recordCount	speciesCount
Arnhem Coast	1900	1970	Introduced	206	19
Arnhem Coast	1900	1970	Invasive	11	3
Arnhem Coast	1900	1970	Native	7754	1791
Arnhem Coast	1971	1975	Introduced	49	19
Arnhem Coast	1971	1975	Invasive	1	1
Arnhem Coast	1971	1975	Native	5570	1658
Arnhem Coast	1976	1980	Introduced	11	8
Arnhem Coast	1976	1980	Invasive	2	2
Arnhem Coast	1976	1980	Native	5945	1163
Arnhem Coast	1981	1985	Introduced	15	10
Arnhem Coast	1981	1985	Invasive	1	1
Arnhem Coast	1981	1985	Native	3032	1116
Arnhem Coast	1986	1990	Introduced	72	26
Arnhem Coast	1986	1990	Invasive	12	3
Arnhem Coast	1986	1990	Native	14929	1736
Arnhem Coast	1991	1995	Introduced	199	41
Arnhem Coast	1991	1995	Invasive	20	6
Arnhem Coast	1991	1995	Native	12067	1572
Arnhem Coast	1996	2000	Introduced	602	59
Arnhem Coast	1996	2000	Invasive	152	6
Arnhem Coast	1996	2000	Native	26124	1849
Arnhem Coast	2001	2005	Introduced	323	45
Arnhem Coast	2001	2005	Invasive	103	5
Arnhem Coast	2001	2005	Native	14594	1490
Arnhem Coast	2006	2010	Introduced	197	38
Arnhem Coast	2006	2010	Invasive	107	9
Arnhem Coast	2006	2010	Native	33248	1649
Arnhem Coast	2011	2015	Introduced	485	52
Arnhem Coast	2011	2015	Invasive	63	5
Arnhem Coast	2011	2015	Native	27975	1350
Arnhem Coast	2016	2020	Introduced	66	10
Arnhem Coast	2016	2020	Invasive	34	3

For example, this is a summary of species occurrence from 1900 to 2020, for Australian terrestrial species organised by bioregion and invasive status.

ibraRegion	yearStart	yearEnd	griisStatus	recordCount	speciesCount
Arnhem Coast	1900	1970	Introduced	206	19
Arnhem Coast	1900	1970	Invasive	11	3
Arnhem Coast	1900	1970	Native	7754	1791
Arnhem Coast	1971	1975	Introduced	49	19
Arnhem Coast	1971	1975	Invasive	1	1
Arnhem Coast	1971	1975	Native	5570	1658
Arnhem Coast	1976	1980	Introduced	11	8
Arnhem Coast	1976	1980	Invasive	2	2
Arnhem Coast	1976	1980	Native	5945	1163
Arnhem Coast	1981	1985	Introduced	15	10
Arnhem Coast	1981	1985	Invasive	1	1
Arnhem Coast	1981	1985	Native	3032	1116
Arnhem Coast	1986	1990	Introduced	72	26
Arnhem Coast	1986	1990	Invasive	12	3
Arnhem Coast	1986	1990	Native	14929	1736
Arnhem Coast	1991	1995	Introduced	199	41
Arnhem Coast	1991	1995	Invasive	20	6
Arnhem Coast	1991	1995	Native	12067	1572
Arnhem Coast	1996	2000	Introduced	602	59
Arnhem Coast	1996	2000	Invasive	152	6
Arnhem Coast	1996	2000	Native	26124	1849
Arnhem Coast	2001	2005	Introduced	323	45
Arnhem Coast	2001	2005	Invasive	103	5
Arnhem Coast	2001	2005	Native	14594	1490
Arnhem Coast	2006	2010	Introduced	197	38
Arnhem Coast	2006	2010	Invasive	107	9
Arnhem Coast	2006	2010	Native	33248	1649
Arnhem Coast	2011	2015	Introduced	485	52
Arnhem Coast	2011	2015	Invasive	63	5
Arnhem Coast	2011	2015	Native	27975	1350
Arnhem Coast	2016	2020	Introduced	66	10
Arnhem Coast	2016	2020	Invasive	34	3

With this summary, it’s simple to visualise changes over time and space, for instance with a heatmap like the one on the right.

These data also let us interrogate how we collect evidence of an occurrence. These summaries include information on the type of record, and for much of the 20th century, our evidence for an occurrence came from museum specimens. More recently, this has changed in favour of human observations, which includes data collected by ecologists, field monitoring programs, and increasingly citizen science projects

This shows the proportion of records that are made up by human observations between 1900 and 1970

… and we can look at how this changes over time… And this is important because it could have implications for users in deciding what sorts of records to include in analyses.

ecoassets.org.au

And these datasets are all available on the website

Shandiya Balasubramaniam
Data Analyst • Science & Decision Support • ALA
shandiya.balasubramaniam@csiro.au
@ShandiyaB
shandiya

Slides: shandiya.quarto.pub/vicbiocon2023

I’d like to acknowledge the rest of the science and decision support team - everything I’ve talked about today has been a joint effort. And if you’d like to take another look at anything, the link to the slides is there. Thank you.