ABS Federated Data Hub Access and Evaluation Projection

ABS Federated Data Hub Access and Evaluation Project

This project provides a demonstration of a novel approach to accessing ABS Census data through an online interface that enables the interrogation, manipulation,  spatio-statistical analyses, visualisation and modelling of the data. The resulting interface to ABS datasets and associated suite of e-research tools (being developed through AURIN) will be incorporated into the AURIN infrastructure.
This collaboration between the Australian Bureau of Statistics (ABS), AURIN, the University of Queensland, University of Canberra, the University of Adelaide and the University of Newcastle, demonstrates how this innovation can improve access to the 2011 Census data and other ABS products, through programmatic connection via Statistical Data and Metadata Exchange (SDMX) services. Essentially the project implements and evaluates a machine-to-machine data hub model for data deliverable to clients external to the ABS.

Project Overview

This project clearly demonstrates the benefits and challenges associated with the ABS’s development of production level statistical data hubs.

The project incorporates four demonstrators that provide examples of ‘stories’ that can be told by enabling improved access to ABS data, focusing on the 2011 Census data and integrating it with other spatial datasets. This demonstrates the advantages of providing online web services to access this data in a programmatic way and for measuring the re-use of data.

This project also demonstrates the existing and future benefits of investment in ABS infrastructure – specifically building infrastructure that is based on open standards such as Statistical Data and Metadata Exchange (SDMX). The project provides the initial experimental base from which a case can be developed to influence decision-makers about funding research and statistical infrastructure, involving initiatives such as AURIN, and through systemic development of capabilities by the ABS for online access and interrogation of data by the user.

The end users of the project are the social sciences research community, who now have faster, streamlined and more efficient access to key ABS and other datasets via the AURIN Portal. Researchers are now able to access statistical analysis and visualisation services through the AURIN Portal in a time efficient manner. The outcome enables researchers to address key urban and regional policy and planning research questions, in a timely fashion, based on empirical evidence.


Project Output

The main objective of the project is to demonstrate innovation in enabling enhanced access to ABS data focusing on the 2011 Census through a federated data hub model for delivering statistical data across Australia and to evaluate the benefits of that innovation.

The diagram below shows how the AURIN Portal user interacts through the user interface when retrieving ABS data. The diagram shows only the AURIN components related to retrieval/storage of data, metadata and structures from the ABS.Stat server. Note that the SDMX Java client does not interact with the Data Registry component directly when retrieving data from ABS.

Flow chart

Specific objectives of the project are to:

  • Target the research sector through the four Demonstrators and produce a report that clearly articulates: the benefits of delivering ABS data via web services; any impediments that currently exist in ABS infrastructure; and any improvements that are required
  • Produce a report for ABS that articulates any technological limitations of current ABS systems and potential investments required to deliver a seamless flow of data from ABS infrastructure through enabling technologies such as ABS .Stat web services

The project established a collaboration between ABS, AURIN and a number of established entities in universities with expertise in spatial data-focused research and the development and provision of eResearch capabilities. Those entities are the University of Queensland (eResearch Lab), The University of Adelaide (Australian Population and Migration Research Centre – APMRC), University of Canberra (National Centre for Social and Economic Modelling – NATSEM), and the University of Newcastle (Centre for Full Employment and Equity – CofFEE). The University of Melbourne was involved as the Lead Agent for the AURIN project.

The project focuses on four demonstrators:

  • Formulating benchmark and employment related measures for 2011 Census data across Australia (eResearch Lab, University of Queensland)
  • Demographic and aged related issues: Provision and access to public toilet facilities (APMRC, University of Adelaide)
  • Identifying clusters of low income households and identifying the spatial correlation between low income and other spatial variables in South Australia (NATSEM, University of Canberra)
  • Analysing spatial dependence in unemployment and employment growth (CofFEE, University of Newcastle)

 

The approach taken to achieve the project’s objectives involves:

  • Confirm the Demonstrators’ data requirements and load Census datasets into ABS.Stat web service
  • Redefine the design of the data architecture and web service design to accommodate ABS.Stat SDMX web service into the AURIN Portal
  • Establish an approach for ingesting metadata into the AURIN Portal and citing and managing derived data
  • Identify, develop and integrate the required data derivation, analysis and visualisation tools and services into the AURIN Portal
  • Undertake four demonstrator projects to show the utility of the ABS Data Hub and integration with the AURIN Portal
  • Identify what could be done to improve the performance/usability of ABS data hubs to meet end-user needs

Project Team Overview

This project is being run by the eResearch Lab in the School of Information Technology and Electrical Engineering (ITEE) at the University of Queensland. The eResearch Lab comprises a team of academics, PhD students and software engineers who are developing innovative eResearch services to enable more efficient and collaborative data sharing, integration and analysis in order to accelerate scientific discovery. The eResearch Lab currently collaborates across a range of disciplines including bioinformatics, environmental sciences, materials science and social sciences.

Demonstrator Projects

This project involves four demonstrator projects. The leaders for each of the Demonstrator Projects are:

Tung-Kai (Paul) Shyy is a Research Fellow in the e-Research Group in the School of ITEE at The University of Queensland in Brisbane, Australia. His research interests include GIS-based spatial modelling and visualisation, spatial optimisation, crime mapping, urban quality of life and web-based GIS applications.

Dr. Lisel O’Dwyer is currently a senior researcher in the School of Social and Policy Studies at Flinders University. Her research interests include social processes with equity, health, inclusion and social justice implications such as volunteering, and provision of services such as public toilets that facilitate mobility and social participation, human-animal relations (particularly where these intersect with human health) and ageing. She is a strong proponent of mixed methods research designs and the use of spatial perspectives.

Professor Robert Tanton has a PhD in Economics from the University of Canberra, and has been at NATSEM since June 2005. Robert’s main interest is spatial disadvantage and he is a recognised Australian and international expert in this area. His principal areas of research are spatial disadvantage and wellbeing; spatial statistics and small area estimation; and community wellbeing. Robert also leads work using a spatial microsimulation technique to derive small area estimates of many indicators of disadvantage and wellbeing. So far, this technique has been used to derive small area estimates of poverty and housing stress; subjective wellbeing; and indigenous disadvantage.

Professor Bill Mitchell holds the Chair in Economics and is the Director of the Centre of Full Employment and Equity (CofFEE), an official research centre at the University of Newcastle. Robert has published extensively in the academic literature on spatial disadvantage and social exclusion, and has presented results of his work to a number of national and international conferences. The Centre seeks to promote research aimed at restoring full employment and achieving an economy that delivers equitable outcomes for all.

Jane HunterProf Jane Hunter (Main Contact)
Director
eResearch Lab
The University of Queensland
07 3365 1092
j.hunter@uq.edu.au
Paul ShyyTung-Kai (Paul) Shyy
Research Fellow
School of ITEE
The University of Queensland
07 3365 4337
t.shyy@uq.edu.au
Lisel-ODwyer-2Dr Lisel O’Dwyer
Senior Researcher
School of Social and Policy Studies
Flinders University
08 8313 1103
lisel.odwyer@flinders.edu.au
Robert Tanton betterProf Robert Tanton
Research Director (Regional and Urban Modelling)
NATSEM
University of Canberra
02 6201 2769
robert.tanton@canberra.edu.au
Bill MitchellProf Bill Mitchell
Director
The Centre of Full Employment and Equity (CofFEE)
University of Newcastle
0419 422 410
bill.mitchell@newcastle.edu.au

 


Project partners

UQ Print UoA_col_horz_crop
UON_ALT_MONO CofFEE Logo Only Blue

Further Reading

  • Hunter, J, 2014, Recommendations for the citation and management of derived data within AURIN, eResearch Lab University of Queensland, <http://espace.library.uq.edu.au/view/UQ:351984>
  • Stimson, R & Shyy, T-K, 2013, ‘And now for something different: modelling socio-political landscapes’, The Annals of Regional Science, vol. 50, no. 2, pp. 623-643.
  • McCrea, R, Shyy, T-K & Stimson, RJ, 2014, ‘Satisfied Residents in Different Types of Local Areas: Measuring What’s Most Important’, Social Indicators Research, vol.118, no. 1, pp. 87-101.