Mitchell Harley and his team are manually sorting visual data to train the AI tool RipEye.
Scientia Associate Professor Mitchell Harley is an expert in coastal technology. Based at the UNSW Water Research Laboratory (WRL) in Manly Vale, he uses innovative technologies to interpret visual data, to understand and manage coastal hazards.
In 2017, Mitch founded CoastSnap – a community-driven initiative that invites beachgoers to take an image of the beach with their smartphones in strategically placed camera cradles. These images are aggregated with others in the app and help to monitor changing coastal landscapes over time. It is now the largest coastal monitoring program worldwide with more than 650 stations in 37 countries.
A CoastSnap camera cradle at Manly Beach.
While this project has been hugely successful, accruing and categorising enough data to accurately monitor rip currents remains a challenge. This problem led Mitch and his team to develop RipEye.
“Using cameras to measure rip currents is cumbersome and not very flexible. It requires a trained human eye to mine through the database and identify rips from images, then organise them.
“We started noticing some interesting AI tools, particularly coming out of driverless car technology where cameras had the ability to detect objects in real-time. We realised this could be adapted to detecting rip currents,” he said.
Collaboration, computer science, coding and RipEye
Working with an interdisciplinary research team, Mitch leads the development of RipEye – an AI-powered tool that detects rips through footage from smartphones. While Mitch’s research usually focuses on environmental impacts, he recognised the crossover and how his research could support beach safety, so RipEye focuses on reducing the number of Australian drownings each year.
RipEye is a collaborative project, with a number of UNSW academics like Scientia Professor Toby Walsh from the School of Computer Science and Engineering, Dr Amy Peden from the School of Population Health and Professor Rob Brander (also known as Dr Rip) from the School of Biological, Earth and Environmental Sciences (BEES), helping to develop the tool.
Mitch has also connected with Scientia Associate Professor Yang Song from the School of Computer Science and Engineering. Mitch said this is “a successful example of collaboration through the UNSW Scientia Program”.
“The Program is a good opportunity to connect the dots between your own research and others whose expertise or research may support it. I call myself an accidental AI person because it’s really this collaboration with Computer Science that’s allowed us to develop this technology. I’m very solution focused and have always sought to find the best way to identify rip currents, using whatever technology might be available,” he said.
The computer scientists are doing the hardcore coding to optimise algorithms, helping them get better at detecting rips by drawing on a large beach image database .
“At the Water Research Laboratory we’ve been collecting images of beaches for more than 25 years, so we have one of the largest databases worldwide. One of my PhD students has spent the first 12 months of their research mining through 10,000 images of different beaches at WRL,” he said.
Through this laborious task RipEye has learnt to detect rip currents and adapt its knowledge to changing conditions.
“A lot of AI tools are underpinned by humans performing cumbersome tasks to train them, particularly in vision data, and UNSW’s WRL is in a unique position because of its ‘treasure trove’ of imagery”, Mitch said.
Partnering with Surf Life Saving Australia
Australia’s coastline has 11,000 beaches, so having active patrol across all beaches throughout the year isn’t feasible.
Eventually Mitch hopes that RipEye will be available to anyone with a smartphone, to help beachgoers choose the safest spot to swim. For now, Surf Life Saving Australia (SLSA) will use the tool in controlled testing environments. SLSA plans to implement RipEye to assist in training lifeguards and, in turn, to help Mitch and his collaborators assess its accuracy.
“We’re about six months away from pilot testing this with various groups with SLSA. The tool currently has about 80% detection success which is impressive considering human rip detection is around 50% from a trained eye.
“The challenge is identifying different kinds of rips, particularly flash rips. These rapidly form and disappear, so there’s very little data on them and they’re hard to spot. That’s really the one thing that concerns me,” Mitch said.
As well as training, SLSA are planning to use RipEye to reach high-risk demographics like young males, school groups, regional communities and international students.
With a particular focus on unpatrolled beaches, the project aims to support SLSA’s goal of reducing drowning deaths by 50% by 2030.
- Log in to post comments