Teaching machines to hear what the ear already knows

First published by The University of Adelaide

In aged care, disability support, and allied health settings, the ability to communicate clearly and be understood shapes the quality of care delivered and the experience of the people receiving it.

Yet while national averages show that 41 per cent of the workforce speak English as a second language [1], in major Australian metropolitan hubs, health providers report that a significantly greater proportion of their frontline staff come from primarily non-English speaking backgrounds. Until recently, there has been no tailored linguistic tool to help them develop the English language communication skills their work demands.

Adelaide-based company Vocare exists to close that gap. Founded by linguist and speech specialist Seshnie Taylor, Vocare helps migrant healthcare workers identify and correct pronunciation errors that arise from the transfer of speech habits from their first language to English.

"Many English as a second language learners never get one-on-one feedback on their pronunciation, the actual phonemes [units of sound that distinguish one word from another in a language] they produce," said Taylor. "This leads to being misunderstood, written off, or often feeling frustrated or isolated."

To address this issue, Taylor decided to upscale Vocare's operations. The company had primarily relied on manual annotation, listening by ear, to identify pronunciation differences from health support staff. Compounding the challenge was the absence of any automated tools trained on Australian accents.

Seeking a solution, Vocare partnered with AIML through the Industrial AI SME Grant Program, an AIML initiative supported by the SA Government via the Department of State Development 's Research and Innovation Fund. The program connects small and medium enterprises in South Australia with AIML's machine learning engineering expertise to explore how AI can solve real business problems at scale.

Taylor connected with the program in 2025 with a clear vision. She knew she did not want an off-the-shelf tool, but something built from the ground up to encode deep linguistic expertise into a scalable system.

"From the beginning, I wanted to build a defensible moat around Vocare," she said. "The opportunity was translating that expertise into something systematic, measurable, and ultimately defensible as proprietary technology."

"[The Industrial AI program] was a golden opportunity for us at the exact right time," said Taylor.

Building a phonologically aware machine learning engine

AIML Machine Learning Engineer Sam Koshy Thomas worked closely with Taylor to unpack the physical mechanics behind pronunciation errors that no existing automated system could detect.

"Seshnie was really keen on getting this product running and that made her really invested throughout the project," said Thomas. "She helped me understand speech by… pointing out what these errors [sounded] like."

"We used [machine learning] models trained across more than 50 diverse accents to produce phonemes," Thomas explained. "This helped to exactly map… generated phonemes to phonological features,” or the system of sounds that form a language.

“[These features] encapsulate what physical action every phoneme produces, [such as] where you place your tongue, whether your lips are rounded, whether your vocal cords vibrate, or whether air flows through your nose."

The result was a system that moves well beyond simply flagging that a sound was incorrect. The tool pinpoints precisely where a learner’s pronunciation diverged from a native speakers at the level of tongue position, voicing, or airflow. The system was then delivered as an accessible user interface in Vocare's existing app.

Deployment, impact, and what comes next

The classification system now serves as one of the core engines behind Vocare's voice error detection platform and is integrated directly into their production workflows. In practice, it processes voice data in real time, identifies errors with high reliability, and returns outputs that can be used immediately.

"This has allowed us to embed [the system] seamlessly into our existing infrastructure, rather than treating it as a separate or experimental component," said Taylor.

"The speed and accuracy of the system mean we can increasingly rely on it in high-volume environments, reducing manual intervention, and improving overall efficiency."

For Vocare, the implications reach well beyond operational efficiency.

"Our API [application programming interface] is now significantly more accurate and faster, enabling us to deliver more reliable outcomes for our users," she said. "Most importantly, this has translated into real-world impact. The people we serve are benefiting from quicker responses, improved precision, and a more seamless experience overall."

Looking ahead, Taylor sees the project as establishing something more strategically significant than a single product improvement.

"By building a sovereign, Australian-built and owned capability in voice error detection, we are establishing a locally governed alternative that prioritises performance, security, control, and scalability," she said. "Over time, we see this capability supporting wider adoption of reliable, real-time voice intelligence in sectors where accuracy and trust are critical."

For AIML, the project demonstrated the value of pairing machine learning expertise with deep domain knowledge and the kind of outcome that becomes possible when both sides of a partnership are equally invested.

"The clarity [Seshnie] had towards her problem helped me understand the problem better as well," said Thomas. "It was a real pleasure working with her."

Taylor, too, reflected on what the collaboration had meant for Vocare.

"This collaboration has been hugely beneficial — not just from a technical standpoint, but in helping us build sovereign capability and move closer to our mission with tangible, measurable results," she said. "There is no doubt that this partnership positively impacted Vocare, and we are very grateful for this opportunity.”