Early parent–child interactions lay the groundwork for communication, social and cognitive development. Yet many of the behaviours that signal how a child is developing are subtle, fleeting and difficult for untrained eyes to interpret.
Researchers at the Singapore University of Technology and Design (SUTD) set out to test whether artificial intelligence (AI) could learn to observe these interactions the way speech-language pathologists (SLPs) assess children's communication development.
Their study, “Towards Aligning Multimodal LLMs with Human Experts: A Focus on Parent–Child Interaction”, examined whether AI systems that process video, audio and text could match the observational practices of experienced SLPs. The team focused on joint attention—a key developmental behaviour closely tied to language and social communication.
Joint attention refers to moments when a child and another person share focus on the same object or activity while recognising that the other person is engaged too. For example, a child may look at a toy, glance at a parent, and then look back at the toy, creating a shared moment of attention.
As joint attention appears early in life, it is widely seen as a marker of communication development. But spotting and interpreting these behaviours takes specialist training.
"Developmental delays affect an estimated 10-15% of preschool children, with language delays among the most visible early concerns. Yet access to speech-language assessment and intervention is limited by a shortage of specialists, long wait times, and uneven support for families seeking early help," said Assistant Professor Kenny Choo from the Information Systems Technology and Design pillar at SUTD.
"The expertise that could catch a concern early often isn't within reach of the families who need it. We wanted to explore whether AI could help by learning to observe interactions the way experts do."
For the study, the team worked with three experienced SLPs to understand how they evaluate joint attention during parent–child interactions. They then used those insights to build an AI workflow around three key behavioural cues: where a child looks, what the child does, and what the child says.
The AI first had to describe these behaviours before attempting to assess the quality of joint attention in each interaction.
The results revealed a clear distinction between what AI does well and where it falls short.
The system achieved approximately 81 percent accuracy when identifying behavioural cues such as gaze, actions and vocalisations. These observations closely matched how SLPs describe the same behaviours in their own assessments.
However, when the AI had to go beyond observation and make expert-level judgements about the quality of joint attention, performance dropped substantially.
Perhaps surprisingly, the researchers found that the challenge was not solely an AI problem.
"The most striking finding was about the experts, not the AI," said Assistant Professor Choo.
"We assumed the hard part would be getting a model to match expert judgement against a clear standard. Instead, we found that experienced SLPs often agreed on what they saw, but differed on what it meant. One focused heavily on eye contact, another prioritised emotional engagement, while another looked for communicative intent in less conventional behaviours."
This variability suggests that there is often no single "correct" interpretation for an AI to learn.
Rather than revealing a failure of AI, the findings highlight an important distinction between observation and judgement. Experts tend to agree on what they see, but their interpretations can differ based on experience, training and clinical perspective.
For the researchers, this distinction has practical implications for how AI should be designed for developmental support and healthcare settings.
"AI is well suited to the observational work—spotting cues, organising information and creating structured records," said Assistant Professor Choo.
"It is not suited to making the clinical call. That judgement should rest with the clinician. In this instance, AI should be designed to act as an assistant that surfaces evidence rather than asking it to make decisions it isn't equipped for."
In practice, the team sees AI as a support for clinicians, not a replacement for professional expertise.
An SLP reviewing a video of a parent and child playing, for example, could receive an AI-generated timeline showing where the child was looking, what actions they performed and when particular behaviours occurred. This would let clinicians focus on the most relevant moments while retaining responsibility for interpretation and assessment.
Such systems could also lighten the documentation load of developmental assessments by providing structured, timestamped observations that clinicians can review, edit and verify.
For families, particularly those with limited access to specialist services, future tools built on reliable behavioural observation could potentially help caregivers better understand the cues that matter in their children's development. However, the researchers emphasise that any parent-facing applications would need far more testing before they could be deployed.
The work also highlights the value of interdisciplinary research in designing AI systems that work alongside people rather than replacing them. By combining expertise in human-computer interaction, AI and developmental communication, the team sought to understand not only what AI can do, but also where human expertise remains indispensable.
Next, the team plans to expand the study with larger and more diverse groups of clinicians, longer recordings from real-world home and school settings, and broader developmental populations, including autistic children. They also aim to investigate whether future systems should be tailored to individual clinicians' assessment styles or shaped by consensus among multiple experts.
"Early parent–child interactions contain valuable signals about a child's development," said Assistant Professor Choo. "Our findings suggest that AI can help surface those signals more efficiently—and potentially support more structured, clinician-led assessment—but understanding what they mean still requires human judgement, context and care."
