AI systems fail to read clocks and decode calendars

Conducted by a team at Edinburgh University, the research investigated the capabilities of multimodal large language models (MLLMs) to answer time-related questions by looking at pictures of clocks or calendars. The AIs were tested using various clock designs, including some with Roman numerals, with and without second hands, and different coloured dials. It was found that the AI systems interpreted the correct clock-hand positions less than a quarter of the time.

Roman numerals or stylised clock hands induced more mistakes. According to the Edinburgh team, the AI systems did not perform any better when the second hand was removed, suggesting fundamental issues with hand detection and angle interpretation.

AI models were also tasked with a range of calendar-based questions, such as identifying holidays and working out past and future dates. It was found that even the best-performing AI model got date calculations wrong 20 per cent of the time. According to the researchers, the combination of spatial awareness, context and basic maths required to understand clocks and calendars is clearly a weakness of current AI models.