Description

This case study explores how Edinburgh University used open-source Drupal AI technology to develop and test three distinct chatbots: a search assistant, content design helper, and IT support tool – using a rigorous, user-centred research framework. Rather than building ‘another cool AI thing’, the project prioritised human needs from the outset. The session includes authentic insights from extensive user testing with staff and students in real contexts, revealing how people develop trust with chatbots and what drives their varied expectations and perceptions of chatbots. I’ll uncover how interaction patterns and user intent data informed design decisions, the critical role of pre-prompting and disclaimers in managing expectations, and how design patterns like onboarding wizards helped users understand each chatbot’s distinct purpose. Attendees will discover an iterative research methodology that balanced multiple variables while continuously improving chatbot performance with thoughtful prompt design changes and UI adaptations. I’ll share how we managed ethical challenges associated with conversational AI, including transparency, expectation management and responsible disclosure of AI capabilities and limitations, as well as practical tips for handling widespread chatbot requirements like encouraging understanding of chatbot purposes and capabilities, crafting conversational flow and ensuring error handling data validation however users express their intent.