Developing international policy standards for ‘automated empathy’ AI

We may marvel at the capacity of AI to create things for us, but we are still in the early days of the technology.

One strand of the ‘next-generation’ of AI applications can be described as ‘automated empathy’ systems. These use AI to respond appropriately to perceived human emotions and human psychology.

But with such developments come a whole host of ethical and practical questions. Will AI understand subtle cultural, racial or gender differences? How are we to ensure that individual’s needs receive an appropriate response? How can we ensure that ‘AI partners’ are programmed to serve our interests rather than someone else’s? Is current law doing a good enough job of protecting society and allowing us to enjoy the benefits of new technologies?

A research project led by Andrew McStay, Professor of Technology & Society at Bangor University is dedicated to critically understanding the implications of human-AI partnering and to developing technical standards and global governance of emulated empathy and human-AI partnering.

Professor McStay who leads the UK’s Emotional AI Lab, has a longstanding shared research portfolio with counterparts in Japan, where automated personal assistants and ‘social robotics’ have been of interest. The Emotional AI Lab also has a partnership with Japan’s National Institute of Informatics.

Current general-purpose artificial intelligence products are often marketed as ‘empathic partners’, ‘personal AI’, ‘co-pilots’, ‘assistants’, and related phrasing for ‘human-AI partnering’. Here, ‘emulated empathy’ is used within artificial intelligence systems for human-AI partnerships from work settings, for therapy, education, life coaching, legal problems, to fitness, entertainment, and more.

These systems raise ethical questions that are global in nature yet benefit from exploration of ethical thought from around the world. Some ethical questions such as transparency, accountability, bias and fairness are familiar, but others are specific and unique, including psychological interactions and dependencies, child appropriateness, fiduciary issues, animism, and manipulation through partnerships with general-purpose artificial intelligence systems.

Professor Andrew McStay

The research is funded by UKRI through Responsible AI UK.

The resulting IEEE (Institute of Electrical and Electronics Engineers) standard will define ethical considerations, detail good practices, and augment and complement international human rights and regional law.

Professor McStay added,

“We believe that the use of human-state measurement to engage with qualitative dimensions of human life is still in its infancy. Emotional AI, and wider automated human-state measuring, and use of large language models to gauge theory of mind, requires ongoing social, cultural, legal and ethical scrutiny.”