Chatbots should not be used at all for supporting mental health solutions, a researcher in artificial intelligence (AI) and technologies has warned.
Speaking at the Association of Medical Insurers and Intermediaries (AMII) Autumn Summit, Cubic Garden founder Ian Forrester emphasised that human support in mental health was vital.
He told Health & Protection that private medical insurance (PMI) providers should tread carefully in implementing AI when designing effective mental health solutions.
Insurers Health & Protection spoke to following following Forrester’s session suggested the risks of open source AI such as ChatGPT were far higher than when using specialised systems designed to deal with mental health.
Double red flags
Health & Protection asked Forrester specifically how providers should use AI in building effective health solutions, such as where the chatbot needs to check out from the process and whether they should be used at all.
Forrester responded by saying in his view that they should not be used at all.
“That is my view,” Forrester said.
“That really upsets a lot of people because that’s more people, it’s more load.
“If they have to be used, they should be used alongside someone who actually knows what they’re doing.”
Touching on recent cases where ChatGPT has encouraged young people to commit suicide, Forrester added: “There are things that this bot is saying to the person, saying it’s okay, you should be with me. Don’t tell anyone else.
“There are red flags. This is a double red flag.
“Someone should have been saying, this isn’t right. This needs to stop right now.
“That’s what I think. If you must use a chatbot.
“Why are we using a chatbot? There are other ways to access this information which doesn’t mean you have to chat with it in that way.”
Engaging people with poor mental health
Forrester added there are other potential ways to engage people who are suffering with their mental health.
“There are other ways to have a meaningful conversation or meaningful exchange that doesn’t require you to type in text,” he continued.
“And those haven’t even been explored yet.
“This is something where someone in this room could find a much better way to interact with people in a way that is all done through painting – we already know the benefits of doing an activity.”