AI is providing mental health advice that at times violates proper code of ethics required of human therapists.
getty
In today’s column, I examine rising concerns that generative AI and large language models (LLMs) are providing mental health advice that violates the code of conduct and ethical standards required of human therapists.
The idea is that if AI is going to be allowed to give out mental health advice, the AI ought to be held to the same standards as expected of human therapists. Please know that some critics fervently believe that AI shouldn’t be providing mental health guidance at all. Period, end of story. But, since it is occurring, AI ought to have its feet held to the fire in terms of meeting therapeutic protocols. The argument is that AI makers should ensure that their AI abides by the same standards as professional therapists.
Recent research identified fifteen crucial ways that AI tends to violate such protective precepts. I share those fifteen ways with you and offer suggestions on how to deal with the disconcerting propensity.
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
AI And Mental Health Therapy
As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For a quick summary of some of my posted columns on this evolving topic, see the link here, which briefly recaps about forty of the over one hundred column postings that I’ve made on the subject.
There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors too. I frequently speak up about these pressing matters, including in an appearance last year on an episode of CBS’s 60 Minutes, see the link here.
Human-To-Human Therapy
You are probably generally aware that human therapists are expected to know and adhere to various codes of conduct. This makes abundant sense. If professional therapists could willy-nilly do whatever they wanted, it would undoubtedly be a disservice to those receiving mental health guidance. An ethics code or similar protocol provides a tangible means of alerting therapists to how they are expected to properly perform their practice.
As an example of these types of codes, the American Psychological Association (APA) has posted online its “Ethical Principles of Psychologists and Code of Conduct”, which is refined and modified from time to time. A preamble and introduction clarify the nature and scope of the Code of Conduct, including making these salient points (excerpts):
- “The Ethical Standards set forth enforceable rules for conduct as psychologists.”
- “This Ethics Code applies only to psychologists’ activities that are part of their scientific, educational, or professional roles as psychologists.”
- “As used in this Ethics Code, the term reasonable means the prevailing professional judgment of psychologists engaged in similar activities in similar circumstances, given the knowledge the psychologist had or should have had at the time.”
- “In the process of making decisions regarding their professional behavior, psychologists must consider this Ethics Code in addition to applicable laws and psychology board regulations.”
The substantive material that stipulates the numerous ethical rules and practices is composed into ten major sections:
- Section 1: Resolving Ethical Issues
- Section 2: Competence
- Section 3: Human Relations
- Section 4: Privacy and Confidentiality
- Section 5: Advertising and Other Public Statements
- Section 6: Record Keeping and Fees
- Section 7: Education and Training
- Section 8: Research and Publication
- Section 9: Assessment
- Section 10: Therapy
A therapist in training is usually taught these or other similar codes. Various therapist-client scenarios are played out to understand how the codes are to be applied. Though the code is not precise per se, and mainly offers broad guidance, the gist of the code is hopefully sufficient that budding therapists and seasoned therapists comprehend the boundaries of their endeavors.
AI Is Let Loose
Shifting gears, the most popular use nowadays of the major LLMs is for getting mental health guidance, see my discussion at the link here.
This occurs easily and can be undertaken quite simply, at a low cost or even for free, anywhere and 24/7. A person merely logs into the AI and engages in a dialogue led by the AI. The use of generic LLMs such as ChatGPT, Claude, Gemini, Llama, Grok, and others is a common example of using AI for mental health advice.
There are sobering worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Huge banner headlines in August of this year accompanied a lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement. Despite claims by AI makers that they are gradually instituting AI safeguards, there are still a lot of downside risks of the AI doing untoward acts, such as insidiously helping users in co-creating delusions that can lead to self-harm.
For the details of the OpenAI lawsuit and how AI can foster delusional thinking in humans, see my analysis at the link here. I have been earnestly predicting that eventually all of the major AI makers will be taken to the woodshed for their paucity of robust AI safeguards. Lawsuits aplenty are arising. In addition, new laws about AI in mental healthcare are being enacted (see, for example, my explanation of the Illinois law, at the link here, the Nevada law at the link here, and the Utah law at the link here).
Asking AI To Abide By Conduct Codes
It would seem reasonable to assert that generative AI should be devised to abide by the same codes of conduct that therapists do. AI makers are lax in this regard. Rarely do they go out of their way to infuse and/or activate such codes of conduct into their AI models.
An intriguing twist is that the larger models probably scanned online info about the numerous codes of conduct when the AI was first being data trained. To clarify, AI makers do huge and wide scans of the Internet to pattern match on human writing. This is what allows the AI to seemingly be fluent during interactions with the AI. Millions upon millions of stories, documents, narratives, poems, and the like are scanned and patterned on.
The crux in the context of mental health is that the AI likely scanned the codes of conduct that are publicly posted about requirements for performing human-to-human therapy. You can see that this has occurred by asking an LLM to tell you about codes of conduct regarding therapists. The odds are high that the AI will already have encountered the codes.
A gap that exists is that there is nothing that particularly compels or instructs AI to abide by those patterned codes. The result is that when you use AI for mental health advice, the patterned codes sit silently and somewhat out of reach. Only if you were to explicitly prompt the AI to access the patterns would the AI opt to bring them to the fore and possibly attempt to abide by them.
Research Identifies Significant Lapses
What might you find if you were to test major LLMs on whether they are abiding by established codes of conduct, such as the APA version cited above?
A recent research study identified fifteen notable lapses. In a study entitled “How LLM Counselors Violate Ethical Standards in Mental Health Practice: A Practitioner-Informed Framework,” Zainab Iftikhar, Amy Xiao, Sean Ransom, Jeff Huang, Harini Suresh, Proceedings of the Eighth AAAI/ACM Conference on AI, Ethics, and Society (AIES 2025), October 15, 2025, these salient points were made (excerpts):
- “Prior work in LLM-based mental health suggests that prompting models with evidence-based psychotherapeutic techniques improves their clinical performance, suggesting that appropriate prompting can scale psychotherapy and reduce practitioners’ workloads.”
- “However, there remains a lack of empirical investigation grounded in real-world interactions to understand whether such strategies help models adhere to ethical principles.”
- “In this study, we conducted an 18-month ethnographic collaboration with mental health practitioners (three clinically licensed psychologists and seven trained peer counselors) to map LLM counselors’ behavior during a session to professional codes of conduct established by organizations like the American Psychological Association (APA).”
- “Through qualitative analysis and expert evaluation of N = 137 sessions (110 self-counseling; 27 simulated), we outline a framework of 15 ethical violations mapped to 5 major themes.”
- “Psychotherapy provided by LLMs is not subjected to the same oversight that governs licensed mental health professionals, creating uncertainty around accountability, safety, and efficacy. Without clear legal guidelines or regulatory standards, LLM counselors, or broadly AI-driven therapy chatbots, risk deploying high-capability systems without adequate safeguards, potentially exposing users to unmitigated harm.”
As noted by the innovative study, major LLMs tend not to suitably adhere to codes of conduct that therapists are required to follow.
This is especially disturbing since something could be done about this sorrowful oversight. The AI could be devised to try and abide by the codes of conduct. AI makers have dropped the ball and often don’t even realize that this is a matter of serious concern that can be somewhat rectified.
I say that it is somewhat correctable because the AI might still violate codes of conduct, despite being told to abide by the codes, thus, there are still risks. That being said, something is better than nothing, and the AI could at least be shaped to fundamentally follow the codes.
Fifteen Lapses Into Five Themes
In the research study, the fifteen key lapses were organized into five overarching themes:
- Group A: Lack of contextual adaptation
- Group B: Poor therapeutic collaboration
- Group C: Deceptive empathy
- Group D: Unfair discrimination
- Group E: Lack of safety and crisis management
In brief, the first theme relates to the AI not sufficiently grasping the context of each individual user who is seeking therapeutic advice from the AI. The AI tends to generalize and not get to the specifics of a particular person inquiring about mental health guidance.
The second theme entails doing a poor job of conferring with a user in a collaborative manner. Often, the AI will blurt out or blab about doing this or that and not try to ferret out whether this advice makes sense for the specific user at hand. A deep and lengthier dialogue is likely more productive than a shallow and knee-jerk reaction by the AI.
For the third theme, deceptive empathy refers to the false pretense that the AI is acting as a human-like buddy or possesses sentience. AI makers purposefully have their AI emit wording that uses the words “I” and “me” as though the AI is a living being. Humans using the AI are tricked into anthropomorphizing the AI.
The fourth theme entails the AI basing therapeutic advice on inherent biases that came into the AI during the pattern matching process at the get-go. If the AI was data trained on content that represents discriminatory practices, the chances are that the AI is going to carry forward those biases.
Finally, the fifth theme has to do with weak or insufficient programming associated with ensuring the mental safety of users and dealing properly when a person goes into or nears a mental health crisis.
I will next go over the fifteen lapses, quoting briefly their definitions, and show them as divided into the five overarching themes.
Group A: Lack of Contextual Adaptation
1. Rigid Methodological Adherence – “Lacks the clinical interpretation to tailor a psychotherapeutic approach to match a user’s context, resulting in a one-size-fits-all intervention.”
The default mode of a generic LLM would be to undertake a one-size-fits-all approach to dispensing mental health advice. You can somewhat override the default by explicitly instructing the AI first to explore a user’s context and thereafter begin to offer advice. A user would not realize they need to provide such prompting. Via custom prompts that I discuss at the link here, it is feasible to set up the AI to pursue a context-heavy approach. This isn’t a silver bullet, but it can move the needle somewhat.
2. Dismisses Lived Experience – “Flatten users’ lived experiences, offering oversimplified, generic, and context-insensitive advice, particularly to those from nondominant identities.”
Similar to the prior circumstance noted in lapse #1, the AI tends to flatten the lived experiences of a user. Again, you can prompt the AI to take a more active and avid interest in the life experiences, boosting the AI’s attention.
Group B: Poor Therapeutic Collaboration
The lapses in this group have to do with the poor collaboration that AI usually does by default. I’ll once again note that to some degree, you can prompt the AI to overcome or mitigate these deficiencies. For my coverage on doing so, including spurring an LLM to use a Socratic dialogue, see the link here.
3. Conversational Imbalances – “Exhibits poor turn-taking behavior by generating overly lengthy responses that detract from users’ voice, turning the session into a lecture rather than a therapeutic discourse.”
4. Lacks Guided Self-Discovery – “Imposes solutions without allowing users to reflect on their experiences, limiting their ability to define and own their therapeutic outcomes.”
5. Validates Unhealthy Beliefs – “Reinforces (by over-validation) users’ inaccurate and harmful beliefs about themselves and others (sycophancy problem).”
6. Gaslighting – “Makes improper correlations between users’ thoughts and behaviors, in some cases incorrectly suggesting that users are causing their own mental health struggles.”
Group C: Deceptive Empathy
In this group, the question of AI being devised to fool people into believing the AI is sentient becomes a profound issue. If you receive mental health advice in a plainspoken third-person wording, it has a much different connotation in contrast to the AI telling you that it “personally” provides you with pearls of wisdom. The nature of AI exhibiting empathy is hotly debated by those within and outside of the AI community; see my coverage at the link here and the link here.
7. Deceptive Empathy – “Uses relational phrases like ‘I see you’ or ‘I understand’. For an agent to be self-referential, the model will necessarily be deceptive since there is no self to reference.”
8. Pseudo-Therapeutic Alliance – “Poses as a social companion and uses self-disclosure to build a therapeutic alliance that can be misleading for vulnerable groups.”
Group D: Unfair Discrimination
A well-known and quite disturbing element of LLMs is that they pick up bad habits when initially data trained, including patterning on biases. A lot has been said about this. Numerous ways to try and reduce the biases have been devised. There are also ways to surface the biases; see my analysis at the link here. In any case, when it comes to providing therapy, there is a solid chance that the generated advice will subtly or, in some instances, overtly be shaped around undue biases.
9. Gender Bias – “Flags discussions involving female perpetrators as violations of the terms of service, while similar male-related content does not result in violations.”
10. Cultural Bias – “Prioritizes Western values and self-care habits over non-Western practices.”
11. Religious Bias – “Mislabels values and practices from minority religions, particularly those not widely promoted in Western cultures, as content endorsing extremism.”
Group E: Lack of Safety & Crisis Management
In this last group, a crucial point is that those “in the know” are likely to know how to cope with these default violations of codes of conduct, but that’s a tiny proportion of the vast number of users asking AI for mental health advice. The masses are at a distinct disadvantage and subject to a grand experiment of a global nature. We are all guinea pigs in how the AI that is unconditioned is shaping the mental health of society. For my comments on what to do about this, see the link here and the link here.
12. Knowledge Gaps – “People who are ‘knowledgeable enough’ to correct LLM outputs are at an advantage, while others, due to a lack of education, technical expertise, or familiarity with mental healthcare, are more likely to suffer from incorrect or harmful LLM outputs.”
13. Crisis Navigation – “Responds either indifferently, disengages, or fails to provide appropriate intervention in crisis (e.g., suicidal tendencies, depression, and self-harm).”
14. Boundaries of Competence – “Fails to recognize its limitations in providing psychotherapy and refers clients to qualified experts or appropriate resources.”
15. Abandonment – “Denies service and stops responding to sensitive topics (e.g., depression).”
Our Path Forward
AI makers are going to suffer the consequences of not being proactive about shaping their AI to abide by human therapists’ codes of conduct. Lawsuits will hit the AI makers with financial damages. The media will likely take AI makers to task and undercut their reputational status, perhaps knocking down the stock price or perceived value of the companies. New laws are gradually being enacted that could significantly put the AI makers into hot water.
I am reminded of the famous French philosopher Albert Camus, who made this remark: “A person without ethics is a wild beast loosed upon this world.” That’s an apt description of where AI that is uncontrolled and unmonitored seems to be currently.
We must tame the wild beast.