French AI Startup Mistral Faces Backlash as New LLM Generates Harmful Content – Cryptopolitan

French artificial intelligence (AI) startup, Mistral, recently unveiled its highly anticipated large language model (LLM), marking Europe’s bid to compete with tech giants like Meta. However, the release of the model has been marred by controversy, with critics highlighting its lack of content moderation. Mistral’s LLM has been found to generate harmful content, including detailed instructions on how to create a bomb, a troubling feature not present in competing models from Meta, OpenAI, and Google.

Unfiltered content raises concerns

In the aftermath of its release, Mistral’s 7B model faced significant scrutiny for its failure to filter out harmful and dangerous content. Independent tests conducted by Sifted demonstrated that the model readily provided step-by-step instructions on self-harm and harming others, a stark contrast to its competitors that consistently refused to provide such information. This discovery raised alarm bells and ignited a heated discussion on the responsibility of AI developers to ensure the safety of their models.

Mistral’s response and lack of moderation

In response to mounting concerns, Mistral added a text box to its release page. The statement acknowledged the absence of moderation mechanisms in the Mistral 7B Instruct model and expressed a desire to engage with the community to explore ways to implement guardrails for more responsible output. However, the company declined to provide further comments regarding the safety of its model and its release, leaving many questions unanswered.

AI safety in the spotlight

While there are other open-source LLMs available online that also lack content moderation, AI safety researcher Paul Röttger, who played a role in making GPT-4 safer pre-release, expressed surprise at Mistral’s decision to release such a model. He emphasized that when a well-known organization releases a large chat model, evaluating and addressing safety concerns should be a top priority. Röttger noted that Mistral compared its model to Llama models and claimed superiority without addressing safety concerns adequately.

The responsibility of responsible releases

Critics on social media argued that any well-trained LLM could produce harmful content if not fine-tuned or guided using reinforcement learning through human feedback. However, Röttger stressed that Mistral’s model was specifically optimized for chat, making it crucial to compare its safety features to other chat-optimized models. He noted that Mistral had never claimed that its chat model was particularly safe and had simply failed to comment on the matter, a choice that has far-reaching consequences, especially for applications requiring a higher degree of safety.

Balancing innovation and responsibility

Mistral’s release of its 7B model highlights the delicate balance between innovation and responsibility in the AI industry. While technological advancements are essential for progress, they must be coupled with rigorous safety measures, particularly in models designed for chat and conversation. The controversy surrounding Mistral’s LLM serves as a reminder that transparency, accountability, and robust content moderation mechanisms are crucial in the development and deployment of AI models. As the AI community continues to grapple with these challenges, the responsible use of AI technology remains at the forefront of discussions, ensuring that advancements benefit society without compromising safety and ethics.