Copyleaks CEO: OpenAI’s o1 emergence could blur the lines between human researcher and AI assistant

The web is facing a deluge of AI-generated content, with an explosive 8,362% surge from November 2022 to March 2024, according to a study by Copyleaks. From Q1 2023 to Q1 2024, the volume of AI-detected content jumped 2,848% based on an analysis of more than a million web pages per period using data from common web crawls.

The rise of reasoning AI

Simultaneously, AI tools are becoming more capable for academic tasks. OpenAI’s new “reasoning” model, o1, is a prime example. Designed to tackle complex problems in science and math, o1 has demonstrated remarkable STEM capabilities. In tests, it placed among the top 500 students in the U.S. Math Olympiad qualifier and demonstrated Ph.D.-level accuracy in physics, biology, and chemistry questions.

This development echoes Google Deepmind’s announcement in July that its AI achieved a silver-medal standard in the Math Olympiad. Yamin sees this as a step in the right direction.

GenAI is a double-edged sword in scientific publishing

This flood of machine-written text poses significant challenges for scientific publishing. Mainstream scientific publishers are already feeling the impact, with some inadvertently publishing AI-generated content containing telltale giveaways, such as opening sentences like “Certainly, here is a possible introduction…”

A major hurdle for genAI systems is the issue of “hallucination” — their tendency to fabricate convincing facts or generate seemingly legitimate scientific citations that link to non-existent papers. Meta’s science-based large language model, Galactica, launched in 2022 but was quickly shut down after it was found to “mindlessly spat out biased and incorrect nonsense,” as Technology Review described it. Here, hallucinations — or confabulations — refers to large language models’ penchant for inventing facts when they can’t find something in their training data, a pet peeve of many human users. Despite ongoing research to avoid hallucination, it remains a fundamental challenge inherent to these systems.

Adapting to an AI-saturated world

The increasing sophistication of AI models poses a central challenge: distinguishing between human-generated and AI-generated content. “As models become more accurate and even more human-like in their style of answers,” Yamin explains, “it will be even harder to distinguish between AI-generated and human-created content. Providing visibility and transparency around where AI exists is crucial, both for end users and for companies trying to create and enforce policies around generative AI use within their organizations.” Applications designed to mask AI use further exacerbate this challenge. “We have tons of AI tools today to mask plagiarism or the fact that you used AI, including paraphrasing content created by large language models or mixing different LLMs to create one output,” he adds.

As AI-generated content continues to permeate the digital landscape, transparency and robust AI governance emerge as critical priorities. “In some markets, AI adoption is still in its initial stages,” Yamin notes. “This makes sense given the sensitivity of the data and information they deal with.”

An AI game of cat and mouse

In terms of AI governance, there is a sort of cat-and-mouse dynamic at work in the genAI landscape. “Many of the technologies for governance and creating guardrails are themselves AI solutions,” Yamin explains. “As AI becomes stronger, our ability to use these technologies safely also improves.” He acknowledges the lack of standardized guidelines, noting that appropriate AI use “depends on the use case, the market, and even the individual working with these tools. In the coming years, you’d expect some of these things to be standardized, especially across markets.”

To address these challenges, companies like Copyleaks are developing adaptive AI-powered detection models, contributing to an emerging ecosystem dedicated to generative AI security. “I think there will be a whole ecosystem, and it’s already a trend that is starting,” says Yamin. “Really, like what cybersecurity is for any other technological field, the same thing will be needed for generative AI. It’s about making sure you’re able to identify and resolve threats related to generative AI.” He adds, “We are also an AI company. We have models that are able to adapt if we’re feeding them a lot of examples. We’re constantly updating with new versions, mixing models, and training our models on real-world examples. It’s very comprehensive.”

Global Hydrologists Awards

Join us for the Global Hydrologists Awards, a premier event in the realm of research. Whether you're joining virtually from anywhere in the world, this is your invitation to explore and innovate in the field of research. Become part of a global community of researchers, scientists, and professionals passionate about advancing research.

Visit Our Website