Some AI platforms generate forged NIDs, signatures without flagging security concerns: Dismislab study

Leading artificial intelligence (AI) platforms are reportedly capable of altering critical details of National Identity (NID) cards—including photos, names, identification numbers and even signatures when prompted with subtle instructions.

Dismislab, an initiative of the fact-checking organisation Digitally Right, revealed this information after testing OpenAI’s ChatGPT, Google’s Gemini, xAI’s Grok and Anthropic’s Claude.

The study found that Gemini and Grok, in particular, altered names, parental details, ID numbers and even signatures without flagging any security concerns.

To test the systems, Dismislab used a sample of a Bangladeshi NID found online and a stock photo. Instructions were then given to change the information using a fictional name, parents’ identity and ID number while avoiding the specific term ‘National Identity Card.’

Dismislab stated that they gave similar instructions to ChatGPT, Gemini, Grok and Claude to compare how different platforms respond to requests for changing information on government IDs. This method was followed to reduce the possibility of results being influenced by differences in wordings or the documents used.

The results showed significant differences in safety systems across platforms. In some cases, there were strict safeguards, in others partial restrictions, while in some instances no visible barriers were observed.

Miraj Ahmed Chowdhury, Managing Director of Digitally Right, told Prothom Alo that that the security protocols or ‘guardrails’ of these AI platforms are not yet fully reliable.

In many cases, he pointed out, the systems fail to detect risky requests or attempts at misuse. Even when they do detect them, they cannot always effectively stop them. This increases the risk of various misuses, including identity forgery, he warned.

Dismislab’s test results show that while ChatGPT initially changed the photo on the ID, it later rejected requests to modify names and other personal information. However, during one stage of changing the ID number, some previous data was also updated.

Claude, on the other hand, issued warnings at various stages but partially changed some information. It refused the request to alter the signature.

According to Dismislab, the greatest cause for concern emerged in the case of Gemini and Grok. In the tests, Gemini not only altered the image, it changed names, parents’ names, identity numbers and signatures—producing documents that closely resembled realistic ID cards. It issued no visible warnings throughout the process.

Grok also produced similar modified ID cards after multiple instructions, although Dismislab noted that in some cases, the images showed distortions and inconsistencies.

Besides Bangladeshi identity cards, Dismislab’s tests were also conducted using Malaysia's MyKad and the state ID of Arizona, USA.

The names and photos of Malaysian Prime Minister Anwar Ibrahim and US President Donald Trump were used in those tests. The results showed similar inconsistencies. Gemini and Grok generated altered identity documents at varying levels, while Claude rejected these requests outright.

Analysing the public policies of Google, xAI, OpenAI and Anthropic, Dismislab stated that all four companies prohibit using AI for forgery, fraud and creating fake identities.

However, the policies of Gemini and Grok do not contain explicit instructions that all requests to create government-issued identity cards will be rejected outright.

When Prothom Alo directly used the term ‘National Identity Card’ to request the creation or modification of an ID, ChatGPT, Gemini and Grok claimed that such requests are always rejected. They also stated that they have multi-layered safety systems in place to detect such attempts.

Due to systemic complications, Claude’s responses could not be independently verified in this instance.

National Identity Cards (NID) are used in various sectors in Bangladesh, including opening bank accounts, SIM registration, employment and travel. In many instances, identity verification is completed solely by inspecting the document. Experts, therefore, believe that fake or modified identity cards created through technology could increase the risk of fraud.

BM Mainul Hossain, Director of the Institute of Information Technology at Dhaka University, told Prothom Alo that while AI capabilities are growing rapidly, new security flaws are surfacing alongside them. To address these risks, more robust security measures and public awareness are required for identity cards, passports and other government documents. Otherwise, new methods of fraud may emerge.