Most of us are busy trying to keep up with AI; there are always new tools, features, and updates that appear all the time, and we are still learning where AI can be useful in our work and where we need to be more careful.
While the technology keeps changing, one responsibility remains the same: we need to take care of the information people trust us with.
It is also unrealistic to expect every person using AI to read and understand every page of every provider's privacy policy, terms, and technical documentation before using it. What we can do is control the part that is in our hands: notice what we are sharing, and avoid sharing personal information when the task does not need it.
What can happen when personal information gets into the wrong hands?
Not every disclosure of personal information will cause harm, and sharing information with an AI service does not automatically mean that the information will become public or be used for fraud. But personal information has value because it tells somebody something about a real person. Once it is exposed or obtained by the wrong people, it can be used in ways that are difficult to control.
- Targeted scams and impersonation. Scam messages become much more convincing when the person sending them knows something about you. A message that knows your name, employer, role, colleague or recent activity is easier to mistake for something genuine. Europol warns that criminals increasingly use personal information alongside AI to create more tailored social engineering and scam messages. AI can help criminals produce these messages at scale and make them fit a person's context more closely.
- Stolen information can be reused and traded. Personal data does not necessarily stay with the person who first obtains it. Europol's 2025 cybercrime assessment describes an active criminal market in which stolen credentials and datasets are sold, repackaged and resold. The information can then be used for fraud, extortion, account access and other forms of abuse.
- Information that looks harmless can become harmful when combined. A person's job title may not seem particularly sensitive. Neither might their location, age or employer. But several small pieces of information can be put together to build a much more detailed picture of somebody. Europol describes this as a common part of social engineering: criminals collect fragments from different places and combine them to make their approach more believable.
- Identity theft and financial fraud. Some information carries a higher risk because it can help somebody prove or assume another person's identity. Identity documents, dates of birth, addresses, account information and similar details can contribute to identity fraud. The ICO warns that identity theft can lead to financial loss and difficulties obtaining credit or financial services. It also recognises identity theft, fraud, financial loss, reputational damage and loss of confidentiality among the possible consequences when personal information is compromised.
There are legal responsibilities too
For organisations covered by data protection laws such as the GDPR, using AI does not remove the responsibilities that already exist around personal data.
One of the basic principles is data minimisation. Personal information should be adequate and relevant for the purpose, and limited to what is necessary. Put simply, if an organisation can achieve the same purpose using less personal information, it should not process more simply because that information happens to be available.
This matters when using AI. If the purpose is to summarise a participant's feedback, including their passport number cannot be justified simply because the passport number happened to be in the document that was copied into the tool.
The responsibility for complying with data protection requirements ultimately sits with the organisation when it is the data controller. Organisations need appropriate policies, technical safeguards, guidance, training and clear responsibilities around how AI and personal information are used. The ICO specifically identifies staff awareness, training and operational guidance as part of good AI governance and data protection practice.
The consequences of getting this wrong are not only theoretical. Depending on the circumstances, organisations may have reporting obligations when personal information is breached, and serious failures can lead to regulatory action. More importantly, the impact falls on the people whose information was involved. The ICO lists possible consequences including financial loss, discrimination, identity theft, damage to reputation, loss of confidentiality and, in serious situations, physical harm.
Using AI responsibly is everyone's job
Organisations have to do their part. They need to choose appropriate tools, understand how those tools are being used, provide clear policies and make it possible for people to follow them.
But policies and safeguards cannot make every decision for us.
The person typing the prompt is often the person who can see that a participant's name is included. The person uploading the spreadsheet can see that there are five columns of personal information that are not needed for the analysis. The person pasting an email can notice that the sender's contact details have come with it.
That is where individual responsibility comes in.
It does not mean that every employee needs to become a privacy lawyer or AI expert. It means applying a basic level of care to the information in front of us.
Before sharing personal information with AI, we can ask whether it is needed. We can remove names when identity does not matter. We can redact identification numbers. We can share the relevant part of a document instead of the entire file. We can use the tools approved by our organisation and ask when we are unsure.
Organisations also have a responsibility to help people develop this awareness. In the EU, the AI Act now requires organisations that provide or deploy AI systems to take measures to support AI literacy among the people using those systems on their behalf.
Tools such as Guardrails are one part of this. They can help us spot personal information at the point where we still have a chance to remove or replace it. But the broader responsibility belongs to all of us.