Digital Risks, AI Slop, and the Crisis in Third Sector Data Governance

The Risks We See

Running platforms dedicated to LGBTQ+ and neurodivergent communities, the risks I see have moved far beyond standard IT issues into nuanced psychological and infrastructural harms.

First, we are battling an influx of AI-generated "slop." These are off-the-shelf AI resources masquerading as genuine support silos, but they completely lack any safeguards. Alongside this, there is a deeply concerning trend of vulnerable people turning to general-purpose consumer AIs as a primary lifeline. We are increasingly seeing users arrive on our platforms making comments such as:

"I spoke to ChatGPT about this."

Or, heartbreakingly:

"It's nice to actually talk to a human, ChatGPT must be getting bored of me."

The major danger here is that standard generative portals are fundamentally unsuitable for the complex, in-context reasoning required to support someone in crisis. When a generic AI passes unverified information to a vulnerable user verbatim, presenting it as absolute fact, or mimics empathy without any clinical safeguards, it becomes actively dangerous.

Beyond AI, there is a massive hidden risk in the sector's reliance on off-the-shelf tech platforms. Organisations are deploying "convenient" solutions without any real understanding of how the underlying environments, databases, or third-party connections actually operate. We regularly see platforms with fancy user interfaces that hide incredibly basic database architectures with absolutely zero granular access control. We also see reckless "one-click" integrations, where a platform requests sweeping access to a user's entire Google account just to send an email.

The solution to this isn't necessarily abandoning these tools, but demanding radical transparency and platform education. When organisations use these services, the platform must educate them upfront: How is this database operating? What exact connections are we making? This critical information cannot be buried deep in unreadable terms and conditions or hidden behind vague data-scope access windows.

How We Tackle These Risks

We tackle these risks by fundamentally altering how AI interacts with our users through a methodology I've coined Human Centred Model Augmentation. Instead of forcing neurodivergent or LGBTQ+ users to adapt to an AI's corporate language, we reroute the user's existing language — including their tone, cadence, and slang — back into the AI to make the connection deeply accessible and resonant.

To ensure this is done responsibly, we employ three rigid layers of Human-in-the-Loop intervention:

Layer 1: "What you know." At the onboarding phase, we ask organisations to explicitly define what it is they do. This captures their specific context and prevents the AI from interpreting or guessing how to handle a sensitive request.

Layer 2: "What we know." All overarching information on our sites is written by humans, verified by psychology professionals, and anchored in our strict safeguarding pillars. The AI draws only from this vetted truth.

Layer 3: "What we do." While we use AI to assist organisations with onboarding, every single listing requires human verification. This active human layer extends across all messaging and platform moderation.

Behind the scenes, we secure this entire process by cascading our functions vertically. Every step of the AI's process encounters a strict pass-or-fail gate. Instead of letting the AI generate a full response in one go, we break the process in half. We run the first half and immediately intercept it with a basic, non-AI algorithmic check — such as verifying an organisation's ID against our actual database records. Only when we mathematically prove the response contains factual database records and zero hallucinations do we allow the processing to continue.

This not only guarantees safety but drastically reduces our AI token usage, allowing us to maintain a continuously updated, highly accurate, and low-cost knowledge bank.

Alongside this, I take absolute personal responsibility for data privacy. We spin up custom, in-house analytics to prevent cross-site tracking from giants like Google and Facebook. I also maintain an uncompromising stance against the external monetisation of our support services. Grassroots fundraising and in-platform community offerings are entirely necessary, but I strictly oppose external monetisation through targeted ads, data-sharing agreements, and the commercial profiling of vulnerable users.

How Digital Risks Are Changing

My frontline experience has exposed a severe crisis in how the sector handles data and security, largely driven by cost and a massive knowledge gap. Organisations are blindly handing over sensitive beneficiary data because the realities of how that data is used are buried deep in corporate privacy policies.

A prime example is the inadvertent deployment of seemingly basic tools like Google Analytics. Charities often plug these in as standard practice, not realising that off-the-shelf configurations give advertisers and tech giants backdoor access to incredibly sensitive support ecosystems. These tools are rarely strictly necessary, and if they must be used, they should be absolutely scaled down to limit data capture. In their default state, they are actively harmful. They track a vulnerable person long after they leave a safe space, ensuring that their future journey on the web is influenced or targeted based on a moment of crisis.

Furthermore, standard security is becoming exclusionary. Cyber Essentials, for example, is a fantastic framework and an absolute must. It brilliantly standardises critical baselines — covering how we use and access our devices, physical infrastructure security, and digital safeguards. However, it isn't understood well enough for grassroots organisations to even know to ask for it. And when they do, a single-platform audit, excluding necessary infrastructure and monitoring, can easily exceed £10,000. This simply prices out the grassroots groups doing the most vital work.

What Must Change

First, the sector must abandon the naive reliance on off-the-shelf platforms and AI, and instead embrace mandatory Human-in-the-Loop guardrails designed specifically around the language of need, harm, and connection. Support organisations must also stop outsourcing their data governance to opaque third-party privacy policies and take direct ownership of their data flow.

Second, we need a radical shift in corporate accountability regarding how big tech interacts with the third sector. To give you an idea of scale: our platform is currently signposted as a primary support mechanism on the world's largest dating app. That is a half-billion-dollar business reaching over one million active monthly users in the UK alone.

There is a statement that needs to be made here. When massive corporate entities use grassroots services as a safeguarding mechanism to tick boxes for their own corporate governance and Online Safety Act compliance, they need to pay their share. The sector needs to confidently say to these businesses: if you are utilising our specialised, heavily moderated platforms to keep your users safe and meet your legal metrics, you must financially support the grassroots infrastructure that makes it possible.