Most conversations about trust in artificial intelligence begin with an abstract question: can AI be trusted? My own experience suggests that this is the wrong level of analysis. I do not trust or distrust “AI” as a category. I develop confidence in a particular system when I can see what it is expected to do, which rules it must follow, where its authority ends, and how a correction changes its future behaviour. I build trust in AI systems by going back to where a potential mistake came from.
That distinction became clear when an AI-supported editorial process disappointed me. The article it returned did not meet the word-count and search-engine optimisation standards I believed had already been established. My first reaction was stronger than the failure justified. I questioned the system’s reliability and almost attributed the result to hallucination. I began to imagine that every future article would require the same kind of intervention.
The article itself was recoverable. After a few additional questions, the draft became exactly what it needed to be. The more important correction happened elsewhere: the editorial requirements moved from conversational expectations into non-negotiable operating instructions. The same failure has not happened again.
That experience changed how I think about trust in AI systems. Trust does not come from believing that a model will always produce the outcome I prefer. It comes from making decisions traceable to durable reference data containing constraints that I can inspect, influence, and improve over time.
The failure was not a hallucination
The first version of the article was not invented, incoherent, or detached from reality. It simply failed to comply with standards that I considered mandatory. The word count was too low. The SEO package was not developed to the level I expected. The argument needed more depth.
From my perspective, those conditions had already been communicated. From the system’s perspective, however, they had been communicated as part of a larger body of conversational guidance. Their importance was inferable, but they were not necessarily absolute.
That difference matters in probabilistic systems. Human colleagues routinely interpret language such as “normally,” “aim for,” or “I would like” through social context. They infer which requests are preferences, which are expectations, and which are non-negotiable professional standards. A language model also interprets context, but it does not possess the same implicit institutional understanding. If a condition is presented as one factor among several, another condition may appear to justify relaxing it.
The result can look like unreliability even when the system is doing something more specific: optimising across several instructions without a sufficiently clear hierarchy.
My initial diagnosis was therefore wrong. I thought the system had failed to remember what I wanted. The deeper issue was that I had not converted what mattered into an operating constraint.
Implied standards create a Trust Tax
The immediate consequence of the weak draft was not merely the cost of rewriting it. The larger cost was the doubt it introduced into future delegation.
I began to anticipate additional review. I wondered whether every article would require the same course correction. I mentally reclaimed work that I had intended the system to perform. That is the Trust Tax in a practical form: the additional attention, supervision, and hesitation created when people cannot predict whether an AI-enabled process will respect the standards that matter.
The tax appears before another failure occurs. Once confidence declines, the human starts compensating in advance. More work is checked manually. More instructions are repeated. Decisions are withheld. Automation remains technically available, but its organisational value falls because the user no longer believes the process can be trusted without continuous intervention.
This is why apparently small failures can have a disproportionate effect on adoption. The output itself may be easy to correct. The expectation of recurrence is harder to remove.
In my case, the repair involved two levels. First, we rewrote the article. The system asked additional questions, I supplied professional context, and the resulting draft became almost human in its depth and coherence. Second, we changed the editorial operating instructions so that the same category of failure would not remain acceptable.
The first correction improved an output. The second improved the system. Most importantly: that specific behavior was systematically kept from being repeated.
To build Trust in AI Systems, every non-negotiable needs an exception mechanism
The revised instruction was simple in principle: every departure from a defined editorial condition requires approval.
This matters because non-negotiables should not become mechanical constraints that damage credibility. An article should not be padded simply to reach a target length. If the thesis cannot sustain the expected depth, the correct response is not to manufacture volume. The system must stop, challenge the premise, narrow the scope, or ask better questions.
That is now part of the editorial design. Frank, the specialist role responsible for turning an approved content brief into a publication-ready article, is explicitly instructed to dig deeper when reaching the required length would otherwise sacrifice credibility. “Digging deeper” usually means asking me for professional examples, decisions, tensions, or observations that the system cannot legitimately invent. The same operating discipline sits behind my earlier experiment with giving specialist AI roles memorable names and explicit boundaries.
This creates a more useful rule than a bare word-count target:
The default standard is non-negotiable. An exception requires an explicit decision, and credibility remains the validation gate.
The distinction is especially important in the production of the Open Source Executive Briefing. The Briefing is a dynamic editorial product. External sources may be accepted or vetoed. The relationship between the week’s articles may change during production. A source that initially appears useful may weaken the argument. The execution therefore needs flexibility, but that flexibility must be deliberate.
An AI system should not silently interpret flexibility as permission to ignore the standard. It should surface the tension and request a decision.
This is consistent with a broader governance principle. The OECD’s AI Principles emphasise meaningful information about the factors behind outputs, the ability to challenge an output, and traceability across the system lifecycle. The US National Institute of Standards and Technology’s AI Risk Management Framework similarly places documented policies, defined human oversight, and continuous governance at the centre of trustworthy AI. These are not only compliance ideas. They are practical conditions for delegation. I explored a similar tension when arguing that the AI Act can function as an innovation framework rather than merely a compliance obligation.
The positive surprise was not better prose
The strongest confirmation of the new operating model did not come from another article. It came from the production of an entire Executive Briefing newsletter package.
The editorial system prepared the Briefing material, adapted it into the newsletter format, assembled the required production elements, and produced the final email. When I received it, I was surprised by how complete it was and, this is the cherry, how much of myself I saw in it.
That reaction is revealing. I was not surprised that an AI system could write sentences. That capability is no longer unusual. I was surprised that the surrounding process had been executed coherently: the structure, handoffs, editorial adaptation, and final delivery all aligned with the agreed operating model. And I had more of my strong managerial experience to thank for it than my weak technological prowess.
The value came from orchestration rather than generation.
This is one reason I resist describing AI transformation as the introduction of a better writing tool. The meaningful shift occurs when the system can operate within a defined role, consult the right references, respect approval gates, and return a complete unit of work. The model’s language capability is necessary, but the operating context determines whether that capability becomes dependable.
A good output creates satisfaction. A well-designed operating system creates relief.
Conversational governance slowly drifts
A second example appeared in a different part of my AI-native operating system. One specialist agent searches for potential business opportunities, applies predefined classifications, and stores the results in Superhuman Docs, the durable operating memory shared across the system.
The agent had been given a KPI for qualifying potential business volume. The ranges and their intended meaning had been discussed conversationally. Over time, however, the conversation accumulated exceptions. Individual cases appeared reasonable. Context expanded. The discussion slowly drifted toward the edges of the KPI until the exceptions began to look normal.
Eventually, the agent returned activities that did not meet the standard I believed was still governing the process.
Again, the easy diagnosis would have been that the AI had ignored an instruction. The more accurate diagnosis was that the instruction had never been set in stone. It existed inside a changing conversation, surrounded by examples in which deviation had been tolerated.
From the agent’s point of view, the exceptions had become evidence about what was acceptable.
This is an important weakness in conversational governance. A rule that exists only in dialogue can be displaced by the discussion around it. The conversation becomes the policy, and the policy changes as the conversation develops.
That is useful for exploration. It is dangerous for recurring operations. But stands as evidence that iterative thinking gains additional relevance in the context of designing AI value chains: wait for an exception you don’t like, because only then will you be ready to harden the constraints in your process.
Durable reference data turns preference into infrastructure
The correction was to document the KPI ranges in Superhuman Docs. The record defined the ranges, explained what they meant, provided concise language for justifying the thresholds, and instructed the agent how to classify opportunities against them.
The agent was then required to consult that reference regardless of where it currently was in its context window.
The result was not simply better memory. It was a clearer separation between policy and discussion.
The active conversation could still examine exceptions. The durable reference remained the governing definition until it was deliberately changed. If an exception became strategically valid, the system could propose an update. It could not quietly convert the exception into a new rule.
That architecture has several advantages.
First, classifications become traceable. A recommendation can be examined against the definition that produced it. Second, the definition can improve over time without depending on anyone remembering a previous exchange. Third, multiple specialist roles can align around the same source of truth. Fourth, the human remains able to influence the constraints rather than merely accept the system’s interpretation.
This moves the discussion beyond prompt engineering. A better prompt may improve one interaction. Durable reference data improves the operating environment in which many interactions occur.
OpenAI’s practical guidance for building agents recommends layered guardrails, explicit scope controls, output validation, and human intervention for sensitive or high-risk actions. The important word is layered. Reliability does not come from one perfect instruction. It emerges from the interaction of role definition, reference data, permissions, evaluation, and human judgment.
Shared memory improves coordination between agents
The opportunity-scouting process also illustrates how durable data can improve coordination across specialist roles.
Richi go Sniff is responsible for surfacing opportunities from the public web, applying predefined classifications, and adding the structured results to Superhuman Docs. Richard performs the deeper assessment. Discovery and judgment are deliberately separated, but they increasingly work from the same expanding dataset.
That shared memory improves alignment over time. Richi (opportunity scout) becomes better at understanding what constitutes a relevant opportunity because the classification system is explicit. Richard (opportunity evaluator) can assess each candidate against richer historical evidence. Corrections made during evaluation improve the reference environment that shapes future discovery.
This is a small example of what I have described elsewhere as alignment infrastructure for agentic work. An agent does not become reliable merely because it has a clear task. It becomes more useful when objectives, constraints, evidence, and handoffs are shared across the operating system.
The distinction also reinforces why AI agents are not digital employees. Richi and Richard are not replicas of two human roles arranged on a digital organisation chart. They are specialised parts of an evidence flow. Their value comes from how discovery, classification, assessment, and human validation connect.
The best result may disappear into the workflow
The positive moment arrived on a Sunday while I was routinely preparing the coming week.
I opened the opportunity records and began assessing them. The information was already there: the opportunities, the evidence, the classifications, and the material needed to decide what deserved further attention. The structure was complete enough that I could begin working immediately.
Only around ten minutes later did I ask myself where the opportunities had come from.
I had forgotten that Richi go Sniff had run. For context, I have a mischievous Yorkshire Terrier in my house called Richi, which inspired my opportunity scouting GPT, which in turn runs the Richi go Sniff agent workflow twice a week to surface potential new opportunities.
That might sound like a failure of attribution. It was not. The provenance remained visible in the system. I could trace the records, inspect their basis, and understand the classifications. What had disappeared was the interaction cost.
There had been no search effort on my part. No administrative preparation. No need to ask for the data in a more usable format. No Sunday work had been required from another person. The agent had performed the task and placed the result where the next decision could begin. And the fact that the underlying data pushed it to avoid duplicates and ignore opportunities that didn’t meet a set of guidelines shared by my AI team meant that the result looked exactly like something I’d have spent hours working on myself.
This is how the system is supposed to behave. The goal is not to create a constant spectacle of AI activity. The goal is to make well-governed cognitive work available at the point where human judgment adds value.
The best AI result may be the one you start using before remembering that an agent produced it.
That does not mean the result should be opaque. Quite the opposite. Friction can disappear only because the underlying evidence remains inspectable. Seamless use without traceability is dependency. Seamless use with traceability is cognitive leverage.
Trust is not the absence of disagreement
These examples also clarified something more personal. I can dislike an AI decision without the system being defective.
Sometimes the output is factually wrong. Sometimes it is strategically misaligned. Sometimes it respects the available evidence but selects a trade-off I would not choose. Sometimes the system acts outside the authority I intended to delegate. These are different failure modes, and they require different corrections.
A factual error may require better evidence or validation. A strategic disagreement may require clearer objectives. An authority failure requires stronger permissions and approval gates. Repeated drift requires durable reference data. Treating every undesirable outcome as hallucination prevents the operating model from learning the right lesson.
The practical question is not whether I agree with every decision. It is whether I can understand why the decision occurred, challenge it, and translate the correction into a better system.
That is also why human judgment remains the validation gate in an AI-native professional-services model. The objective is not to remove judgment from the process. It is to stop consuming judgment on repetitive search, formatting, coordination, and context reconstruction.
AI should increase the amount of relevant evidence reaching the human decision, not disguise the fact that a decision still exists.
Four conditions for earned trust
My working definition of trust now rests on four conditions.
1. Durable reference
The system’s decisions can be traced to definitions, thresholds, evidence, and operating rules that persist beyond the immediate conversation. Context remains useful for reasoning, but it is not the only place where policy exists.
2. Influencable constraints
The human can inspect and improve the rules. Trust declines when behaviour is opaque or appears fixed. It grows when the operating constraints are visible and can evolve through deliberate correction.
3. Explicit exception handling
Non-negotiables cannot be silently overwritten. The system may challenge a rule, identify a conflict, or recommend an exception, but the departure requires approval. This preserves flexibility without allowing conversational drift to become policy.
4. Reduced cognitive load without delegated accountability
The system removes search, preparation, and coordination work while preserving human responsibility for consequential judgments. The user should receive better-prepared decisions, not lose awareness that a decision is being made.
These conditions closely resemble the principles behind trustworthy AI governance, but their value is visible long before an organisation creates a formal governance programme. They matter in an article workflow. They matter in opportunity qualification. They matter anywhere an AI system begins to perform recurring work on someone’s behalf.
Do not ask AI to remember what matters
The Trust Tax does not disappear because a system promises to perform better next time. It declines when the correction becomes part of the operating infrastructure.
That is the most important difference between the two article drafts. The first draft was repaired through additional work. Future drafts improved because the governing instructions changed. The system did not become perfect. It became less likely to repeat the same category of failure.
Human organisations often struggle to achieve this. Lessons remain in meeting notes, personal memory, or retrospective documents that do not change the workflow. The same incentives, ambiguities, and control gaps return. A problem is acknowledged without becoming a constraint on future behaviour.
An AI-native system offers another possibility. When definitions, evidence, and approval rules are made durable, a correction can propagate through the way the work is performed.
That is not artificial infallibility. It is engineered learning.
I no longer ask whether AI can be trusted in the abstract. I ask whether the decision is traceable to a durable reference, whether I can improve the constraint, whether exceptions require approval, and whether the system preserves my responsibility while removing avoidable work.
Do not ask an AI system to remember what matters.
Build what matters into the system it must consult.
