How to Build Ethical AI - We Did It by Design (And Discovered Something Bigger)
Working with AI at the Danish Business Authority (DBA), we realized from day one that it needed to be done responsibly. When you’re on a mission to take people’s livelihood away by closing their business - and perhaps even having them sent to jail - you want to make sure that what you hit is what you are aiming for.
In 2017, deep learning was all the rage. Everybody was focused on models and how to gain fairness by having a data science team composed of different races and genders. This was wildly immature and completely missing the point on what ethics is - in some cases even proposing measures which contradicted the mission. When dealing with AI, you’re working with advanced statistics. In other words: math.
When LLMs hallucinate, we’re seeing the consequences of next-token prediction trained on statistical patterns, where high confidence in vector space doesn’t distinguish between ‘this sounds right’ and ’this is right. Even though this is very simplified, it still supports the point that machines don’t know what they are doing - so we need to reinforce their weaknesses or protect users from harm. In order to do that, we need to first find out what we are trying to achieve.
The Real Problem We Discovered
We realized quickly that “ethical AI” was asking the wrong question. The real question wasn’t “is this fair?” but “can we quantify what we’re trying to achieve and measure whether we’re achieving it?” Ethics wasn’t a constraint on our work - it was forcing us to build organizational capabilities that most businesses lack: the ability to know what their systems are doing and why.
This matters because ethical behavior isn’t possible if we haven’t defined a set of goals - and discussing ethics is meaningless if not in context of those goals. But here’s what we discovered: the same infrastructure that ensures ethical behavior also ensures that AI systems actually do what stakeholders think they do. That’s not just an ethical requirement. That’s competitive advantage.
From Ethics to Organizational Coherence
In our case, with the distinct goal of preventing fraud, it would be ethically reasonable to state that you shouldn’t accuse someone of committing fraud without hard evidence. This seemingly simple rule hides a plethora of complexity - but more importantly, it forces a question that most organizations never answer: what exactly constitutes success?
I chose the foundational approach that ethos is the ability for an organization to follow its guidelines. The mission of preventing fraud meant that a universal idea of what constitutes ethical behavior was too vague. We instead had to populate our approach with the guidelines we wished to adhere to: as an authority, that meant legislation, human rights, GDPR, good governance in public institutions in general, and the then-upcoming European AI Act (AIA) in particular.
But once you start formalizing goals in relation to AI usage, you’re not just doing ethics - you’re building organizational intelligence. You’re creating the capability to translate business intent into measurable properties, detect drift from those properties, and maintain continuous feedback loops between what you intend and what actually happens. Most organizations with AI systems can’t answer basic questions: What is our system actually optimizing for? How do we know when it’s working? What happens when context changes?
The Framework: X-RAI as Business Intelligence
The implications for measuring ethics fall in line with monitoring AI systems in general. As with model observability, ethics became quantifiable aspects that the AI platform and development and operation procedures had to encompass. We approached this through three dimensions: business requirements, ethics data, and component support.
Business Requirements: The Forcing Function
First, the ‘black box’ dilemma of the technology as well as legislation quickly established the need for a ‘human in the loop’ approach to all AI output affecting case handling. It simply isn’t legal to make automated decisions based on machines, so establishing ‘the burning platform’ was easy.
But the more important realization was this: AI output has inherent quality issues that don’t just affect ethics - they affect business value through lost opportunity. Cleaning up errors is costly, and slow reaction makes this even costlier as errors affect more of the business. Not hitting what you’re aiming for isn’t just poor ethics - it’s bad for business. The requirements were the same.
Ethics Data: Operationalizing Goals
Formalizing a statement of purpose in relation to all AI usage enables seeing AI output as quantifiable properties in relation to that goal. Ethics can have Key Performance Indicators (KPI), but if the business purpose is vague then the ability to measure KPI suffers.
When dealing with prediction, vagueness is not your friend. This is counter-intuitive - if we could predict precisely, we wouldn’t need AI prediction in the first place. However, you can neither predict precisely nor set vague goals if you are to measure them. The solution is to set business goals that are your best bet and then establish a plan for when and how to evaluate them.
There is no right goal, there is only the correct assumption that you will probably need to check up on any goal you set and reevaluate. The superpower of this approach is that it lets you measure whether you’re approaching or deviating from what you’re trying to accomplish. This isn’t ethics theater - it’s decision intelligence.
Component Support: Infrastructure for Decision Operations
The productionization of AI has a long chain of dependencies. Imagine that containerization, data pipelines, model training, interpretability tools, and the rest have already been taken care of. What we established beyond this standard MLOps infrastructure were components specifically designed to maintain continuous alignment between business intent and system behavior.
The X-RAI Framework We co-sponsored an industrial PhD fellow with the IT University of Copenhagen who worked embedded with the project. Per Rådberg Nagbøl’s formidable work on “Theoretical and practical approaches to the responsible management of artificial intelligence” (available here and well worth a read) produced the X-RAI framework - Transparent (X-ray), Explainable (X), Responsible (R), Accurate (A) Artificial Intelligence.
X-RAI consists of four questionnaire-based sub-frameworks that create something unusual: a system where business intent, technical implementation, and operational reality are continuously reconciled. The questionnaires aren’t bureaucratic overhead - they’re structured tools that prevent the normal drift between what stakeholders think a system does and what it actually does.
Artificial Intelligence Risk Assessment (AIRA): AIRA assesses pre-production risk from early development to production readiness. It advocates involvement of stakeholders from different backgrounds - at minimum, domain experts and data scientists. The questionnaire, following principles of “structured intuition,” forces consideration of performance metrics, consequences of correct and incorrect predictions, explainability, fairness, personal data, and data quality before code is written.
The critical innovation here: AIRA captures this as version-controlled YAML files stored in Git alongside model source code. Have you ever looked at documentation and wondered if it relates to the running code? This simple reuse of software development tools accentuated the need for business purpose to be available in the correct version for the code in production. Technical documentation synchronized with implementation state isn’t just good practice - it’s the foundation for knowing what your system actually does.
Evaluation Plan: Before an AI system goes into production, the Evaluation Plan guides managers, domain experts, and data scientists in agreeing on how, when, and by whom the AI system will be evaluated. By requiring answers to these questions before go-live, the framework ensures that evaluations are planned and required resources are available. This prevents the common failure mode where teams ship systems without knowing how they’ll determine if those systems are working.
Evaluation Support: This framework supports ongoing evaluation of AI systems in production, ensuring the system meets its quality criteria. It structures and documents the ongoing evaluation, including decision-making about whether the AI system should continue in production, be decommissioned, retrained, or redeveloped. Critically, it addresses two specific failure modes we identified:
First, false negative blindness: human oversight typically focuses on positive cases (businesses likely to commit fraud). Caseworkers rarely examine negative cases (low fraud scores), so chances of detecting a system drifting toward more false negatives are low.
Second, self-reinforcing bias loops: when caseworkers see AI predictions (say, 98% likelihood of fraud), they investigate those cases more thoroughly. If these biased investigations become ground truth for evaluation, the bias gets reinforced. Over time, humans come to believe the wrong relationships, making bias progressively harder to detect.
The solution: formal evaluations with deliberate sampling and blinded scores, ensuring comprehensive assessment across the prediction space.
Retraining/Redevelopment Execution: This guides domain experts, data scientists, and managers in planning retraining or redevelopment, including identifying dependencies to other AI systems and training data needs. When a system is slotted for change, AIRA and the Evaluation Plan must be revisited - the cyclical arrow in the framework diagram represents this continuous improvement loop.
The Component Architecture: Decision Infrastructure
Beyond X-RAI, we built specific infrastructure to operationalize these principles:
The Racetrack: Data scientists could deploy their models to production through this platform, with metrics endpoints exposing observability data. Almost all models leveraged the mature SOA architecture of DBA to be queried as services. More at Github - TheRacetrack
The Control Tower: Between model output and receiving systems, a control tower enabled domain specialists to mute or adjust thresholds for parameter effects. This wasn’t just instrumental to business acceptance - it provided stakeholders with genuine agency over AI behavior.
This capability is fascinating from a decision operations perspective. Something can be mathematically correct but contextually too potent. If you’ve ever eaten ice cream while drinking hot coffee, you’ve experienced that what is usually a safe temperature can suddenly make your nerves scream. The ability to set parameter impact enabled business to adjust the ‘experience’ to a comfortable level.
Naturally, all settings were logged and provided feedback to data scientists. If business keeps turning down the heat, it’s a signal that the model may be drifting. This isn’t just monitoring - it’s a feedback loop between operational reality and model expectations.
The Catwalk: Once in production, this purpose-built system handled individual model evaluation, allowing formal verification of whether parameters were producing correct outputs. This generated markup data and provided information on model or data drift. More importantly, it addressed the tedious but critical task of comparing actual model inputs with outputs to verify correctness.
The RecordKeeper: Underlying all data input, data handling, and output generation, the RecordKeeper kept records of all events. It deliberately solved an edge case problem that telemetry tooling normally handles poorly: events may occur years apart, yet need to be queried together. A model may produce output with long periods of inactivity, or you may need to reconstruct what happened years ago.
This is an edge case in normal IT landscapes because 99% of daily jobs are actual state or single pipes of activities. Reconstructing what happened in an enterprise with billions of moving parts is not. The RecordKeeper followed a Directed Acyclic Graph (DAG) design with formal causal properties - confounders represented explicitly in the graph structure.
Operational Requirements: The Unchangeable Production Environment
To build a machine that tells time accurately while also remembering all the times you reset or adjusted measurement is complicated. Especially when you’re dealing with multiple clocks, some bearing unusual design principles. AI is hard and technologies vary - sometimes even exhibiting non-deterministic behavior. You want to make sure that what you think took place actually took place.
Therefore, part of the design of the production chain rested on the principle that the production environment was unchangeable, with configuration assuring that source code was equal to production code. This wasn’t paranoia - it was recognizing that maintaining synchronized state across the technical, organizational, and operational layers requires deliberate architectural constraints.
The Hidden Competitive Advantage
Here’s what we discovered: organizations that can systematically translate goals into measurable properties, deploy systems that expose those properties, and maintain continuous feedback loops between intent and outcome have a fundamental advantage. This isn’t just about AI ethics - it’s about organizational intelligence.
At DBA, we called this capability “ethical AI” because regulatory requirements forced its development. But the same infrastructure that ensured we weren’t unfairly targeting businesses also meant we could:
- Rapidly adjust to new fraud patterns
- Quantify model drift before it caused business problems
- Maintain stakeholder trust through demonstrated control
- Answer questions about system behavior from years ago
- Detect when models needed retraining based on operational feedback
These are business capabilities, not just compliance activities.
The architecture we built - with X-RAI forcing explicit goal articulation, components providing operational control, and infrastructure maintaining synchronized state - solved a more general problem: how do you ensure that AI systems actually achieve organizational objectives over time?
Most organizations can’t answer that question. They deploy AI systems, hope for the best, and only discover problems when business metrics decline or something breaks publicly. They lack the infrastructure to detect drift, the frameworks to articulate goals measurably, and the feedback loops to maintain alignment between intent and execution.
Decision Operations: The Emerging Field
What I’m describing is Decision Operations (DecOps) - the discipline of operationalizing organizational goals through continuous measurement, adjustment, and stakeholder alignment. Just as DevOps transformed how organizations ship software by treating infrastructure as code and operations as continuous practice, DecOps will transform how organizations ensure their AI systems achieve business objectives.
The core components aren’t unique to ethics. Any organization with AI systems needs:
Structured articulation of goals: What AIRA provides through its questionnaire-based approach. Before you build, can you articulate what success looks like, what failure costs, who needs to be involved, and how you’ll know if things go wrong?
Evaluation planning before production: What the Evaluation Plan enforces. Most teams ship first and figure out evaluation later. By that point, they’ve lost the opportunity to instrument properly or secure resources. Forcing this conversation before go-live changes everything.
Continuous monitoring against goals: What Evaluation Support enables. Not just technical metrics (accuracy, latency) but alignment metrics - is the system achieving what stakeholders expect? Are predictions creating the intended business outcomes?
Capability to detect and respond to drift: What the component architecture supports. The Control Tower for operational adjustment, Catwalk for formal verification, RecordKeeper for historical reconstruction. Together, these create closed feedback loops between system behavior and business intent.
Synchronized state across layers: What version-controlled X-RAI documentation and unchangeable production environments provide. When technical implementation, business documentation, and operational configuration stay synchronized, you can actually know what your systems are doing.
Organizations that build these capabilities won’t just have “ethical AI” - they’ll have AI systems that actually do what stakeholders think they do. In a world where AI deployment is accelerating, that’s not just good governance. That’s competitive advantage.
The Path Forward
The ability to reflect and realize whether your organization is following its guidelines enables you to adjust course or adjust the guidelines. This way, ethics isn’t just about doing what is right, but about knowing what you’ve decided your goals are and having the infrastructure to pursue them systematically.
As AI progresses and businesses evolve to extract value from it, Decision Operations will emerge as an inherent capability. Not just for running ethical AI, but for running businesses that actually achieve the goals they set. The infrastructure we built to meet regulatory requirements turned out to solve a much larger problem: how do you maintain organizational coherence when intelligent systems are making thousands of decisions per day?
Most organizations are discovering this the hard way - through failed deployments, drift they detect too late, and stakeholder trust they lose when systems don’t behave as expected. We had the advantage of regulatory pressure forcing us to build the capability first.
The question for your organization isn’t whether you need Decision Operations. It’s whether you’ll build it deliberately, or keep learning through failure.