Picture this scenario. A customer uses a pharmacy’s AI-powered digital health tool to find foods compatible with their new medication. The tool powered by a general-purpose language model makes a suggestion. The customer follows it. There is an adverse reaction. The pharmacy is asked to explain why that product was recommended. The answer cannot be “the algorithm suggested it.” In regulated environments, that is not a defence, it is a liability. And the regulatory framework is now hardening around exactly this kind of accountability gap. The EU AI Act, now in force, designates AI systems used in health-sensitive contexts as high-risk, with full compliance obligations taking effect by August 2027 requiring documented risk management processes, human oversight mechanisms, and full traceability of outputs. For businesses that have not yet built that traceability into their AI stack, the clock is running.
The explainability requirement is not merely a compliance checkbox. It is a fundamental design question about what kind of AI you are deploying. A black box model, even an accurate one, cannot tell you why it produced a given output. There is no rule to inspect, no logic to audit, no chain of reasoning to present to a regulator, a customer, or a legal team. A glass box system one built on explicit, traceable, dietitian-validated rules can answer every one of those questions. If a product is flagged as suitable for a coeliac user, there is a rule explaining exactly why: the ingredient list was checked, cross-contamination risk was assessed, and the output was determined by a logic chain that a clinician reviewed and validated. Every output is traceable. Every decision is defensible. This is not a technical distinction, it is an organisational risk distinction.
The consistency problem makes this even more pressing. A customer asking a chatbot, filtering a product search, and scanning a barcode should receive the same answer and yet, in most enterprise AI deployments, they frequently do not. Nutrition logic is duplicated across teams and tools, rules conflict, and the same product can be flagged differently depending on which part of the platform the customer is using. That inconsistency is not just frustrating; for a business with a duty of care, it is a live liability. A single source of truth, a centralised reasoning layer that governs outputs across every channel is the only architecture that eliminates this risk at scale. The EU AI Act requires that high-risk AI systems provide appropriate levels of accuracy, robustness, and transparency, with a documented, ongoing risk management process covering the entire AI lifecycle.
There is also a commercial logic to explainability that often goes underappreciated. Consumers trust recommendations they understand. When a platform can tell a user not just what to eat but why explaining the dietary logic, the clinical reasoning, the specific ingredient checks that went into a recommendation that transparency is itself a loyalty driver. It transforms a product filter into a trusted advisor. It moves the platform from “this app suggested it” to “I know why this was recommended for me.” In a market where 43% of consumers say they do not trust AI for nutrition advice, closing that trust gap is not just a regulatory imperative. It is a competitive one.
Sources:
Gardner Law / EU AI Act (2025). The EU AI Act Has Arrived. https://gardner.law/news/eu-ai-act-compliance-timeline
Legal Nodes (2026). EU AI Act 2026 Updates: Compliance Requirements. https://www.legalnodes.com/article/eu-ai-act-2026-updates-compliance-requirements-and-business-risks
Nutrition Insight (2026). US consumers turn to unaccredited nutrition advice. https://www.nutritioninsight.com/news/us-consumers-social-media-ai-nutrition-advice-survey.html
More in this series
Clinical to Kitchen – Building the Infrastructure for Food as Medicine
The Audit Trail – “Because the AI Said So” Is Not a Legal Defence
