Transparency & Trust
We are hot on the trails of all things AI, analyzing recent regulations and legal affairs and now case studies across the board and in particular segments. These build on the digital meetings we started hosting when we pivoted our research agenda earlier this year and sets the stage for the Outsell Signature Event, co-produced with JEGI Clarity and The Journey to Intelligent Work — an AI focused agenda for sure.
And as the days go by and we plan, there is plenty of analytic banter inside Outsell. Our Chief Research Fellow, David Worlock — the great mind and expert behind Outsell FutureScapes — is paying close attention to what’s happening in the arena and just this week sent his views on Transparency. His views are so powerful I felt strongly to include them here and in essence make him this week’s contributor. Thank you, David!
In his words:
There seems to be a gathering consensus that AI, at some point in its development, must be subject to regulation of some form. With draft legislation already in place in the European Union, this seems likely to begin sooner rather than later. In all of the discussion, the word “transparency” is widely used, but seems to denote different regulatory qualities according to who is using the term. Here are a few examples:
1. Transparency Over Training Data. There have been a variety of demands that system developers indicate clearly what data has been used in training and development mode. This should be done with greater clarity than the Google indication that they used two files, “books A” and “books B.” There are clearly sensitivities here when it comes to the use of copyrighted works and the definition of fair use in different jurisdictions, but the publication of a detailed analysis of training content seems fundamental to most regulatory demands.
2. Scope & Intent. Transparency is also brought into play during discussions of the scope and intent of algorithms. One of the problems of “hallucinations” in generative AI undoubtedly stems from the inbuilt requirement for “plausibility” of results, as distinct from demonstrable accuracy. There is therefore an indication that regulators may require AI system developers to clearly identify the priorities and objectives of service use.
3. Bias. There were early reactions to generative AI in terms of inherent bias. Regulators may well demand transparency in this area, requiring developers to audit bias, or to include warning statements about possible bias in contractual or promotional material.
4. Claims. Regulators could demand transparency around claims made for what can be accomplished by a chatbot or by a generative AI environment. They may seek to limit the ability of developers to claim wide-ranging but unproven benefits prior to those benefits being systematised into validated results. It is clear that in some countries advertising regulation and trade practice rules are already being looked at in these terms.
5. Predictability. Regulators may well ask that systems developers make statements about the predictability of results. These of course would be based upon operational experience and trials. This information may become a condition of lease or sale.
6. Explainability. The ability of an AI system to describe how it reached its conclusions may well become a critical point. Since we are moving beyond the “black box” era, and because banking regulations in some countries demand that institutions are fully accountable for the reasons why they came to a particular decision in a particular case, the ability of AI systems in certain sectors to develop a rationale for their conclusions is likely to be important in the future. Another critical area for a Pattern Recognition justification is medical diagnosis, where trust in the relevance of diagnostic alternatives can only be created where the rationale is clear and auditable.
The current draft EU AI Act, which should become law by the end of this year, is strong on the categorisation of risk, and fairly strong on the transparency issues around training data. It also talks about the registration of high-risk AI systems. Its critics say that it needs to look more closely at the issues which need to be clear to users prior to engagement. There is also little sense of the importance of regulating a hitherto unregulated industry in order to gain widespread public and commercial trust in its efficacy. It will be hard to deliver its undoubted contribution to the improvement of society as a whole without that trust.
Thank you, David!
On this note, I am optimistic that even though governments are usually behind when it comes to technology, somewhere, someplace the lights are seemingly on, and we will have some visibility about what’s in our information, knowledge, and decision-making that may eventually look and feel like what’s in our real diet. It’s time.