Off a Cliff
So, my colleagues and I are standing back watching the stampede on generative technologies. We are seeing automation of knowledge and vast new abilities that are powerful and awe-inspiring. And yet, the rush, induced by FOMO, leaves critical questions unanswered. It reminds me of circa 2001 when the news industry rushed to the web and drove themselves off a collective cliff, eviscerating the industry and their business models in the process. Important distinctions in the print world (which could have held) in a digital one got left by the wayside, gutting the news world. It never recovered.
And another wave is upon us. Just this week:
1. The technology confused one of our analysts with his father and put his place of employment with the other Outsell in Minnesota. So, now our analyst’s father worked in Minnesota at Outsell. The term “hallucinations” has emerged to describe the phenom this analyst described: It’s like ChatGPT has got access to the parallel version of me from Everything Everywhere All at Once.
Do we really want our technology on acid?
2. One client told me it couldn’t differentiate between two crimes, close enough in name, but sufficiently different enough that any lawyer trusting the machine could be disbarred mixing them up.
3. A couple weeks ago, one company in our industry announced it was using the technology in its offering. The company failed to notify data vendors whose data were used to train the solution. It received phone calls asking for a check. Whoops!
As my colleague David Worlock wrote this week in a note inside Outsell about gun violence and data sets analyzing it by region in the US:
In our current enthusiasm for generative AI on the cheap, we are more likely to load into the training data, the repeated and ignorant bias of Twitter and Facebook than we are to load the underlying data on where gun deaths occur in the US. Clearly, until we know what we have loaded into the machine, we have no way of knowing, or qualifying, the value of its output, let alone, depending upon that output to make decisions.
We must slow down because many of the industries we serve and use cases we address are life and death, get out of jail, have a safe flight decisions.
Please take some time to ask and answer a few essential questions:
• Is any new technology we’re “bandwagoning” about really using AI? Or is it simply a rules-based system or is it automated or robotic workflow enhancement?
• Does it really add intelligence? (This reminds me of everyone becoming a dot.com in 2001 or a “SaaS” a decade later…) If we describe ourselves as an AI company, we might as well be describing our homes as having heat and running water. It’ll be part of everything and it’s dumb.
• Does the system really create new knowledge from the data that it is examining? By joining data points, is a new insight created? Do you own this insight? Is it reliable? Or is it a faster way of getting to knowledge that could’ve been laboriously created by hand? Either could be genuine.
• If it really is AI, how well is it “trained”? How extensive was the training data? How up to date is it? How was the data obtained? (If it was copyrighted data, was it licensed, or scraped from someone’s website?!)
• If content is free, is it by definition intended or authorized for use in these systems?
• Is the AI designed to fill a knowledge gap that could not be otherwise filled (i.e., gaps in the genome research market for potential funding)? Or is the AI claim simply press release enhancement? (There’s that heat and running water again!)
• Is there legal risk under GDPR if use of AI has been trained on a corpus of data that includes PII?
• Does corporate insurance cover suits for using this technology incorrectly?
And so on, and so forth. We are seeing more and more experiments in marketing and sales, and with enhancements to our production processes. It’s when we embed this technology in the products and solutions our industry delivers and sells, that we must be clear on our claims and fine print. We can’t rush off a cliff.