Generative AI was introduced to technology leaders as a transformative addition to their company’s toolbelt. Yet organizations are still struggling to reap the technology's full benefits and accurately assess its impact.
Organizations are predicted to abandon nearly one-third of generative AI projects after the proof of concept stage by the end of next year, according to Gartner research, which blamed rising costs and unclear business value as contributing culprits.
Tuning, customizing or simply deploying generative AI models costs businesses an average of at least $5 million. Naturally, companies are expecting a return on their investment.
“We’re a pretty well-funded IT department, so we have access to the resources and tools we need,” Brent Harris, CIO at Tulsa, Oklahoma-based community mental health center Family & Children’s Services, said. “But we’re also nonprofit, and we try to be responsible with the money we’re given.”
The organization is assessing generative AI’s ability to improve workflows, provide better data-driven decisions and reduce repetitive tasks by creating cheat sheets for frequently-asked-questions and step-by-step guides.
Tech leaders want to ensure generative AI use cases are set up to succeed. To avoid pouring money into failed pilots, executives are weighing implementation costs against the potential value and risks the technology could bring.
“We certainly imagine that list [of use cases] is going to grow,” Harris said. “We’ve tried to be mindful of being on the leading edge, but not the bleeding edge. We don’t want to be the guinea pigs, but we don’t want to just keep up.”
Where generative AI pays off
Businesses land on a wide spectrum when it comes to AI readiness. This can impact their ability to determine ROI.
Poor data quality and management stifle aspirations. Caution around cybersecurity and protecting sensitive information presents additional hurdles. Plus, generative AI tools can open the business up to risks related to the tech’s inaccuracy and bias.
Despite potential risks, one of the leading adoption barriers for enterprises is an inability to estimate and demonstrate the value of AI projects, according to a Gartner survey published in May. Less than half of AI projects ever make it to the production stage.
“A lot of companies plunged in with hype, converting everybody and any use case that came up, they tried it,” Shohreh Abedi, EVP and chief operations and technology officer and membership experience at AAA - The Auto Club Group, told CIO Dive. “We didn’t do that. I set up a structure with a methodology and a process and put guardrails around it.”
Businesses that rush into generative AI adoption have reported buyer’s remorse. Around 1 in 4 IT leaders said they regretted investing in AI so quickly, according to a March Asana survey.
Still, tech leaders are trying to keep pace with broader business goals, priorities and responsibilities.
At Family and Children's Services, Harris worked to find a balance between innovation and performing the “basic blocking and tackling,” as finding staff, therapists, clinicians and other experts has become more difficult.
“The more time we can spend automating what they do and making documentation easier for them it helps with retention,” Harris said. “If, relative to our peers, it’s much more difficult to use our systems, we’ll have a hard time bringing on and retaining quality staff, so there’s a balance but nothing is done at the risk of security.”
Eager adopters have found value in taking a measured approach to piloting the technology, explicitly aligning use cases with business goals and success metrics.
AAA’s second-largest club focused its own generative AI push on content management, internal support services and coding tests as it laid out its strategy last year. The organization has a dedicated team sifting through use case proposals, led by Abedi alongside the group’s chief architect and CISO.
“I give the final approval on what use cases we’re going to look at, and they have to meet certain criteria,” Abedi said. “The use cases have to meet a business goal, either growth, cost containment or efficiency from a productivity increase.”
When a use case meets the standards, and only then, does the organization move forward with a proof of concept.
