Introduction 

This is part two of two regarding forming teams to investigate new technologies, such as Large Language Models (LLMs) and Generative AI (Gen AI).   

Part one explored the components of improvement teams to provide a framework for this process.  In part two, I strive to make it real using a fictitious, albeit realistic, scenario on how an improvement team can form, be chartered and pursue the question of whether and how to adopt new technologies.  

Please keep in mind, this is one scenario among a multitude of possibilities. It was chosen because it provides examples of the typical steps for multi-disciplinary teams. 

Genesis: Forming the Team 

The Question 

Edna is the Director of Advancement in a prestigious university whose mission is to raise funds from alumni and other sources.  These funds are used for a variety of philanthropic purposes, ranging from scholarships for students to funding research in medicine and engineering. 

Edna has been reading about LLMs and Gen AI and how they might streamline fundraising and improve fundraising efficacy.  Since Edna does not have deep knowledge of AI, it is not clear how these technologies can improve fundraising at her university, or what the downsides and risks are.  She had several questions, including: 

• What exactly are the benefits and risks of using LLMs and Gen AI in fundraising for our institution?   
• Where in the fundraising process will these technologies provide these benefits?   
• What is the cost of adopting these technologies (e.g., purchasing the technologies and the labor required to change the fundraising process to use them)?   
• What limitations do these technologies have (i.e., will they deliver the results we need)? 

Since these questions are beyond the expertise of her division, she determined that a cross-functional team is the best approach to use to answer them.  Her intent is to form a team consisting of members from different departments of the university who have knowledge about these topics. 

Edna first discussed the idea with her immediate boss, Emelio, the Provost and Executive Vice Chancellor.  Emelio liked the idea of exploring LLMs and Gen AI in principle but was concerned that the team needed executive guidance to ensure the correct questions would  be asked and answered.  Emelio’s reasoning was a decision to adopt these new technologies would likely require approval from the university’s Chancellor, hence the team must be able to provide recommendations with the content and quality required for that level of senior leadership.  The Chancellor had a reputation for asking tough, but fair questions. 

Emelio approved the formation of the team, but required Edna to be a member to provide guidance and communication - that is, she would represent senior leader needs to the team. 

Emelio also asked Edna to draft a team charter for his approval before forming the team. 

The Charter and Team Formation 

Edna realized there was a breadth of expertise across the university that would be very helpful -  namely,  she could tap into faculty members’ knowledge  for the information required across multiple technical/engineering domains.  

Hence, she decided to include faculty from the engineering and healthcare areas since both had experience using LLMs and Gen AI technologies.  Since her team had not documented their processes, and therefore could not easily identify which steps would benefit from AI, she also decided to engage the Industrial Engineering department. 

Edna created the team charter using a template from an online search (for example, see Free Quality Improvement Team Charter Template to Edit Online).  Some areas in the template seemed to go beyond her required scope, so she trimmed it down to the essentials. 

The charter she presented to Emelio included these topics: 

• Objectives 
• Members 
• Desired outcomes (i.e., recommendations from expert input and research) 
• Timeline 
• Budget (in terms of average hours per week expected of each team member) 

Edna determined that if the team stayed focused in meetings, then roughly 2 hours per week for about 8 weeks should result in consensus by the team on technology recommendations. 

Emelio approved the charter and approached the different departments to explain the initiative and ask them to be part of the team. He received the department’s support and the names of individuals who would participate. 

Progress and Solution: Team Activities 

Kickoff 

The first team meeting lasted one hour and consisted of reviewing the charter with the team to obtain their input and consensus on the approach.  The team discussed the desired expertise needed from each member, and clarified what constituted consensus for each decision made.   

During the  meeting, the representative for Industrial Engineering suggested that he/she be given a few weeks to interview members of Edna’s department to develop a high-level, summary process for fundraising.  That way, the full team would have  a process model for identifying where and how AI tools might be useful.  The team agreed, and team meetings were put on hold while this person documented the fundraising process at a high level. 

Progress 

A few weeks after kickoff, the team met to review the high-level process and brainstorm ways AI could improve fundraising. In the next few meetings, they discussed these brainstorming ideas in detail to assess their practicality and effectiveness. After that, the team met to form a consensus.   

Team Consensus 

Those with experience with LLMs and Gen AI spoke about how these technologies were new and consequently had problems with hallucinations, which occur when AI invents things that don’t exist, draws erroneous conclusions, or assumes invalid facts.  These team members provided examples. 

The team reached consensus that this issue not only undermines AI effectiveness but could also damage the university's reputation when approaching potential donors. 

Results: Technology Recommendations 

The team recommended that the adoption of AI within Advancement be put on hold until it objectively does not produce hallucinations.  They also suggested that rather than pursue AI tools, predictive models would provide short-term benefits by identifying donors with the greatest propensity and giving capacity to donate to the university. 

Summary 

Note that the planning prior to forming the team was emphasized in this part of the series:  The more that up-front planning occurs , including obtaining support by senior leaders (Emelio, the Provost and Executive Vice Chancellor), the faster the team can form, begin work, and reach their objectives. 

Also note that while AI is a hot topic in the media, the team identified shortcomings that could undermine fundraising efficiency.  Having a cross-functional team provided the insight to identify this risk, which had a major impact on team recommendations.