Coming Out of the Shadows: Making AI Use Transparent in Organizations

Imagine you are in a video conference with an important client who will share highly confidential and proprietary findings from a major medical study. You close your office door, make sure no one in the hallway can hear, and prepare to take careful notes. At that moment, the video conferencing platform pops up a prompt: "Enable Video Assistant." You click the link, and it shows that the platform can take notes, summarize the meeting, and draft a thank-you email to the client afterward. You think it sounds useful, so you enable the assistant.

A week later, the IT team sends out a survey, and one question asks whether you have used any AI products at work in the past year. You know that company policy requires prior approval before using AI tools, yet you click "No" and close the survey. But in reality, you did use AI—you used it to record and process sensitive client data. This makes you part of an emerging challenge many employers now face: how to identify and address "shadow AI" use within their organizations.

What is shadow AI?

Over the past decade, AI capabilities have been embedded in many software platforms that businesses rely on, often without enterprise users realizing it. For example, Office365's anti-phishing and spam filters have used AI and machine learning since the mid-2000s to improve detection; services offered by major cloud providers also include AI capabilities such as natural language processing, computer vision, and recommendation systems. However, few organizations consider themselves to be using AI because of these features.

Today, AI companies are more explicit about the AI capabilities of their tools, but a new issue has emerged: employees use publicly available and often free AI products for work, either intentionally or unintentionally, without disclosing it to their employers. This "shadow AI" use refers to employees using AI tools without the knowledge, approval, or oversight of the IT department or management. It can stem from individual or departmental use outside of a formal, centrally managed AI strategy, or it may reflect a lack of awareness that the tools are powered by AI.

Here are several examples of shadow AI use:

AI integration in enterprise software.In some cases, organizations may not realize that AI exists underneath. Cloud computing is one example—many businesses do not fully recognize its AI-driven nature.

AI as a product feature.As in the video conferencing example, AI is a feature of a larger software product that is used for non-AI purposes (such as video conferencing). Document management systems may also be like this—they might use AI to assist with content indexing and search optimization, but organizations typically view them only as document management tools.

AI-enhanced hardware.Similar to software features, hardware devices (such as cameras, sensors, and IoT devices) can integrate AI for image recognition, speech recognition, or predictive maintenance. Organizations may use these devices without recognizing their AI components.

Unofficial AI tool implementation by individuals or business units.In these situations, employees or business units understand that they are using AI tools, yet they still choose to use them, even if it may violate the organization's acceptable use guidelines. These tools are typically publicly available and often free. ChatGPT, Llama, and Bard are common examples of tools used by employees personally. There are real cases where employees' personal use of public AI tools has led to the leakage of confidential business information or proprietary client information. Often, employees either do not fully understand the potential risks of the tools or are willing to ignore the risks, believing that the improvements brought by the tools are in the best interest of the business.

Addressing shadow AI use

Why is shadow AI a potential problem? Despite the risks its use brings, many organizations have implemented some form of third-party vendor management process for software and hardware, including security and privacy risk assessments and contractual confidentiality commitments. Therefore, even if the AI computing is not fully understood, the solution itself has been deemed safe by the organization. But this is not the case for unofficial implementations by employees or business units, which means it poses the greatest risk to the organization and must be prioritized in any shadow AI strategy. This is especially true when shadow AI tools are used to process confidential or proprietary company information, or information regulated by privacy, medical, or financial laws.

Strategic components

An effective shadow AI strategy includes several essential elements.

Employee surveys.Surveying employees can provide valuable insight into the scope of shadow AI use, while also giving employees a channel to share their feelings about AI adoption. For the survey to be effective, respondents should be informed that their participation will not affect their job tenure, and consideration should be given to whether to allow anonymity.

A typical survey should examine AI use from multiple angles:

• Usage: Ask employees whether they use AI tools in their daily work. A list of common AI tools should be provided, with space left for respondents to specify the tools they use. Organizations should clearly state that personal use of AI tools related to work should be disclosed in the survey.
• Purpose: Ask what specific tasks or purposes the AI tools are used for—data analysis, customer support, marketing, or other functions?
• Frequency: Determine how often employees use AI tools—daily, weekly, monthly, rarely, or never.
• Satisfaction: Ask about the level of satisfaction with the AI tools used.
• Barriers: Ask whether there are any barriers or challenges preventing them from using AI tools.

Open-ended questions should also be provided to allow employees to share any comments related to their use of AI tools.

Based on the survey results, develop an action plan to address identified issues or opportunities, which may include improving AI training, addressing challenges, or optimizing AI tool use. Consider communicating the survey results to employees to show that their feedback is valued and to keep them informed about the organization's AI initiatives.

Acceptable use policies.Many organizations already have policies that define acceptable, restricted, and prohibited behaviors for employee use of AI tools. Like other company policies, the effectiveness of an acceptable use policy depends on the company's communication, training, and enforcement. To that end, companies should clearly communicate the policy to all employees, contractors, and stakeholders, through training, distribution of written guidelines, and regular reminders of the policy's existence and scope. Organizations should help employees understand the potential risks of non-compliance and the company's commitment to responsible AI use.

Monitoring and auditing are key components of policy enforcement. Organizations can implement technical controls to track AI system usage and data handling practices. These controls can include logging AI system activity, conducting regular audits, and using AI monitoring tools to detect potential violations. Automated alerts and reporting mechanisms can be set up to notify relevant personnel when deviations from policy occur, enabling rapid intervention.

The consequences of policy violations should be clearly defined and consistently enforced. An AI acceptable use policy should outline the disciplinary measures that will be taken if employees or teams violate the policy, ranging from warnings and retraining to more severe measures such as suspension, depending on the severity of the violation. Enforcing consequences conveys the organization's commitment to responsible AI use and helps deter potential misconduct.

Technical controls.Organizations can leverage existing technical controls to identify shadow AI use within the organization. For example, establishing network traffic monitoring systems or security gateways can detect data flows and application usage patterns. By analyzing this data, IT teams can locate unexpected AI-related activities, websites, and applications that may not be formally recognized or authorized by company policy.

Second, organizations can deploy endpoint detection and response (EDR) solutions focused on devices and endpoints where shadow AI implementations may occur. EDR tools can identify unauthorized AI software installations, track their behavior, and provide insight into potential security risks. Additionally, these tools can monitor for unusual file or process activity associated with shadow AI applications, helping organizations act quickly to mitigate potential threats or breaches.

An organization's ongoing vulnerability scanning and penetration testing activities can proactively identify security weaknesses related to shadow AI use. These technical controls assess the security posture of organizational systems and applications, including any unauthorized AI implementations. Regular testing helps uncover vulnerabilities and weaknesses that could be exploited by malicious actors or expose sensitive data.

Finally, depending on the business risk profile, blocking known external AI websites to prohibit access from company devices and networks may be appropriate.

By combining these technical controls, organizations can develop a comprehensive strategy to detect and manage shadow AI, ensuring all AI initiatives comply with security and compliance requirements while minimizing risks associated with unauthorized AI deployments.

Reporting channels.Companies can also consider using internal reporting channels and hotlines to detect and address shadow AI use. These mechanisms provide employees with a confidential and secure way to report concerns or suspicions about unauthorized AI tools or implementations. As with other uses, employees should be educated about the existence of these channels and assured of their confidentiality and protection. The reporting process should be simple and straightforward, ensuring employees can easily submit concerns without fear of retaliation. Offering anonymous reporting options can further encourage employees to provide information about shadow AI use. Investigations should be thorough, and if shadow AI is found, appropriate corrective measures should be implemented promptly.

Third-party vendor management agreements.When shadow AI originates from AI features embedded in vendor software or products, updating existing third-party vendor management agreements can be helpful. When bringing in third-party vendors or partners, organizations should include AI-related questions in their due diligence, requiring vendors to disclose whether their products embed AI or include it as part of the service, and to provide detailed information. This information helps organizations identify potential shadow AI risks early in the vendor relationship. Once AI use is identified, more challenging issues may surface that can be addressed through the vendor contract process, including data ownership of AI outputs and models, auditing of AI outputs, how the effectiveness of the AI models involved is measured, and indemnification/liability for AI defects or damages.

Strategic integration

Of course, shadow AI detection should be part of an organization's broader AI governance strategy. An effective strategy helps organizations navigate the complex and evolving AI regulatory landscape while assessing the responsible, ethical, and compliant use of AI technologies.

Such a strategy typically includes creating a cross-functional team or committee of AI stakeholders responsible for AI governance. This team should consist of members from different departments, each with distinct roles and responsibilities, collectively forming a comprehensive AI governance framework:

• Executive leadership.The team should include senior executives or C-level members to provide strategic direction and endorsement for AI governance efforts. They set the overall vision and priorities, allocate resources, and ensure AI aligns with the organization's strategic goals.
• Legal and compliance experts.Legal professionals are crucial for navigating the complex AI regulatory environment, ensuring AI initiatives comply with relevant laws and regulations, such as employment/HR laws, data privacy requirements, intellectual property, and industry-specific regulations.
• Data stewards and data scientists.Data stewards oversee data quality, integrity, and access. Data scientists contribute their expertise in data analysis, model development, and model validation. They collaborate to ensure the data used for AI is accurate, representative, and ethically sourced.
• Ethics and diversity experts.Depending on the business use of AI, the team may include ethics or diversity experts focused on issues of fairness, bias mitigation, transparency, and accountability. This stakeholder works to ensure AI systems do not discriminate against any group and that ethical principles are woven throughout the AI lifecycle.
• IT and security professionals.IT experts play a key role in protecting AI systems, data security, and implementing technical controls, ensuring AI applications are resilient to cyber threats and that AI assets are adequately protected.
• Risk management and audit teams.These professionals assess and manage risks related to AI, conduct regular audits of AI systems, and provide recommendations for improvement, helping the organization stay vigilant about potential issues and liabilities.

The AI team or committee should work together to develop AI policies, oversee the monitoring of AI tool deployments, assess related risks, and recommend adjustments to AI implementations. In addition to rolling out AI usage policies, the AI stakeholder committee should also work to communicate company policies internally to employees and raise awareness of shadow AI use within the organization.

In an era where AI is increasingly integrated into vendor software and business operations, identifying AI systems that operate outside official oversight is an important business priority. By combining comprehensive technical controls, robust reporting mechanisms, and a culture of transparency, organizations can illuminate shadow AI and mitigate the risks of unauthorized AI use.