Measuring Copilot and Gen AI Success and Risks with Viva Insights and Purview – ECS 2025

Measure Copilot and Gen AI Success and Risks Using Viva Insights, Microsoft Purview, and the Microsoft 365 Admin Center reports

Earlier this month, I had the pleasure of speaking at the European Collaboration Summit 2025 in Düsseldorf, Germany. It was an incredible opportunity to connect with fellow Microsoft 365 professionals and explore a timely, strategic question:

How do we measure the success—and manage the risks—of Microsoft Copilot and Generative AI?

In my session, “Measuring Copilot and Gen AI Success and Risks with Viva Insights and Purview,” I shared practical, real-world strategies for going beyond basic usage statistics to understand the full picture: adoption, impact, sentiment, and security posture.

Why Measuring AI Success and Risk Matters

“🧭 We cannot measure what we do not know.”

As Microsoft 365 Copilot and other generative AI tools become embedded in daily workflows, organisations face a critical choice: measure their impact—or operate in the dark.

Without meaningful measurement, it’s like throwing darts into a black hole—there’s no way to know what’s working, what’s not, or where the risks lie.

Effective measurement empowers organisations to:

• 📈 Prove ROI on AI investments
• 🤝 Drive responsible adoption across teams
• 🔐 Identify and mitigate compliance, privacy, and security risks
• 🎯 Target training to improve user experience and outcomes

Measurement isn’t a luxury—it’s a foundation for trust, transparency, and transformation.

What Happens If You Don’t Measure AI?

Ignoring AI usage metrics opens the door to serious business risks:

🚫 Untracked use of third-party Gen AI (Shadow IT)
⚠️ Compliance violations and legal exposure
💸 Unexpected licensing costs
📉 Missed productivity improvements
🕵️‍♀️ Lack of visibility into how data is accessed and shared

---

Step 1: Start with Microsoft 365 Admin Center Reports

If you’re early in your AI journey, the Microsoft 365 Admin Center offers a solid baseline with its Usage and Adoption Reports:

• Copilot Readiness Reports – Assess technical and organizational preparedness
• Copilot Overview & Usage – Track rollout progress and engagement
• Copilot Agents & Chat – Analyze licensed vs. unlicensed usage
• Adoption Score – Benchmark digital maturity

These tools provide critical starting points for IT and leadership, but advanced insight requires more.

---

Step 2: Use Viva Insights for Copilot Analytics

Organizations with Microsoft 365 Copilot licenses get access to Copilot Analytics in Viva Insights at no extra cost.

The Copilot Dashboard delivers visibility into:

📈 Readiness – Are users prepared for AI tools?
👥 Adoption – How broadly is Copilot being used?
💼 Business Impact – What value is Copilot delivering?
💬 User Sentiment – How do employees feel about Copilot?

💡 Pro Tip: Use the dashboard in Microsoft Teams to spot trends, address friction points, and tailor enablement efforts.

Note: Some features require a minimum of 50 Copilot licenses.

Coming Soon: Business Impact Report

Microsoft is previewing a new Copilot Business Impact Report built in Power BI. This will map Copilot usage to business results, offering even deeper performance insights.

---

Step 3: Manage Risk with Microsoft Purview DSPM for AI

To balance adoption with governance, use Microsoft Purview’s Data Security Posture Management (DSPM) for AI. It helps you identify where generative AI is used and how sensitive data may be exposed—within both Microsoft Copilot and third-party tools.

Even without E5 Compliance, you can:

🔍 Discover Microsoft Copilot usage – Identify activity across Microsoft 365 Copilot and other Copilot experiences
🛡️ Understand oversharing risks – See what sensitive data is being shared with Copilot
📋 Govern AI behavior – Monitor usage patterns and track potential policy violations

These core features help organizations monitor in-product usage of Microsoft Copilot and understand early data exposure trends.

With a Microsoft 365 E5 Compliance license, you unlock deeper insights and stronger protections:

🔍 Detect third-party Gen AI tools – Identify unsanctioned use of tools like ChatGPT, Gemini, or Claude
🛡️ Track oversharing with third-party AI – See when users paste sensitive data into external Gen AI websites
🛡️ Protect sensitive data – Enforce endpoint Data Loss Prevention (eDLP) and apply Insider Risk Management (IRM) policies

🔐 Key Insight: 78% of employees bring their own Gen AI tools to work. Without visibility and control, this introduces serious data security and compliance risks.

---

A Unified Framework for Gen AI Success and Risk

In my ECS 2025 session, I introduced a unified model to help organizations take a structured, data-driven approach to AI governance:

📊 Quantitative Metrics

• Usage data
• Adoption rates
• Readiness indicators

💬 Qualitative Insights

• User feedback
• Behavioral patterns
• Business outcome alignment

🔐 Risk & Compliance Monitoring

• Shadow IT and third-party AI detection
• Sensitive data exposure tracking
• Policy compliance and audit readiness

By combining insights from the Microsoft 365 Admin Center , Viva Insights, and Microsoft Purview, you can build a proactive, resilient AI governance strategy that drives adoption while protecting your organization.

---

Ready to Start Measuring?

Without meaningful measurement, it’s like throwing darts into a black hole—there’s no way to know what’s working, what’s not, or where the risks lie.

Whether you’re piloting Microsoft Copilot or scaling an enterprise-wide AI program, it’s time to treat AI measurement as a business-critical function.

By measuring both success and risk, you gain full-spectrum visibility—into where AI is delivering value, where it’s falling short, and where it may be putting your organisation at risk.

With the right tools, you can illuminate:

• 🌐 Adoption and usage trends
• ⚠️ Policy violations and compliance risks
• 📈 Business value and user sentiment

📡 Reporting gives you well-rounded visibility—like holding the world in your hand and seeing exactly where AI is helping or hurting.

🎯 Don’t let Gen AI be like throwing darts into the abyss.
Make it measurable. Make it manageable. Make it meaningful.

📥 View or download the full slide deck from my ECS 2025 session:

Let’s make your AI journey successful, secure, and measurable.

Together, we can ensure your AI journey is both successful and secure.

---

❓ Frequently Asked Questions (FAQ)

---

📚 Explore Official Microsoft Learn Resources

To dive deeper into Microsoft 365 Copilot, Viva Insights, and Purview capabilities, check out these curated Microsoft Learn resources:

🔎 Copilot in Microsoft 365

• What is Microsoft 365 Copilot? | Microsoft Learn
• Copilot Success Kit – Microsoft Adoption
• License options for Microsoft 365 Copilot | Microsoft Learn

📊 Viva Insights and Copilot Analytics

• Copilot Analytics introduction | Microsoft Learn
• Microsoft 365 Copilot impact report | Microsoft Learn

🔐 Microsoft Purview for AI Governance

• How to use DSPM for AI Data Risk Assessment to Address Internal Oversharing | Microsoft Community Hub
• Considerations for deploying Microsoft Purview Data Security Posture Management for AI & data security and compliance protections for Microsoft Copilot and other generative AI apps | Microsoft Learn

These resources are continually updated by Microsoft and provide the foundational knowledge and implementation steps for anyone managing AI in the Microsoft 365 ecosystem.