Pillar 8Capabilities Produces: Domain-Specific Implementation Canvas

Implementation & Operations

Run AI as sustained operations, not a launch: resource it, manage the change, integrate it, monitor it, and keep improving.

Aligns withNIST · Measure / ManageISO/IEC 42001Incident response

The idea

This pillar covers how initiatives are piloted, evaluated, scaled, monitored, and retired across the four application domains. It addresses two problems: promising pilots that never scale, and deployed systems that go unmonitored and degrade. AI is an operational capability with a lifecycle, not a project with an end date.

It is the most concrete pillar: pilots with success criteria set in advance, scale-or-stop gates, production monitoring for performance, drift, equity, and incidents, and feedback that returns lessons to the Compass. The six building blocks below organize the work. The biggest risk is rarely the technology. It's underestimating the human and operational work of making AI useful and trustworthy at scale.

Why it matters

Strategy that never reaches sustained operations is just intention; this is where value is realized or lost. Monitoring and feedback also close the loop: they catch a system going wrong before it harms someone, and make each initiative faster, safer, and cheaper than the last.

Small-unit formStart, watch, and stop criteria for each use.

Six operational building blocks

Six blocks turn an approved initiative into a reliable, adopted service. Most efforts stall on the human ones: resourcing and change management.

Resourcing & infrastructure

Budgeting, staffing, compute, and vendor support, the foundation everything else stands on.

  • Multi-year AI funding, not one-time pilot money
  • A build / buy / borrow talent strategy
  • Cloud, GPU/TPU, and a shared-services model
Connects to Campus Readiness · GRC

Change management & adoption

The cultural shift, training, and workload impact that decide whether a tool is actually used.

  • Role-based AI competency & upskilling
  • Communication playbooks & champions networks
  • Structured change plans (e.g., ADKAR, Kotter)
Connects to Engagement & Collaboration · Roles & Responsibilities

Deployment, integration & workflow

Moving pilot to production, and wiring AI into the systems people already work in.

  • MLOps / LLMOps adapted for higher ed
  • Integration via LTI, APIs & SSO into SIS, LMS, CRM, ERP
  • Human-in-the-loop templates; pilot-to-production checklist
Connects to GRC (security & privacy by design) · Readiness

Monitoring, maintenance & optimization

Performance, accuracy, bias, drift, incidents, and cost in live systems.

  • Monitoring dashboards & routine AI health checks
  • Incident-response playbooks for AI failures
  • Regular model/prompt reviews; cost tracking
Connects to Compass Track & Reflect · ongoing GRC audits

Scaling, sustainability & improvement

Growing what works responsibly, and planning for long-term viability.

  • Clear scaling criteria & roadmaps
  • Decommissioning protocols & a lessons-learned repository
  • Sustainability reviews, energy, cost, equity
Connects to Maturity & Readiness · future Compass evaluations

Operational governance & accountability

Day-to-day ownership, service expectations, and escalation paths.

  • An AI service catalog of what's approved & supported
  • Operational SLAs & clear central/unit handoffs
  • Integration with the central AI governance body
Connects to GRC · Roles & Responsibilities

AI impact & evidence

The Strategic Compass judges an initiative before it starts. Impact is the after-the-fact counterpart: the documented, mission-aligned effect on individuals, communities, institutional performance, and the public good. It's how a campus shows an initiative worked, responsibly, and for whom.

Select with the Strategic Compass (before), prove with impact evidence (after). The two are a pair.
Four dimensions of impact
Outcome impact
What measurably changed?

Changes in completion, quality, access, speed, safety, or productivity, measured against a baseline, with comparison groups where feasible.

Equity impact
For whom, and who was left out?

How benefits and harms fall across populations, especially historically marginalized groups. Report disaggregated outcomes, not just totals.

Governance impact
Was trust strengthened or weakened?

Whether transparency, accountability, oversight, and compliance improved or degraded. Efficiency gained through opacity or bias fails the test.

Societal impact
What changed beyond the institution?

Contribution to the communities served: workforce, healthcare, climate resilience, public service.

Three kinds of evidence, shown together

A credible evidence package combines three categories; any one alone is incomplete.

Innovation evidence

It solves a meaningful problem in a new or better way: the baseline, what AI made possible, and why AI was the right approach.

Impact evidence

Outcomes changed and it matters: before-and-after measures, broken out by population, plus qualitative evidence where effects are experiential.

Governance evidence

It was reviewed, documented, monitored, and adjusted: risk and privacy assessments, oversight plans, incident logs, user notices, approvals.

Innovation without impact is theater; impact without governance is risk; and governance without either is paperwork.
The one-page AI Use Case Evidence Profile

One comparable record per AI initiative: light enough to adopt, structured enough for inventories, governance reviews, and annual reports. Impact data fills it over time.

How you can tell

When it's working
  • Pilots have explicit success criteria and a fixed evaluation window.
  • Scale-or-stop gates resolve pilots instead of letting them linger.
  • AI is resourced as ongoing operations (multi-year funding, named staff, compute), not one-time pilot money.
  • Change management is explicit: role-based training, communication, and a champions network.
  • AI is integrated into existing systems and workflows (SIS, LMS, CRM, ERP) with human-in-the-loop by design.
  • Production systems are monitored for performance, drift, equity, and incidents.
  • Every production AI service has a named owner, an SLA, an escalation path, and a service-catalog entry.
  • Lessons feed back into the Compass and governance.
When it's missing
  • Pilots multiply but none scale or are formally stopped.
  • Success is declared at launch and never measured.
  • Pilots are funded once, then starve in production with no owner for cost or staffing.
  • The tool ships but nothing else changes: no training or adoption plan, so it goes unused or worked around.
  • AI is bolted on beside existing systems, duplicating data and breaking workflows.
  • Systems run unmonitored; drift and bias surface only when someone complains.
  • Each initiative starts from scratch because nothing was captured from the last.

How to operationalize it

Rate your campus

Each practice is something you can build, assign, and evidence, with a first move, an owner, and what proof looks like. Move one practice up one level at a time.

The five levels: 1 Initial · 2 Emerging · 3 Defined · 4 Managed · 5 Optimizing. Each practice describes levels 1, 3, and 5. Level 2, Emerging: some level-3 behavior exists in pockets, but informally. Level 4, Managed: level-3 behavior is applied consistently, measured, and reviewed on a set cadence.

1

Initiative pipeline & portfolio management

AI initiatives move from idea to deployment through a managed, transparent pipeline.

Not yet rated
1InitialInitiatives launch wherever there's budget and enthusiasm; nobody sees the whole picture.
2EmergingGood practice exists in pockets, but it is informal and has not been institutionalized.
3DefinedA single pipeline screens, scores (Compass), and sequences initiatives into a visible portfolio.
4ManagedThe practice is applied consistently, measured, and reviewed on a defined cadence.
5OptimizingThe portfolio is rebalanced on a cycle against capacity, readiness, and strategy; the roadmap is published.
Next moves
  1. Route all AI initiatives through Compass scoring before resourcing.
  2. Maintain one visible portfolio view: pipeline, active, paused, retired.
  3. Sequence into a 12–24 month roadmap with readiness gates between phases.
Who owns it

AI lead · PMO

Evidence it exists
  • Compass scorecards
  • Portfolio register
  • Published roadmap
2

Measurement, baselines & review

Every initiative defines outputs (KPIs) and outcomes (OKRs) with baselines before launch.

Not yet rated
1InitialSuccess is declared by anecdote; nothing was baselined.
2EmergingGood practice exists in pockets, but it is informal and has not been institutionalized.
3DefinedInitiatives define KPIs and OKRs with baselines at approval; review cadences are set.
4ManagedThe practice is applied consistently, measured, and reviewed on a defined cadence.
5OptimizingMeasurement drives decisions, initiatives are scaled, redesigned, or retired based on evidence.
Next moves
  1. Require KPI/OKR definitions and baselines as a condition of Compass approval.
  2. Set a review cadence per initiative proportionate to its risk tier.
  3. Hold kill/scale reviews, and actually retire what isn't working.
Who owns it

Initiative owners · Institutional research

Evidence it exists
  • Baselined metrics at approval
  • Review calendar
  • Scale/retire decisions
3

AI incident response & monitoring

A defined way to detect, triage, and respond when an AI system misbehaves or causes harm.

Not yet rated
1InitialIncidents surface through complaints or media; response is improvised.
2EmergingGood practice exists in pockets, but it is informal and has not been institutionalized.
3DefinedAn AI incident response policy defines detection, triage, roles, and communication.
4ManagedThe practice is applied consistently, measured, and reviewed on a defined cadence.
5OptimizingIncidents are drilled, post-mortems feed policy updates, and monitoring catches issues before users do.
Next moves
  1. Extend the existing incident-response framework with AI-specific scenarios (hallucination harm, bias finding, data leak via AI tool).
  2. Define severity levels and notification duties, including to affected individuals.
  3. Run a tabletop exercise on one AI incident scenario this year.
Who owns it

CISO · AI governance body

Evidence it exists
  • AI incident response policy
  • Tabletop exercise record
  • Post-mortem reports
4

Annual learning cycle & leadership reporting

Assess → plan → track → reflect, closed annually with a report to cabinet and board.

Not yet rated
1InitialPolicies are created and never revisited; nobody asks whether governance is working.
2EmergingGood practice exists in pockets, but it is informal and has not been institutionalized.
3DefinedAn annual cycle reviews policy adequacy, enablement, equity resourcing, and financial sustainability.
4ManagedThe practice is applied consistently, measured, and reviewed on a defined cadence.
5OptimizingThe cycle visibly changes course (retired policies, rebalanced budgets) and the board sees the evidence.
Next moves
  1. Put the annual cycle on the governance calendar: reassess maturity, review policy adequacy, reflect on incidents.
  2. Include enablement adequacy, equity resourcing, and AI financial sustainability (licenses, tokens, TCO) in the review.
  3. Deliver an annual AI governance report to cabinet and board.
Who owns it

AI governance body chair

Evidence it exists
  • Governance calendar
  • Annual report
  • Documented course corrections

In practice

Worked example

Closing the loop

An institution runs a dozen AI pilots. A year later all are still pilots, and leadership can't say which worked. Under this pillar each gets success criteria and a 90-day decision gate: two scale, three are stopped, the rest iterate. Scaled systems get drift and equity monitoring, and a quarterly review feeds lessons into the Compass, so the next round is screened against what the institution learned, not what it hoped.

Watch for

  • Endless pilots with no decision gate.
  • Treating implementation as a technology rollout and underestimating the change-management work, the most common failure.
  • Declaring victory at deployment and never checking outcomes.
  • No operational owner or SLA, so no one is accountable when a live service breaks.
  • No monitoring, so a drifting or biased system runs until it causes harm.

Readiness & maturity questions

Readiness asks: are we prepared? Use these to surface blind spots before you build. Every “no” shows where the work is. Representative prompts for reflection, not a scored test. For the scored version, use the Maturity Assessment.

Organizational Change & Adoption
  1. Is there a change-management strategy for AI?
  2. Are staff and faculty trained on adopting AI?
  3. Do we track readiness to change?
  4. Is communication transparent and anticipatory?
  5. Are user concerns proactively addressed?
  6. Are early adopters supported as champions?
  7. Are resistance patterns analyzed?
  8. Are adoption metrics defined?
  9. Is there alignment between tech and behavior change?
  10. Are staff consulted pre- and post-implementation?
Resourcing & Infrastructure
  1. Are budgets allocated for AI planning and deployment?
  2. Is there capacity for infrastructure scale?
  3. Are staffing plans AI-inclusive?
  4. Is there funding flexibility for rapid AI evolution?
  5. Are existing systems being reviewed for AI compatibility?
  6. Are cloud/data contracts AI-aware?
  7. Are procurement teams familiar with AI-specific needs?
  8. Are support teams equipped to manage AI operations?
  9. Is equity considered in resource allocation?
  10. Are long-term maintenance needs planned?
Systems & Processes
  1. Have business processes been mapped for AI insertion?
  2. Are systems documented and interoperable?
  3. Are data pipelines reliable and secured?
  4. Is automation already part of the operational mindset?
  5. Are feedback loops designed into workflows?
  6. Are there sandbox environments for testing?
  7. Is performance benchmarking included?
  8. Are system change impacts known?
  9. Are existing tools AI-compatible?
  10. Are manual processes being reviewed for AI value?

Apply this pillar