The final pillar concerns operations across the four application domains: how initiatives are piloted, evaluated, scaled, monitored, and retired. It addresses two opposing problems: pilots that show promise but are never scaled, and deployed systems that are not monitored and degrade over time. AI is treated here as an operational capability with a lifecycle rather than a project with a defined end date.
Implementation is the stage at which strategy is realized or lost, and this pillar is correspondingly the most concrete: structured pilot design with success criteria defined in advance; explicit scale-or-stop decision gates so that pilots reach resolution; production monitoring for performance, drift, equity, and incidents; and feedback loops that return operational learning to the Compass and governance. The institution improves subsequent deployments by capturing the results of previous ones.
In practice this pillar spans six operational building blocks: resourcing and infrastructure (multi-year funding, talent, and compute); change management and adoption (the cultural and workload shift that decides whether a tool is actually used); deployment and integration (moving from pilot to production and wiring AI into the systems people already work in, the SIS, LMS, CRM, and ERP); monitoring, maintenance and optimization (performance, drift, bias, incidents, and cost); scaling, sustainability and decommissioning; and operational governance (service ownership, SLAs, escalation paths, and a service catalog). The biggest risk here is rarely the technology, it is underestimating the human and operational work required to make AI useful and trustworthy at scale.
Why it matters
Strategy that never reaches sustained operations is just intention; this pillar is where value is actually realized or lost. It also closes the loop, monitoring and feedback are how the institution catches an AI system going wrong before it harms someone, and how each initiative makes the next one faster, safer, and cheaper.
Six operational building blocks
Implementation is where strategy meets reality. These six interconnected blocks turn an approved initiative into a reliable, adopted, mission-aligned service, and the human ones, resourcing and change management, are where most efforts actually stall.
Resourcing & infrastructure
Budgeting, staffing, compute, and vendor support, the foundation everything else stands on.
Multi-year AI funding, not one-time pilot money
A build / buy / borrow talent strategy
Cloud, GPU/TPU, and a shared-services model
Connects to Campus Readiness · GRC
Change management & adoption
The cultural shift, training, and workload impact that decide whether a tool is actually used.
Role-based AI competency & upskilling
Communication playbooks & champions networks
Structured change plans (e.g., ADKAR, Kotter)
Connects to Engagement & Collaboration · Roles & Responsibilities
Deployment, integration & workflow
Moving pilot to production, and wiring AI into the systems people already work in.
MLOps / LLMOps adapted for higher ed
Integration via LTI, APIs & SSO into SIS, LMS, CRM, ERP
Connects to GRC (security & privacy by design) · Readiness
Monitoring, maintenance & optimization
Performance, accuracy, bias, drift, incidents, and cost in live systems.
Monitoring dashboards & routine AI health checks
Incident-response playbooks for AI failures
Regular model/prompt reviews; cost tracking
Connects to Compass Track & Reflect · ongoing GRC audits
Scaling, sustainability & improvement
Growing what works responsibly, and planning for long-term viability.
Clear scaling criteria & roadmaps
Decommissioning protocols & a lessons-learned repository
Sustainability reviews, energy, cost, equity
Connects to Maturity & Readiness · future Compass evaluations
Operational governance & accountability
Day-to-day ownership, service expectations, and escalation paths.
An AI service catalog of what's approved & supported
Operational SLAs & clear central/unit handoffs
Integration with the central AI governance body
Connects to GRC · Roles & Responsibilities
AI impact & evidence
Selecting an initiative is an ex-ante judgment; the Strategic Compass handles that. Impact is the ex-post counterpart: the documented, mission-aligned effect of an AI initiative on outcomes for individuals, communities, institutional performance, and the public good. This is how an institution shows that a deployed initiative worked, responsibly, and for whom — and it is where the proof a campus can later point to is generated.
Select with the Strategic Compass (before), prove with impact evidence (after). The two are a pair.
Four dimensions of impact
Outcome impact
What measurably changed?
Changes in completion, quality, access, speed, safety, or productivity — with baseline and post-implementation measures, and comparison groups where feasible.
Equity impact
For whom, and who was left out?
How benefits and harms are distributed across populations, especially historically marginalized groups. Outcomes are disaggregated, not reported only in aggregate.
Governance impact
Was trust strengthened or weakened?
Whether transparency, accountability, oversight, and compliance improved or degraded. A system that gains efficiency while introducing opacity or bias can fail a responsible-AI impact test.
Societal impact
What changed beyond the institution?
Contribution to the communities the institution serves — workforce development, healthcare, climate resilience, and public service.
Three kinds of evidence, shown together
A credible evidence package combines three categories; any one alone is incomplete.
Innovation evidence
That the initiative solves a meaningful problem in a materially new or improved way: the prior-state baseline, what AI makes newly possible or practical, and why AI was the appropriate approach.
Impact evidence
That outcomes changed and the change matters: baseline and post-implementation measures, disaggregation by population, and qualitative evidence of lived experience where effects are experiential.
Governance evidence
That the initiative was reviewed, documented, monitored, and adjusted: risk and privacy assessments, human-oversight plans, incident logs, user notices, and records of approval.
Innovation without impact is theater; impact without governance is risk; and governance without either is paperwork.
The one-page AI Use Case Evidence Profile
A single, comparable record for each AI initiative — lightweight enough to adopt, structured enough to support inventories, governance reviews, annual reports, and institutional storytelling. This is the container real impact data fills over time.
How you can tell
When it's working
Pilots are designed with explicit success criteria and a fixed evaluation window.
There are real scale-or-stop gates, so pilots resolve instead of lingering indefinitely.
AI is resourced as sustained operations, multi-year funding, named talent, and the compute behind it, not one-time pilot money.
Change management is explicit: role-based training, communication, and a champions network so tools are adopted, not just deployed.
New AI is integrated into existing systems and workflows (SIS, LMS, CRM, ERP) with human-in-the-loop by design, rather than bolted on.
Production systems are monitored for performance, drift, equity, and incidents.
Every production AI service has a named owner, an SLA, an escalation path, and a place in a service catalog.
Operational learning feeds back into the Compass and governance, improving the next deployment.
When it's missing
Pilots multiply but none scale, and none are formally killed, the graveyard fills.
Success is declared at launch and never measured against real outcomes.
Pilots are funded once and then starve in production, with no one owning the ongoing cost or staffing.
The tool ships but the people don't change, no training or adoption plan, so it goes unused or is quietly worked around.
AI is bolted alongside existing systems instead of integrated, creating duplicate data and broken workflows.
Systems run unmonitored; drift and bias surface only when someone complains.
Each initiative starts from scratch because nothing was captured from the last.
This is where the pillar stops being a principle and becomes work. Each practice below is something you can build, assign, and evidence, with a concrete first move, an owner, and what proof looks like. Move one practice up one level at a time.
1
Initiative pipeline & portfolio management
AI initiatives move from idea to deployment through a managed, transparent pipeline.
5The cycle visibly changes course (retired policies, rebalanced budgets) and the board sees the evidence.
Next moves
Put the annual cycle on the governance calendar: reassess maturity, review policy adequacy, reflect on incidents.
Include enablement adequacy, equity resourcing, and AI financial sustainability (licenses, tokens, TCO) in the review.
Deliver an annual AI governance report to cabinet and board.
Who owns it
AI governance body chair
Evidence it exists
Governance calendar
Annual report
Documented course corrections
In practice
Worked example
Closing the loop
An institution runs a dozen AI pilots; a year on, all are still 'pilots,' none scaled, none stopped, and leadership can't say which worked. Under this pillar each pilot gets success criteria and a decision gate at 90 days: two scale, three are killed cleanly, the rest iterate. Scaled systems enter monitoring with equity and drift checks, and a quarterly review feeds lessons back into the Compass, so the next round of pilots is screened against what the institution actually learned, not what it hoped.
Watch for
Endless pilots with no decision gate, momentum without resolution.
Treating implementation as a technology rollout and underestimating the human and change-management work, the most common failure of all.
Declaring victory at deployment and never checking real-world outcomes.
No operational owner or SLA, so a live AI service has no one accountable when it breaks.
No monitoring, so a drifting or biased system runs unnoticed until it causes harm.
Readiness & maturity questions
Readiness asks: are we prepared? Use these to surface blind spots before you build, honest “no” answers are where the work is. Representative prompts for reflection, not a scored test \u2014 for the scored version, use the Maturity Assessment.
Organizational Change & Adoption
Is there a change-management strategy for AI?
Are staff and faculty trained on adopting AI?
Do we track readiness to change?
Is communication transparent and anticipatory?
Are user concerns proactively addressed?
Are early adopters supported as champions?
Are resistance patterns analyzed?
Are adoption metrics defined?
Is there alignment between tech and behavior change?
Are staff consulted pre- and post-implementation?
Resourcing & Infrastructure
Are budgets allocated for AI planning and deployment?
Is there capacity for infrastructure scale?
Are staffing plans AI-inclusive?
Is there funding flexibility for rapid AI evolution?
Are existing systems being reviewed for AI compatibility?
Are cloud/data contracts AI-aware?
Are procurement teams familiar with AI-specific needs?
Are support teams equipped to manage AI operations?
Is equity considered in resource allocation?
Are long-term maintenance needs planned?
Systems & Processes
Have business processes been mapped for AI insertion?
Are systems documented and interoperable?
Are data pipelines reliable and secured?
Is automation already part of the operational mindset?