AI Strategic Compass
A conversation-driven framework to screen, score, and select AI initiatives, then plan how success will be measured. Scores are a starting point for dialogue, not a verdict.
The decision engine of the framework, a structured, conversation-driven way to weigh candidate AI initiatives on the same six dimensions, including whether there's evidence it actually works, instead of approving them one at a time on instinct.
Turning a scattered list of AI ideas into a prioritized, governed portfolio, deciding what to pursue, at what level of scrutiny, and how you'll know it worked.
Scores start a conversation; they don't end one. A high score isn't automatic approval and a low score isn't a veto, both are prompts to discuss why.
Plan sets exit criteria and re-review triggers for every band. An initiative that scales is registered in the AI Registry, then operated and monitored.
The six dimensions you score on
Every initiative is rated 1–5 on each of these. Scoring all six (not a single gut-check) surfaces the trade-offs a good idea can hide: an exciting tool can be strong on innovation and weak on compliance, or impressive in a demo with no evidence it actually works.
Whether the initiative advances the institution's mission and current priorities, or is impressive but beside the point. Considers alignment with the strategic plan, the academic mission, and leadership goals.
What could go wrong for the people involved and for the institution. Considers bias and fairness, transparency and explainability, data privacy and security, and legal/regulatory compliance (FERPA, ADA, GDPR, and peers).
Whether the institution can afford it and whether the return justifies the spend. Considers up-front and ongoing cost, staff time and resources required, and short- and long-term ROI and sustainability.
Whether it actually improves how work gets done. Considers efficiency gains, added capacity, fit with existing workflows, and the degree of real innovation rather than novelty for its own sake.
Whether the AI will actually do what it promises. Scored twice: at proposal, on outside evidence plus the quality of the evaluation plan; at the scale decision, on campus pilot results. Scored like the others: 1 = strong evidence it works, 5 = little or none.
How the people affected will experience the initiative. Considers impact on students, staff, and faculty, equity for those most affected, and public trust and institutional reputation.
How success is measured
Scoring decides whether to pursue an initiative. Measurement decides whether it worked. Three measures are set for each initiative: KPIs, OKRs, and efficacy measures. Maturity and Readiness are institution-level context from the Maturity Assessment and are not scored per initiative.
Tangible deliverables, tools deployed, processes automated, people trained. Counts, completion, and adoption.
Mission-level shifts the outputs were meant to produce, retention, equity, satisfaction, research capacity.
Whether the AI itself performs as intended, accuracy, reliability, fairness, error and override rates, and user trust. An output can ship while its efficacy is poor.
How institutionalized the capability has become, on the five-level scale, from ad hoc experimentation to continuous optimization.
How prepared the institution is to activate and scale, leadership, culture, infrastructure, data, and people.
Proportionate review paths
The score doesn't just rank initiatives, it sets how much scrutiny each one needs. A faculty AI writing helper shouldn't clear the same bar as a system shaping financial-aid decisions. The path, the approver, and the requirements scale with the band.
The low band is the normal lane for unit-level work. Check the three escalation questions to confirm it belongs there.
- Self-assessment with the unit lead against the acceptable-use checklist
- Scored rubric, short description, and a named owner
- Monitor at the end of the pilot and at the annual review
- One cross-functional review before launch, with written comments
- Document data sources, access, and retention
- Draft KPIs and OKRs up front
- Monitor at mid-pilot and end of pilot
- Full team review plus specialist reviews
- Efficacy plan with exit criteria
- Risk mitigations with named controls
- Bias / equity and privacy impact assessment
- Stakeholder consultation (including those affected)
- Ongoing monitoring, reviewed at least annually
- First ask: does a lower-risk alternative achieve the goal?
- Affected groups consulted
- Alternatives analysis and algorithmic impact assessment
- Legal, privacy, and compliance review
- Phased pilot with explicit scale-or-stop gates
- Documented rollback / contingency plan
- Continuous monitoring, audit trail, and incident response
The system owner and the unit lead, plus IT security, privacy, and accessibility as the band requires. Include people from the groups affected. When Efficacy is in question, add institutional research or assessment staff, faculty with methods expertise, and the vendor's validation evidence.
Before any decision, ask who is affected, whether it works equally well for each group, whether it meets accessibility requirements, and whether people have a non-AI alternative.
GRC Readiness Lens
Overlay each rubric dimension with Governance, Risk, and Compliance maturity. Rate your institution against the framework's Level 1 / 3 / 5 indicators to see where safeguards are strong and where the gaps are, an institutional self-assessment that complements the per-initiative scores above.