For IT, data & security

Build it, secure it, run it

The leadership page covers what to govern and who decides. This page covers how IT builds, secures, and runs it.

Starting points to adapt to local security policy, not a substitute for it. Threats map to the OWASP Top 10 for LLM Applications. Review control and contract language with your security team and counsel.

The IT path, in order

From surfacing a system to running it safely, in order.

Rapid risk triage

Most decisions come down to one question: fast approval or full review? Triage on three axes: the data it touches, how much it acts on its own, and who's exposed.

Light path: days, not weeks
  • Public or low-sensitivity data only
  • Human reviews every output; the tool can't act on its own
  • Internal audience, easily reversible
  • Reputable vendor with terms already vetted
Full review: slow down
  • Touches FERPA, PII, health, or research data
  • Acts autonomously or feeds a consequential decision
  • Student- or public-facing at scale
  • New vendor, or our data may train their model

The AI security layer

The threats your CISO will ask about first, each with a campus example and a control.

Prompt injection

LLM01

Hidden instructions in input or fetched content hijack the model.

On campusA student's pasted text makes a grading assistant ignore its rubric; a malicious page redirects a research agent.

ControlTreat all model input as untrusted, limit tool permissions, and keep a human on consequential actions.

Sensitive information disclosure

LLM02

The model exposes data to the wrong user, or to a vendor that retains it.

On campusStaff paste FERPA records into a personal AI account; a chatbot returns one student's data to another.

ControlDLP at the boundary, no sensitive data in unvetted tools, and contractual no-training / no-retention terms.

Improper output handling

LLM05

Model output flows into SQL, shell, HTML, or email without validation.

On campusAI output is rendered straight into a page or used to build a query, opening injection or XSS.

ControlValidate and encode every output before it reaches another system.

Excessive agency

LLM06

The system can do more, more autonomously, than the risk justifies.

On campusAn agent connected to the SIS can modify records, not just read them.

ControlLeast privilege, read-only by default, and approval gates for state changes.

Supply-chain & embedded risk

LLM03

Risk inherited from vendors, plugins, and AI switched on inside existing systems.

On campusAn LMS update activates AI that processes student work under new data terms.

ControlInventory embedded AI, track subprocessors, and re-review terms when features change.

Unbounded consumption & cost

LLM10

Unmetered usage drives runaway spend and enables abuse.

On campusA looping agent or exposed API key burns thousands in tokens over a weekend.

ControlRate limits, budget caps with alerts, key rotation, and per-app quotas at the gateway.

Incident response: the first hour

When an AI feature leaks FERPA data, the first hour decides the blast radius. Agree on this runbook in advance.

0–15 min

Detect & triage

Confirm the incident, name an incident lead, classify severity by data class and exposure, and open the incident channel.

15–30 min

Contain

Disable the feature, revoke keys and tokens, and cut the data path. Preserve prompt, response, and access logs first.

30–45 min

Assess scope

Determine what data was exposed, whose, and to whom. Map it to FERPA, breach-notification, and data-residency obligations.

45–60 min

Notify

Engage legal, privacy, the CISO, and comms. Start the clock on notification SLAs.

Day 1–3

Remediate & recover

Fix the root cause, rotate credentials, restore service with tighter scope, and document the timeline.

Within 2 weeks

Post-incident review

Run a blameless review and feed the lesson into policy, the registry entry, and controls.

This maps to the incident-response lifecycle you already follow. Run AI incidents through your existing CSIRT; this names the AI-specific moves at each phase.

NIST SP 800-61Computer Security Incident Handling Guide

Follows its preparation → detection/analysis → containment/eradication/recovery → post-event lifecycle.

SANS PICERL6-step incident handling

Preparation, Identification, Containment, Eradication, Recovery, Lessons learned, in the same order.

ISO/IEC 27035Information security incident management

The same plan, detect, assess, respond, learn cycle, useful if you're ISO-aligned.

Working with your vendor when something goes wrong

Some incidents run through the vendor: a shared quota drained, a safeguard that didn't fire, a model change nobody announced. One CIO described a student who used 50,000 tokens in a day, leaving too few for a staff pilot launching that same day. The campus and the vendor traced where the safeguards had failed (the student should never have been able to run that much), and the vendor restored the tokens. Prepare for that conversation before you need it.

Before: usage safeguards
Per-user and per-app quotas, so one account can't drain a shared pool
Alerts at 50%, 80%, and 100% of daily and monthly budgets
Separate allocations for pilots and production
A named vendor contact and escalation path in the registry entry
During: escalate with the vendor
  1. 1
    Capture the evidenceUsage logs, timestamps, account IDs, and the quota or setting that should have stopped it.
  2. 2
    Diagnose with the vendorFind where the safeguard failed: a missing per-user cap, a misconfigured limit, or a vendor-side fault. Agree which side owns the fix.
  3. 3
    Ask for a remedyRequest restored credits or service credits for usage your controls should have prevented, and a date for the fix.
  4. 4
    Close the gapSet the cap, confirm the alert fires, and record the outcome in the post-incident review.

Put it in the contract: usage-cap controls, alerting, a named escalation contact, response times, and credits for consumption caused by vendor-side failures.

When AI causes harm: communicating through it

The runbook stops the damage; communication decides whether people keep trusting you. When AI causes harm (a biased decision, a privacy exposure, a wrong answer that reached someone), the response is as much reputational as technical.

Say it early, even when incomplete“We know X, we're doing Y, more by Z” beats silence, which fills with speculation.
Own it; don't deflect to “the algorithm”A named person or office takes responsibility for the system and its outcome.
Lead with those affectedTell the people harmed first and directly: what it means for them and how to get recourse.
One coordinated voiceRoute through comms, with legal and privacy aligned. Mixed messages compound the damage.
Close the loop publiclySay what changed so it can't recur. Accountability rebuilds trust, not the apology alone.
Who to tell, and in what order
  1. 1
    People directly affectedPlain-language notice: what happened, what it means for them, what you're doing, and how to appeal or get help.
  2. 2
    Internal communityFaculty, staff, students, and governance bodies, before external channels.
  3. 3
    Leadership & boardFacts, exposure, and response, before any public account.
  4. 4
    Public & pressLast, coordinated with legal: factual, no minimizing, no over-promising.

Pre-draft holding statements and an escalation tree. Mid-incident is the worst time to design communications.

Identity, access & the data pipeline

Six controls that put "govern by data" into practice.

A reference architecture

AI has to coexist with the SIS, LMS, ERP, IdP, and data warehouse. A layered reference keeps governance at the seams.

Identity & accessSSO, MFA, RBAC: who can use it, and as whom.
AI gateway / policy layerOne control point: routing, guardrails, DLP, logging, rate and cost limits.
ModelsVendor APIs, hosted, or on-prem, swappable behind the gateway.
Data & retrievalRAG sources with access scoping and classification, not a flat index.
Systems of recordSIS / LMS / ERP / warehouse, read-only by default, least privilege.
ObservabilityLogs, metrics, alerts, and an audit trail across every layer.
Before you integrate, ask
  • What data crosses the boundary, and at what classification?
  • How does it authenticate, and what can it do if a token leaks?
  • What's the failure mode when the model or vendor is down?
  • Which subprocessors touch the data behind the vendor?
  • How do we roll it back, and what's the audit trail when we do?

Procuring AI services, end to end

The sections on this page, in the order you'll use them when buying an AI service.

  1. 1

    Define the need & triage it

    Name the problem and the highest data class it touches, then choose the light path or full review.

    See: Rapid risk triage
  2. 2

    Choose build, buy, or embedded

    Pick the path and price the total cost, not the sticker.

    See: Build, buy, or embedded
  3. 3

    Run the security & risk review

    Score the tool against the AI security layer before it ships.

    Tool Evaluation Matrix
  4. 4

    Negotiate the contract

    Require the DPA clauses before you sign.

    See: Procurement & contract teeth
  5. 5

    Onboard the plumbing

    SSO, scoped keys, DLP, and retention set before anyone uses it.

    See: Identity, access & the data pipeline
  6. 6

    Deploy, monitor & manage the vendor

    Log, alert, and re-review terms when features or subprocessors change.

    See: Monitoring & observability
  7. 7

    Plan the exit

    Decide up front how you'll export data, prove deletion, and stand the tool down.

    See: Decommissioning & vendor exit

Build, buy, or embedded, with full cost

Three paths, when each fits, and the catch, plus the costs a sticker price leaves out.

Embedded vendor feature

Best whenThe capability is a small part of a tool you already run, with acceptable data terms.

Fastest; no new infrastructure.

You inherit their model, data handling, and release schedule, and may not control when it turns on.

Licensed AI platform

Best whenYou need institution-wide breadth and want a vendor to carry the operational load.

Managed, supported, quicker to scale.

Lock-in and per-seat cost; governance and data accountability stay with you.

Build on a model API

Best whenThe use case is differentiated, sensitive, or needs tight control.

Full control of data, prompts, and integration.

You own security, monitoring, and the staff to run it for the life of the system.

The total cost, not the sticker price
Inference / token spend at production volumeData egress and vector-store costsMonitoring & observability toolingThe FTE to operate, tune, and patch itSecurity review and pen-testingAccessibility remediation (VPAT gaps)

Procurement & contract teeth

The cheapest moment to govern AI is before you sign. Require these clauses in the contract or data-processing addendum.

No training on our data

Without an explicit opt-out, student and institutional data can become training material you can't recover.

Deletion & return on termination

A provable path to get data out and destroyed when the contract ends.

Subprocessor disclosure & notice

The vendor's vendors touch your data. You can't govern what you can't see.

Breach notification SLA

A defined window lets you meet your own regulatory and contractual deadlines.

Data residency & location

Where data is processed drives FERPA, export, and some state obligations.

Security attestations (SOC 2 / ISO 27001)

Independent evidence of controls, not marketing claims, and it satisfies audit.

Accessibility (VPAT / WCAG / Section 508)

A current, accurate VPAT is a procurement gate; an inaccessible tool is legal and equity exposure.

Right to audit & model documentation

You may have to explain a consequential AI decision and need the documentation to do it.

Decommissioning & vendor exit

Every AI service ends. An unplanned exit strands data and misses obligations. Plan it while you still have leverage: before you sign.

When it's triggered
  • The tool is retired or replaced.
  • The vendor is acquired, changes terms, or discontinues the product.
  • A pilot ends without renewal; don't let it linger unmanaged.
  • The vendor fails or a continuity event forces a fast cutover.
Data export & portabilityGet data out in a usable, documented format on your timeline, not the vendor's.
Provable deletionGet written confirmation that your data, including backups and embedding stores, is destroyed per contract.
Records & audit continuityKeep the logs and AI decision records you may still owe under FERPA or public-records law.
Access & credential teardownRevoke keys and tokens, remove SSO/SCIM connections, and close SIS/LMS/ERP integrations.
Registry & documentation updateMark the entry retired with the date, reason, and where its data went.
User communication & migrationTell users what's changing, where the capability moves, and what happens to their content.

Clean exits depend on the same DPA terms you negotiate at procurement: deletion and return, portability, and audit rights. Exit planning and contracting are one decision.

Monitoring & observability

Set this before go-live: what to log, what to alert on, and an audit trail that survives a records request.

What to log
  • Who asked what, and what came back (metadata first)
  • Consequential decisions and any human overrides
  • Access, key use, and integration calls
  • Errors, refusals, and guardrail trips
What to alert on
  • Anomalous request volume or cost spikes
  • Jailbreak / injection patterns
  • Rising error or override rates (a drift signal)
  • Access from unexpected identities or locations

Build the audit trail to survive a public-records or FERPA request: complete, tamper-evident, retained to policy, and not itself a sensitive-data store.

A quick path through the framework

  1. 1
    Document the estate

    The layered registry & documentation model, service, system, and model cards; use-case and risk registries.

    AI Registry & Documentation
  2. 2
    Evaluate tools

    A structured due-diligence matrix for vetting AI tools and vendors against your requirements.

    Tool Evaluation Matrix
  3. 3
    Ground the GRC

    The governance, risk, compliance & data-governance pillar, the substrate AI safety stands on.

    Governance & Risk
  4. 4
    Right-size review

    Proportionate review paths so low-risk tools move fast and high-stakes ones get scrutiny.

    Strategic Compass

First step: stand up the use-case registry and surface the shadow AI already in use.