0:00–0:45 | Scope and map
Choose allowed effects and exclusions, assign owners, and trace identity, source documents, tools and downstream actions.
Workshops
Practice checking permissions in an AI document assistant.
The core workshop follows a synthetic document assistant through tool approval, retrieval boundaries and credential revocation. A separate detection module compares a fixed identity rule with a simulated assistant suggestion and an analyst’s final decision. Both have downloadable teaching materials; team-specific delivery is scoped separately.
What you receive
What the engagement covers.
- Three-hour agent workshop: facilitator run sheet, participant packet, worksheets and solution notes
- Separate 90-minute detection module with a labeled fixture, simulated suggestions and local scoring script
- Participant outputs: trust-boundary map, permission matrix, observed test notes and decision record
- Evaluation rubric for boundary reasoning, reproducibility, uncertainty and ownership
- Team-specific agenda and follow-up scope agreed before any facilitated engagement
01
Prepare without production access
- Python 3.10 or later and a text editor run the supplied standard-library examples. No model account, paid API, cloud credentials or package install is needed.
- Pair an engineer with the person who owns the workflow decision. Bring a redacted diagram and one intended use case if adapting the discussion to your team.
- Download and inspect the files before the session. Participants without Python can work from the source and recorded outcomes with a partner.
- Use only the supplied synthetic fixtures. A real system review requires separate authorization, data-handling terms and a test environment.
02
Core workshop: 180 minutes
0:45–1:20 | Approve the exact edit
Check current permissions, changed arguments, stale document versions, replay and revoked reviewer access. Record denials before effects.
1:20–1:30 | Break
Collect unresolved assumptions before the retrieval exercise.
1:30–2:05 | Retrieval and citations
Test two-tenant isolation, revoked access, source versions and citation metadata. Explain what the inert hostile note does and does not test.
2:05–2:30 | Revoke and contain
Compare valid, expired, revoked and wrong-scope fixture grants. Assign real-world connector shutdown and in-flight work questions.
2:30–3:00 | Decide and debrief
Complete a conditional pilot decision, acceptance checks, owners and a review date. Score the quality of reasoning with the supplied rubric.
03
Separate module: evaluating assisted detection in 90 minutes
Start with KevinBytes’ public identity_hunt.py and its recorded eight-case, 35-row fixture. The module adds a clearly labeled simulated suggestion, case-level TP/FP/FN/TN scoring and analyst disposition. No AI vendor or model is run.
- 0:00–0:15: define the rule-match label, time windows and unit of counting.
- 0:15–0:35: run and score the unchanged baseline; inspect tenant and duplicate-delivery cases.
- 0:35–0:55: reveal the simulated suggestion, compare its errors, and record analyst acceptance or rejection.
- 0:55–1:15: distinguish missing telemetry from a clean outcome; write an approval and escalation record.
- 1:15–1:30: reproduce the facilitator solution and debrief. These rule-conformance counts are not attack-detection accuracy or analyst productivity measurements.
04
What participants produce
- An explicit subject/resource/action boundary and at least one legitimate allowed case.
- A reproducible denial case with the point of enforcement and the downstream effect checked.
- A distinction between fixture results, deployment questions and model behavior that was never tested.
- An owner, acceptance criterion and review date for every material rollout condition.
Limits
Scope and limitations.
- The exercises use synthetic data and local runs. They are teaching examples, not reports from client engagements.
- Workshop completion is not certification, production approval or an assessment of the participant’s application.
- The detection exercise uses synthetic event records, a fixed SQL rule and a simulated suggestion. It does not validate Entra, Sentinel, any AI model or real attack coverage.
Related paths
Materials index and setup (Markdown)Facilitator guide and timed run sheet (Markdown)Participant packet (Markdown)Worksheets and evaluation rubric (Markdown)Agent exercise solution notes (Markdown)Detection module and fixture (Markdown)Detection facilitator solution (Markdown)Detection scoring script (Python)Workshop preparation articleControl acceptance testsAI risk board report builder
Next step