How Regulated Firms Fixed Their AI Systems to Pass Review
Recording human sign-off as a workflow step was the most common AI audit readiness fix, named by 10 of 33 regulated firms. Adding records after the build was the most common failure, with 7 of the 26 failures coded in the study.
Study methodology: An open-response survey ran through Connectively and direct outreach between June and August 2026, and 33 of 67 submissions passed screening. Each respondent was asked for a practice that worked and a practice that failed. The coding produced 33 working practices and 26 failed ones.
How regulated firms fixed their AI systems to pass review
| Practice that worked | Firms | Practice that failed | Firms |
| Human sign-off recorded as a step | 10 | Records retrofitted after the build | 7 |
| Evidence captured at decision time | 9 | Periodic reports where live evidence was due | 5 |
| Deterministic core, AI at the edge | 5 | Supplier certification as own validation | 5 |
| Controls at data and identity layer | 3 | Accuracy metrics as compliance evidence | 4 |
| Evidence package per use case | 3 | Prompt or written policy as a control | 3 |
| Change control written before go-live | 2 | Generic checklist for every use case | 1 |
| Evidence pushed to auditor’s system | 1 | Synthetic tests in place of production data | 1 |
| Total coded practices | 33 | Total coded practices | 26 |
Essential statistics: AI audit readiness practices that worked and failed
- Human sign-off recorded as a workflow step was the most common working practice, at 10 of 33.
- Evidence captured at the moment of decision followed, with 9.
- A deterministic core with generative AI confined to the edges worked for 5.
- Controls at the data and identity layer worked for 3, as did a single evidence package per use case with a named owner.
- Records retrofitted after the build were the most common failure, at 7 of 26.
- Periodic reports sent to auditors who wanted live evidence failed for 5, and so did supplier certifications treated as internal validation.
- Accuracy metrics offered as compliance evidence failed for 4, and a prompt or written policy used as a control failed for 3.
Key takeaways
- The 2 leading fixes both create evidence while the system runs. Recorded sign-off and decision-time capture account for 19 of the 33 working practices.
- Evidence that arrived late caused 12 of the 26 failures. 7 firms added records after the build, and 5 sent periodic reports to auditors who wanted live evidence.
- Borrowed assurance failed as often as periodic reporting, with 5 firms treating a supplier’s certification as their own validation.
- 3 firms failed review by using a prompt or written policy as a control. At SynSphere, an auditor rejected a prompt that told a model to withhold a data category.
- 4 firms offered accuracy scores as compliance evidence and failed. CyberMed’s founder noted that the FDA expects traceability from requirement through risk to test evidence.
Actionable insights
- Record each human sign-off as a step the system logs, since 10 of 33 firms named it, more than any other AI audit readiness practice.
- Capture evidence at the moment of each decision, because 9 firms named it as their fix and 7 failed by rebuilding records after the build.
- Keep figures in a deterministic engine and let generative AI write the language around them. CoinLedger and ProAI worked this way, among 5 firms that used the split.
- Run your own validation even when a supplier holds a certification, since 5 firms failed by accepting the supplier’s paperwork as their own.
- Give each use case a single evidence package with a named owner, a setup that worked for 3 firms.
“What did not work was trying to reconstruct that sequence afterward from transcripts, screenshots, application logs, ticketing systems, and separate approval records. Each artifact could be accurate, but the collection still failed to establish a clear chain of custody.” — Ken Herron, Co-founder, VCONify







