About this test. This is a synthetic test: a desk test of one of our own published frameworks, run against 12 invented workflows. No real businesses, customers, or AI tools were involved, and nothing here shows how any workflow would perform in practice. The scenarios, expected answers, and scoring were produced by the AI writing assistant that helps run this site, on October 19, 2026, following a method written down before any scoring. Every input, rule, and score is published below so you can check our work.
The short answer
We built our AI Readiness Checklist to answer one question: is this specific workflow ready for a small AI pilot? So we tested whether it gives the right answer.
Here is what happened across 12 workflows:
- It stopped every dangerous setup. All three workflows where AI would make a call about someone’s health, money, or job without a person checking scored 0 on the “consequence” question and tripped the red flag for consequential decisions without human review.
- It told all five “not ready yet” workflows to fix things first. No false green lights there.
- It could not tell us when AI wasn’t needed. Two workflows were plain rules jobs: send reminders on a fixed schedule, and text staff their weekly schedule. The checklist has no way to say “use a normal automation.” Read generously, it scored both 16 out of 20, which is the “may be ready for a pilot” band.
- It never asked whether a job was worth automating. A task that takes 15 minutes a week got the same treatment as one that takes an hour a day.
- Its scores leaned heavily on facts owners don’t have yet. One question, about pilot boundaries, was unanswerable for all 12 workflows, because none had a pilot plan. That is the thing you’d use the score to decide.
The practical takeaway: the checklist is a good brake and a poor compass. Use it to find reasons to stop. Don’t use it alone to decide that AI is the right tool. Ask two questions before you score anything (below), and treat a 0 on the consequence question as “not this workflow,” not “not yet.”
Why we tested our own checklist
A scorecard that sounds sensible can still give bad answers. If it waves a risky project through, someone loses money or trust. If it blocks everything, nobody uses it. And if it quietly assumes AI is the answer, it will push owners toward tools they don’t need, which is the opposite of what this site is for.
We publish frameworks for readers to apply to real decisions. We should know where ours break.
How the test worked
We set the rules before we looked at the results.
- Wrote 12 workflow cards. Each card describes an invented small business and one workflow: volume, current process, data involved, stakes, owner, review step, tools, and what’s been measured. We spread them on purpose: a few good candidates, several that needed preparation, two that didn’t need AI, and three that were risky by design.
- Wrote down the right answer first. For each card we recorded what a sensible advisor would conclude, using four plain categories: pilot now, fix first, use rules, not AI, or keep a person in charge. We also listed the specific risks a good checklist should flag (35 in total). This was written before the checklist’s scoring criteria were opened during the test. We recorded a time stamp and a SHA-256 fingerprint of that file (
414df36a042bcc5b1afc559f29dd0ee75c1799c7cd29a326d139bac85208a6e5) and are publishing it here, so any later edit to the expected answers would be detectable. - Fixed the scoring rules. Before scoring, we wrote down how the checklist’s three bands map to those four categories and how to handle gaps. The main gap rule: if a card doesn’t state a fact the checklist asks about, that item scores 0 and gets logged as “unknown.”
- Scored every card twice. A strict pass followed the rule above. A generous pass rescored every “unknown” item at the full 2 points, to see whether a verdict depended on missing facts or on real problems. Red flags stayed the same in both passes.
- Compared the checklist’s verdict to the expected answer, and checked each of the 35 risks: caught, partly caught, or missed.
The checklist’s bands, for reference: 16–20 “may be ready for a small, controlled pilot if there are no red flags,” 10–15 “fix the weakest areas before buying or integrating a new tool,” 0–9 “the process probably needs operational cleanup before AI will help.” Any red flag means stop and fix before connecting a tool to live work.
The 12 workflows and what the checklist said
| # | Workflow (invented) | Our expected answer | Score: strict / generous | Checklist verdict: strict / generous | Red flags | Result |
|---|---|---|---|---|---|---|
| S01 | Plumbing company: summarize web inquiries for the dispatcher | Pilot now | 11 / 17 | Fix first / Pilot | — | Too cautious (strict); match (generous) |
| S02 | Accounting firm: missing-document reminders on a fixed schedule | Use rules, not AI | 10 / 16 | Fix first / Pilot | — | Miss: no "use rules" outcome; generous reading says Pilot |
| S03 | Dental office: website chatbot answers symptom questions and books | Keep a person in charge | 2 / 10 | Clean up first / Fix first | RF2, RF3 | Stopped (item 6 = 0 + red flag), but framed as "not yet" |
| S04 | Online shop: AI drafts customer email replies, owner approves | Pilot now | 13 / 15 | Fix first / Fix first | — | Too cautious |
| S05 | Landscaper: "use AI for marketing" | Fix first | 2 / 8 | Clean up first / Clean up first | RF1, RF4 | Match |
| S06 | Property manager: AI triages maintenance requests after hours | Fix first | 7 / 13 | Clean up first / Fix first | RF2, RF3 | Match |
| S07 | Solo attorney: AI drafts engagement letters, attorney signs | Fix first | 12 / 14 | Fix first / Fix first | RF2 | Match |
| S08 | Agency: meeting recordings become action items | Fix first | 8 / 14 | Clean up first / Fix first | RF2 | Match |
| S09 | Bakery: AI cleans customer list, sends promos automatically | Fix first | 4 / 12 | Clean up first / Fix first | — | Match |
| S10 | HVAC company: AI prices jobs from photos, sends quote | Keep a person in charge | 8 / 14 | Clean up first / Fix first | RF3 | Stopped (item 6 = 0 + red flag), but framed as "not yet" |
| S11 | Recruiter: AI ranks applicants, auto-rejects bottom 60% | Keep a person in charge | 7 / 15 | Clean up first / Fix first | RF2, RF3 | Stopped (item 6 = 0 + red flag), but framed as "not yet" |
| S12 | Cleaning company: weekly schedule texts from a spreadsheet | Use rules, not AI | 8 / 16 | Clean up first / Pilot | — | Miss: no "use rules" outcome; generous reading says Pilot |
Red flags: RF1 = nobody can explain a correct output; RF2 = sensitive information and nobody has reviewed how the vendor handles it; RF3 = consequential decisions without meaningful human review; RF4 = no one owns the workflow after setup.
Of the 35 specific risks we expected a good checklist to raise, it caught 16, partly caught 11, and missed 8. The full item-by-item scores are at the end of this article.
What the checklist got right
It is hard to sneak a dangerous project past it. The dental chatbot (S03), the photo-based quoting tool (S10), and the auto-rejecting applicant screen (S11) all scored 0 on “the consequence of a wrong answer is understood,” because the output directly touches health, money, or employment. All three also tripped the red flag for consequential decisions without human review. That held in the generous pass too: even with every unknown filled in at full marks, none of them reached the pilot band, and their red flags still said stop.
It doesn’t let a vague wish through. “Use AI for marketing” (S05) scored 2 out of 20, with red flags for no definable correct output and no owner. That’s the right answer. There’s no workflow there yet.
It catches the data problem people skip. The solo attorney (S07) planned to paste confidential client details into a consumer chatbot without reading its data terms. Everything else about that workflow was in decent shape, and the checklist still flagged it. The bakery (S09) planned to send promotions automatically with no review, and the “human review is a real step” item scored it 0.
Where it broke
1. It has no way to say “you don’t need AI”
The accounting firm’s reminders (S02) run on a fixed schedule from data that’s already structured in its practice software. The cleaning company’s schedule texts (S12) are copy-and-paste from a spreadsheet. Neither involves judgment. The right answer for both is a normal automation, or for S12, arguably leaving it alone.
The checklist only has three outcomes: pilot, fix first, or clean up first. All three assume AI is where you’re headed. On the strict pass, it told both businesses to fix or clean up their process “before AI will help.” On the generous pass, both scored 16, the band where a workflow “may be ready for a small, controlled pilot.”
To be fair to the checklist, this site’s own reading order puts it second. What to automate first and AI vs. automation both tell you to use a normal automation when the rule is predictable. But plenty of readers land on a checklist directly. This one links back to that guide for readers who haven’t picked a task yet, not for readers who have picked one that doesn’t need AI. Five of the eight risks it missed were some version of “this part doesn’t need AI” or “a tool you already have does this.”
2. It never asks whether the job is worth it
The cleaning company’s task takes about 15 minutes a week. The attorney drafts about six letters a month. Item 1 asks you to say how often the task happens, but nothing in the 10 items or 7 red flags scores whether that is enough to matter. A clearly described tiny task gets the same 2 points as a clearly described big one. The checklist’s four-week pilot section does mention you may learn “the task does not happen often enough to justify another subscription,” but that comes after the pilot, not before it.
3. “Not yet” and “not this” get the same answer
For the dental chatbot, the HVAC quotes, and the applicant screen, the checklist’s wording is “the process probably needs operational cleanup before AI will help” or “fix the weakest areas.” That reads as not yet. For an AI that gives medical guidance, sends binding prices, or rejects job applicants with nobody checking, the more useful answer is not this design. A person needs to stay in charge, and the AI, if any, should prepare a draft for them.
The checklist’s text does say a 0 on the consequence question is “a red flag for a first project.” The score bands don’t carry that through.
4. It was stricter than we were on the good candidates
We expected two workflows to be ready for a small pilot: the plumber’s inquiry summaries (S01) and the online shop’s draft replies (S04). The checklist agreed on S01 only in the generous pass. S04 scored 15 even in the generous pass, one point short of the pilot band. Looking at its 1-point items (review criteria not defined, the helpdesk add-on’s data terms not read, reply exceptions not written down), each was fair. We count this as the checklist being stricter than we were, and arguably right.
5. Some questions can’t be answered when you need the answer
- Pilot boundaries (item 9) was unknown for all 12 workflows. That’s partly our card design. But it’s also built in: start and end dates, case limits, and a spending cap are things you write after deciding to pilot. Scoring a workflow down for not having them yet is circular.
- Examples of good and bad outcomes (item 3) and team understanding (item 10) were unknown for 10 of 12. Most owners won’t have these until someone asks. Between the strict and generous passes, 9 of the 12 workflows moved to a different band. The red flags didn’t change, so the risky ones stayed stopped. But the gap between “fix first” and “pilot” often came down to facts nobody had written down.
6. Three questions were ambiguous in practice
- Consequence (item 6): For the attorney’s engagement letters, the output carries legal obligations, which reads as a 0. But a qualified attorney reviews and signs every letter, which reads as a 1. The checklist doesn’t say whether to score the risk before or after review. We scored it 1 and logged the ambiguity.
- “Sensitive information” (red flag 2) isn’t defined. We had to write our own definition before scoring (health, financial account, legal-confidential, employment, images from inside homes, recordings of people). Four workflows involved only ordinary customer details such as names, phone numbers, emails, and addresses, and we couldn’t tell from the checklist whether those count.
- Consent isn’t named. The agency (S08) hadn’t settled whether clients know their meetings are recorded. The bakery (S09) didn’t know who on its list had agreed to hear from it. The checklist asks about vendor terms and “permissions,” but not whether the people in the data know and agreed. We counted both as partly caught.
What we’re proposing to change
These are proposals based on this test. They haven’t been tested themselves yet, and the checklist article won’t change until they’re reviewed.
- Add two questions before the score. Does any step need judgment, or do fixed rules cover it? If rules cover it, use a normal automation and stop here. Is it worth it? Roughly how often does it happen, and how many minutes does each one take? If the answer is a few minutes a week, stop here too. If it’s close, the ROI guide has the math.
- Add a fourth outcome: “not this workflow.” If item 6 scores 0 and the AI’s output would reach a customer, set a price, or decide something about a person without review, redesign it so the AI drafts and a person decides.
- Clarify item 6. Score what happens if the reviewer misses the mistake.
- Define sensitive data with examples, and link to our guide on what data to put into an AI tool.
- Add consent to item 7: the people whose information, voice, or image is in the workflow know, and have agreed where that’s required.
- Move pilot boundaries (item 9) out of the readiness score and into the one-page pilot brief, where it already appears as “Limits.”
How to use the checklist now
Until those changes are made, here’s how we’d use it:
- Before scoring, ask two questions. Does this need judgment, or do rules cover it? Is it frequent and time-consuming enough to matter? If either answer is no, you’re done. Use a rule, or leave it.
- Read the red flags before the total. A single red flag outweighs a good score.
- If item 6 is 0, change the design, not the timeline. Have the AI prepare something for a person to decide on. Don’t wait for the process to “mature” into letting the AI decide.
- Mark unknowns as unknowns. If you can’t answer an item, write “unknown” rather than guessing. Then go find out. Ten examples of past good work and a 10-minute conversation with the person who does the job will fill in most gaps.
- Score after you’ve drafted the pilot brief, not before, if you want item 9 to mean anything.
Try it on your own business. Pick three workflows you’ve thought about automating. Before you score them, write one line each: pilot now, fix first, use rules, or keep a person in charge. Then score them. Where the checklist disagrees with you, one of you is missing something. It’s worth finding out which.
What this test does not show
- It’s a desk test of a scorecard, not a test of AI. No AI tool was run on any workflow. Nothing here says whether summarizing, drafting, or triage works.
- The workflows are invented. They were designed to cover a spread of situations, not to represent how common each one is. Don’t read “5 of 12” as a rate.
- The same assistant wrote the cards, the expected answers, and the scores. We reduced that bias where we could: the expected answers were written before the scoring criteria were opened during the test (though the assistant had read the checklist in an earlier working session, so this was not fully blind), the rules were fixed before scoring, and every item-level score and reason is published. It’s still one scorer. A different person could reasonably score some items a point differently.
- The cards left out facts the checklist asks about. That’s realistic, since owners rarely have all of this written down, but it inflated the “unknown” counts.
- The proposed changes are untested. Testing them fairly needs a new set of workflows, not these 12.
Appendix: item-by-item scores (strict pass)
A “?” means the card didn’t state the fact, so the item scored 0 under the strict rule (2 in the generous pass).
| # | 1 Problem | 2 Process | 3 Examples | 4 Owner | 5 Baseline | 6 Consequence | 7 Data | 8 Review | 9 Boundaries | 10 Team | Strict total |
|---|---|---|---|---|---|---|---|---|---|---|---|
| S01 | 2 | 1 | 0? | 2 | 2 | 2 | 1 | 1 | 0? | 0? | 11 |
| S02 | 2 | 2 | 0? | 2 | 1 | 1 | 1 | 1 | 0? | 0? | 10 |
| S03 | 1 | 0? | 0? | 1 | 0 | 0 | 0 | 0 | 0? | 0? | 2 |
| S04 | 2 | 1 | 1 | 2 | 2 | 1 | 1 | 1 | 0? | 2 | 13 |
| S05 | 0 | 0 | 0? | 0 | 0 | 2 | 0 | 0 | 0? | 0? | 2 |
| S06 | 2 | 1 | 0? | 2 | 1 | 0 | 1 | 0 | 0? | 0? | 7 |
| S07 | 2 | 1 | 1 | 2 | 1 | 1 | 0 | 2 | 0? | 2 | 12 |
| S08 | 2 | 1 | 0? | 2 | 0 | 1 | 1 | 1 | 0? | 0? | 8 |
| S09 | 1 | 0? | 0? | 2 | 0 | 0 | 1 | 0 | 0? | 0? | 4 |
| S10 | 2 | 1 | 0? | 2 | 2 | 0 | 1 | 0 | 0? | 0? | 8 |
| S11 | 2 | 0? | 0? | 2 | 2 | 0 | 1 | 0 | 0? | 0? | 7 |
| S12 | 2 | 1 | 0? | 2 | 1 | 1 | 1 | 0? | 0? | 0? | 8 |
One oddity worth noting: the landscaper’s “use AI for marketing” (S05) got full marks on the consequence question, because the card says the stakes are low. A workflow that doesn’t exist yet probably shouldn’t score well on anything.
The 12 scenario cards and the expected answers, exactly as written before scoring, are below.
Appendix B: the 12 scenario cards, as written before scoring
Each card is invented. "Expected" is the answer we wrote down before scoring; "Should-catch" lists the risks we expected a good checklist to raise. Wording is unchanged from the original file except for formatting.
Show all 12 scenario cards
Each card is invented. “Expected” is the answer we wrote down before scoring; “Should-catch” lists the risks we expected a good checklist to raise. Wording is unchanged from the original file except for formatting.
S01. Plumbing company, 6 staff. Workflow: summarize inbound web-form inquiries into job type + urgency + missing info, for the dispatcher. ~40 inquiries/week. Current: dispatcher reads each form and phones back; steps are written in a one-page SOP. Data: name, phone, address, free-text problem description; forms land in a shared inbox and are copied into the job system. Stakes: a wrong summary delays a callback; dispatcher still reads the original. Owner: office manager, has 2 hrs/week for this. Review: dispatcher sees every summary next to the original form. Tools: already pays for a business email suite with an AI assistant included. Measurement: office manager logged time-to-callback for 2 weeks (baseline exists). Budget: none beyond existing subscription. - Expected: Pilot now (A). Should-catch: minimize personal data (address/phone not needed for summary); keep original visible; define what “urgent” means.
S02. Accounting firm, 4 staff. Workflow: send clients reminders for missing tax documents on a fixed schedule (14, 7, 2 days before deadline). ~150 clients in season. Current: admin sends manually from a checklist; process documented. Data: client name, email, list of outstanding docs, all already structured in the practice-management software, which has a built-in reminder feature not yet switched on. Stakes: moderate (late filings), low per message. Owner: admin. Review: admin spot-checks. Measurement: admin knows it takes ~4 hrs/week in season (estimate, not logged). - Expected: Use rules, not AI (C). Should-catch: the job is fully rule-based on structured data; the existing tool’s reminder feature does it; AI adds cost and risk for nothing.
S03. Dental office, 3 dentists. Workflow: website chatbot that answers patient questions about symptoms (“is this swelling an emergency?”) and books appointments. Volume unknown. Current: front desk answers phones. Data: patient health information typed by patients. Stakes: high (clinical advice, health privacy). Owner: practice manager, no time allotted. Review: none planned; chatbot answers live. Tools: vendor demo seen; no contract. Measurement: none. - Expected: Keep a person in charge (D) (for symptom answering). Booking alone could be a standard scheduler (C). Should-catch: clinical advice by AI with no human; sensitive health data; no review; no owner time; no baseline.
S04. Online shop, 2 people, ~300 orders/month. Workflow: AI drafts replies to customer emails (order status, returns, sizing) for owner approval before sending. Current: owner answers all email, ~1 hr/day; a written returns policy and FAQ page exist. Data: order number, name, email, order status from the store platform. Stakes: low–moderate (a wrong promise on refunds). Owner: the owner. Review: every draft approved by owner. Tools: store platform + helpdesk with optional AI add-on. Measurement: owner tracked replies/day for one week. - Expected: Pilot now (A). Should-catch: many “where’s my order” replies are rules (tracking link template) — use AI only for the rest; refunds/exceptions stay human; policy doc is the source of truth.
S05. Landscaping company, 12 staff. Workflow: “use AI for marketing.” No specific task defined. Current: owner posts on social media occasionally. Data: none identified. Stakes: low. Owner: nobody assigned. Review: undefined. Tools: none chosen. Measurement: none. - Expected: Fix first (B). Should-catch: no defined workflow, no owner, no baseline, no success measure. Not a readiness question yet.
S06. Property manager, 120 units. Workflow: AI triages tenant maintenance requests (text + photos) by urgency and assigns vendor; runs after hours. Current: on-call manager reads requests; emergency line exists. Data: tenant name, unit, description, photos inside homes. Stakes: high for emergencies (gas, flooding, no heat in winter); low for routine. Owner: operations manager. Review: none after hours (the point is to remove the on-call read). Tools: property-management software with request portal. Measurement: request counts known; response times not tracked. - Expected: Fix first (B). Should-catch: safety-critical routing cannot depend on an AI classification alone — need a rules-based emergency path (keywords/phone) first; no after-hours human review; no baseline response time; photos inside homes are sensitive.
S07. Solo attorney. Workflow: AI drafts first-pass engagement letters from intake notes; attorney edits and signs every letter. ~6/month. Current: attorney adapts a template, ~45 min each. Data: client names, matter details (confidential). Stakes: high per document but every letter reviewed by attorney. Owner: attorney. Tools: considering a general chatbot on a consumer plan; has not read its data terms. Measurement: time per letter is known roughly. - Expected: Fix first (B). Should-catch: confidential client data into a consumer tool with unread data terms; low volume (6/month) means small value — a better template may beat AI.
S08. Marketing agency, 10 staff. Workflow: meeting recordings → transcript → action items posted to the project tool, reviewed by the account manager. ~25 client meetings/week. Current: account managers type notes; quality varies; follow-ups get missed. Data: client meeting audio. Stakes: moderate (missed or wrong tasks; client confidentiality). Owner: operations lead. Review: account manager approves tasks before posting. Tools: video-meeting platform has AI notes included. Measurement: none, but missed follow-ups are a known complaint. Consent: whether clients are told meetings are recorded is not settled. - Expected: Fix first (B). Should-catch: recording consent/notice must be settled first; no baseline. Otherwise a good candidate.
S09. Bakery with café, 9 staff. Workflow: AI to “clean up” the customer list (duplicates, typos) and then write and send monthly promo emails automatically. Current: customer list exported from POS, ~2,000 rows, many duplicates, mixed opt-in status. Data: names, emails, purchase history; opt-in status unclear for many. Stakes: moderate (spam complaints, sending to people who did not opt in). Owner: café manager. Review: none planned for sends. Tools: email-marketing tool with dedupe feature. Measurement: none. - Expected: Fix first (B). Should-catch: unclear consent/opt-in must be fixed before any sends; dedupe is a standard feature (C) not AI; auto-sending marketing with no review.
S10. HVAC company, 15 staff. Workflow: AI estimates job prices from customer photos and sends the quote straight to the customer. ~60 quote requests/week. Current: estimator visits or calls; pricing sheet exists but many jobs need a site visit. Data: photos, address. Stakes: high (binding price promises, margin, customer trust). Owner: estimator. Review: none (the point is speed). Tools: vendor pitched it. Measurement: quote turnaround known (~2 days). - Expected: Keep a person in charge (D) (as designed: AI sending prices to customers unreviewed). An internal draft estimate for the estimator to review would be a different, B-level workflow. Should-catch: customer-facing commitments without review; accuracy unverifiable from photos; high cost of error.
S11. Recruiting firm, 5 recruiters. Workflow: AI ranks applicants and automatically rejects the bottom 60%. ~400 applications/week. Current: recruiters skim resumes. Data: resumes (personal data). Stakes: high (decisions about people; fairness and legal exposure). Owner: managing partner. Review: none for rejections. Tools: applicant-tracking system with AI ranking add-on. Measurement: time-to-shortlist known. - Expected: Keep a person in charge (D). Should-catch: automated adverse decisions about people without human review; fairness/legal risk; can’t easily verify errors (rejected people disappear).
S12. Cleaning company, 8 staff. Workflow: send each cleaner their weekly schedule by text, copied from a spreadsheet. Takes ~15 min/week. Current: owner copies and pastes. Data: names, phone numbers, addresses of client homes. Stakes: low–moderate (wrong address). Owner: owner. Tools: spreadsheet + phone. Measurement: 15 min/week (owner estimate). - Expected: Use rules, not AI (C) (simple scheduling-app or spreadsheet-to-SMS automation, or leave as is: 15 min/week is little value). Should-catch: low value; no judgment needed.
Related guides
- The Small-Business AI Readiness Checklist: the scorecard tested here.
- 5 Tasks Your Small Business Should Automate Before Anything Else: start here if you haven’t picked a workflow.
- AI vs. Automation: how to tell whether a step needs AI at all.
- What Data Should You Put Into an AI Tool?: for the sensitive-data question.
- How to Evaluate an AI Pilot: what to do once a workflow does pass.
Want a broader starting point? Take the free two-minute AI ReadyScore assessment, then use the checklist on one specific workflow.