October 3, 2026 · Ahmed · More by Ahmed
What Shopify's AI Self-Review Actually Checks
Shopify's AI self-review checks App Store code requirements before you submit — but listing content, live behavior, and UX still need a human reviewer.
A Self-Review Before You Ever Submit
Shopify now ships an AI-powered self-review as part of its app review tooling. After installing the Shopify AI Toolkit, a developer can run it from an agent chat with a single command: /shopify-app-store-review. It reads the App Store requirements, inspects the local project, and returns a report sorted into likely-passing, likely-failing, and needs-review items.
The same check also appears as a step in the pre-submission checklist on the App Store review page, and it isn't a one-time gate. Shopify's documentation says it can be run any time during development, not just right before submitting.
What It Can and Can't See
The useful framing here is scope, not accuracy. Shopify is explicit that the self-review only covers requirements that can be checked against code. Anything that depends on app listing content, live app behavior, or merchant-facing UX falls outside what it inspects, and the human review team still verifies all of that after submission. A clean self-review result means the code satisfies the checkable requirements — not that the app is ready to pass review outright.
That distinction matters because several of Shopify's own documented common review-failure categories are behavioral rather than structural: an app that fails to redirect to the OAuth grant screen on install, one that switches between embedded and non-embedded views, a user interface that's broken immediately after install, or a submission missing testing credentials. None of those surface from reading source code alone — they only show up when someone actually installs and runs the app.
Review Feedback Moved Into the Partner Dashboard
Alongside the self-review, Shopify restructured where human review feedback lives. Feedback for a submission now appears in the Partner Dashboard under App > Distribution, with its own status tracker and reviewer comments per requirement. Developers address each flagged requirement individually and mark it resolved, and resubmission is blocked entirely until every identified issue is cleared. Shopify also expanded its automated pre-submission checks to flag theme app extension and App Store Listing requirements before a human reviewer ever picks up the app.
There's one carve-out: apps already sitting in Reviewed status with outstanding issues at the time of the change continue through the prior email-based process rather than moving into the new dashboard flow. Apps already Submitted and waiting don't need to do anything — their feedback lands in the new dashboard once a reviewer picks them up.
For teams submitting apps regularly, the practical takeaway is to treat the self-review as a fast first pass that narrows what a human reviewer needs to catch, not as a substitute for actually installing and clicking through the app the way a merchant would.