What “tested” means here
A release is ready only when:- every automated test passes in all four setups we support, on MySQL and on MariaDB, and with a persistent object cache,
- every release check passes,
- the independent security, QA, performance and code reviews have no open release-blocking findings.
How a change gets to a release
1
Design review
Before building, the change is planned, and security, QA and performance plans are written for it.
2
Build
The change ships with its own tests. Every power-up has its own test file.
3
Independent reviews
Separate security, QA, performance and code reviews look at the change. Anything that touches customer data, orders, permissions, deletion, exports or account switching always gets a security and a QA review.
4
Fixes re-checked
Findings are fixed, then checked again by the review that raised them.
5
Release checks
All automated tests and release checks run on the final code. The result is “ready” or “not ready”. Nothing ships as ready while a check is failing.
Automated tests
The 1.0 release has more than 1,800 automated PHPUnit tests. They run against real WordPress and WooCommerce, in four setups:
Tests that need High-performance order storage, such as Advanced order search, are skipped in legacy mode with a reason.
The suite also runs on MariaDB, and with a Redis persistent object cache. Both give the same results as on MySQL.
PHP and database versions
For 1.0.0, with WordPress 7.1 and WooCommerce 11.1, the full suite passed on each of these versions:
On PHP 7.4 and 8.0, WordPress refuses to activate the plugin. If a host later moves a site below PHP 8.1, the plugin shows a notice and stops loading. Its data is kept. See Requirements.
What the tests cover
- each power-up’s rules, settings, tools, emails and background jobs
- power-ups used together: checkout and email combinations, the order lifecycle, and activating, deactivating and updating the plugin
- both checkouts: anything that changes checkout is tested in the classic checkout and the block checkout
- settings stored in an unexpected type, which every settings form must still handle
- power-ups that are off, which must leave no hooks behind
Security tests
Separate suites also try to break things on purpose:- A permission matrix calls every REST, AJAX and admin-post endpoint as a logged-out visitor, a customer and staff. New REST routes are picked up automatically.
- Input fuzzing sends odd input to endpoints and settings forms, such as quotes, HTML, emoji, very long text and huge numbers.
- The order and customer search types are tested with SQL injection attempts.
- Output is checked for escaping, and masked order meta must never appear in the page.
- Switch to customer is tested for permissions, privilege escalation, tampering, password changes and the full switch-and-back cycle with real sessions. See Switch to customer.
- Adversarial suites try to misuse every Developer and Performance tool, Quick stock, Analytics order filters and the payment and shipping rules.
- Absurd page numbers and amounts too large to handle are tried on paged screens and in the order and customer searches.
Real-browser tests
117 end-to-end tests drive a real WooCommerce site in a browser with Playwright. They cover the Powerups screen, settings, payment and shipping rules, block checkout, the checkout blocklist, holiday mode, the Orders and Users lists, emails, the order screen, switching to a customer, the customer tools, Quick stock, Analytics order filters, and the Developer and Performance tools. The tests snapshot the site’s power-up settings before they start and restore them exactly afterwards. Everything they create is marked and removed. If the settings don’t match the snapshot, the run fails.Code quality and security checks
Every release runs these checks:Performance
We measure performance with a probe that counts the plugin’s own database queries on each page. Results are medians of repeated warm runs, with no persistent object cache.Your storefront
With every power-up on, the plugin adds 0 database queries to the front end and the Store API (the block cart and checkout). Page time and memory stayed within run-to-run noise.wp-admin
- Most admin pages: 1 to 4 plugin queries. The settings load in one query.
- Orders list: 1 query.
- Users list with every customer column on: 9 queries per page, however many users are listed.
A large store
We also test on a store with 100,000 orders, 50,000 customers and 10,000 products (plus 1,500 variations), using High-performance order storage. It also holds about 3.5 million order item details, 200,000 comments and 300,000 scheduled actions. The last run used MySQL 8.4, PHP 8.2 and a Redis object cache. Times are medians of five warm runs. They include the 0.3 to 0.4 seconds WordPress and WooCommerce take to load on each request.- Run cold, before the database had its tables in memory, the same order searches took 2 to 5.6 s.
- Database clean-up emptied every check, in steps that stop near their 5-second limit (the longest took 5.7 s). It deletes rows directly at 10,000 to 80,000 a second. Revisions, posts and comments go through WordPress, at about 140 to 1,000 a second.
- No request or step used more than 128 MB of memory. The most was 112 MB.
Fixes from these tests
Each row compares the code before and after a fix, on a store of the same size. The last three rows were measured inside WordPress, without the time it takes to load a page.
Settings that can still be slow on big stores carry a warning or a Slower badge. See Performance.
Privacy and your data
- Each power-up lists what it stores. The Data section on its page and the uninstall both read that list, and tests check every power-up’s list. See Data and uninstall.
- The plugin never deletes customer accounts, saved addresses, order statuses or order notes. Tests check uninstall with and without Keep data if the plugin is deleted.
- Order meta keys that look like tokens, passwords, secrets or card details are hidden in the order meta inspector unless you allow staff to reveal them. Staff who can’t reveal them can’t search them with Advanced order search either. Tests check that masked values never reach the page.
- Emails with a password or verification link (new account, password reset, email address confirmation and back-in-stock sign-up verification) are never stored with their content, copied to extra recipients, previewed, test-sent or offered for resend.