All Quack and No Bite: The problem with NZ Govt’s accessibility measurement system
NZ Government has a world-leading web accessibility measurement system, but the government is failing to use the data to create change. In this post, I outline what steps must be taken to complete the system.
CWAC: Where it’s at now
During my time in NZ Government, I led the creation of the Centralised Web Accessibility Checker (CWAC), an ambitious duck-themed programme to automatically measure web accessibility across the entire government.
In mid 2025, the Government Chief Digital Officer (GCDO) began publicly releasing 3-monthly accessibility data for practically all public-facing government websites. It scans a maximum 100-page sample of each website, using automated accessibility tests like axe-core, Flesch-Kincaid readability metrics, and other custom tests.
I then developed public loserboards based on this data:
The GCDO also released their own leaderboards, however they’re not very well designed, and they utilise a complex scoring system that makes it impossible for people to understand what the numbers actually represent. Additionally, they lack a way to see changes in compliance over time — so they cannot be used to track progress.
While I have criticisms of their leaderboard, overall, I am so impressed with how the project has been brought to completion — it’s honestly a huge achievement and everyone involved should be proud.
Now a few months have gone by since this data was released, and while it has received a decent amount of attention, it has not had the level of impact I hoped for.
We have a beautifully implemented measurement system now in Aotearoa. The data is flowing through the pipes — but now we need to boost the pressure. It should be like a firehose aimed directly in the face of any individual standing in the way of fixing accessibility barriers. Make that data impossible to hide from.
I recently spoke with Louise Upston, the Minister for Disability Issues. Louise Upston was aware of the recent CWAC data — so we know it has ministerial-level attention. Upston even thanked me in her speech for my work in building CWAC — that was a very nice moment to experience. I know the Public Service Commissioner has been briefed on the CWAC data. I know Paula Tesoriero, the Chief Executive of the Ministry of Disabled People, knows about the data. I have also seen CWAC mentioned in a handful of agency annual reports.
So, it’s certainly reached fairly high levels in government, but that’s not enough on its own.
People inside and outside of government have been grumbling for decades about the terrible level of web accessibility compliance across government. New Zealand’s public service, in my view, often gets caught in nihilistic pessimism spirals — continually complaining about systemic problems, rather than actually doing something to fix it. Or, the same interventions that have been running for decades are relied on, where the sands of time have clearly proven they were not effective system-level fixes to begin with. We need more people with the courage to actually throw spaghetti at the government wall to see what sticks.
The Web Standards, while essential as a standard-defining mechanism, are not enough on their own. A “Cabinet mandate” does not work. We have 2 decades of evidence to prove this. Education isn’t enough either. And nor is system-wide automated test data. A real fix to the accessibility crisis will be a multi-pronged strategy; a patchwork of partial solutions that complement each other. One of my past managers taught me to make a “pincer movement”, like a crab claw — if you want something in the public service to happen, attack it from multiple angles simultaneously — it can wake a slow system. 🦀
New Zealand Government does not publish any cohesive digital accessibility strategy. No strategy exists. Ideas and tasks related to digital accessibility are worked on in isolation, without any public view of what the overall future state will be. We see some communications about the future of the Digital Accessibility Standard, however, the Standard is not a strategy — it is just one small aspect of the broader accessibility system.
I find it somewhat odd that the GCDO, which continually pumps out various digital strategies, does not have one for accessibility. Whaikaha and the GCDO should maybe think about creating one.
So what’s the problem with CWAC?
Well, as this post’s title suggests, the current CWAC monitoring programme is all quack and no bite.
A large chunk of the CWAC strategy is currently missing.
Here’s the core of it:
- there are no clear targets to reduce accessibility issue counts over time
- agencies are not being held accountable for their accessibility failures, even though we now have clear monitoring data to prove their failures.
Without targets and accountability mechanisms, the data is being sent into the bureaucratic void; straight into the memory hole. All the investment in gathering the data will go to waste.
The CWAC strategic layer cake
I view the CWAC strategy as having multiple layers, each one building on top of the one before it.
It’s clear some layers are totally absent — and unfortunately, the layers that are missing, are the most important for creating change.
Here’s how I view the layers:
- Layer 1: data layer — this layer is the actual CWAC application, raw accessibility data collection, and data publication.
- Layer 2: presentation layer — this layer is about taking the raw data, and presenting it in ways people can understand e.g. the loserboards.
- Layer 3: accountability layer — and this is what’s currently missing — a layer that takes the collected data, and uses multiple simultaneous mechanisms to ensure the data causes improvement across the system. It may use a combination of new, and pre-existing governmental accountability processes.
Now, Layer 3 — the accountability system — isn’t totally absent, it’s just… almost absent. The public loserboards do create a system of accountability through public shaming of failure; however, it’s not been hugely effective on its own — it definitely needs beefing up.
So, what should accountability look like?
Accountability in New Zealand’s public service is a difficult thing to achieve. In my view, New Zealand’s public service has a confusing and bamboozling accountability system that makes problems like accessibility very hard to solve.
There’s no single system I can add a tick-box to, that will totally solve the accountability problem. Instead, we have to come up with a patchwork of smaller, less effective methods to achieve change. Linking systems together that were previously disconnected.
So, what levers are available to us? And, what combination do we use?
A list of levers
Below is a massive list of ideas I’ve come up with to achieve accountability.
- tie chief executive performance pay to accessibility data
- ensure the Public Service Commissioner includes accessibility data in CE performance reviews
- issue an ‘accessibility failure penalty’ fine to each agency that has identified accessibility issues
- annually, we gather all Public Service chief executives on Parliament’s forecourt, and disabled people get to throw 1 duck egg at each CE for every accessibility issue discovered in CWAC data (I know who can supply the eggs)
- an annual public event to showcase CWAC compliance progress and failure (integrate into the Government Digital Accessibility Forum event)
- mandatory inclusion of CWAC data in annual reports
- Treasury could reject agency budget bids (new and existing) that lack a section demonstrating how digital accessibility is being addressed
- GCDO’s ICT assurance role should be utilised to interrupt agency investments if the agency is failing to demonstrate how digital accessibility is addressed
- change Marketplace policy to revoke registration of web suppliers who deliver non-compliant websites
- modify the Web Usability Standard to enforce a ‘compliance grade’ badge in all website footers
- require the Minister for Disability Issues to table CWAC data to Parliament quarterly
- automatically email all web teams and CEs with their CWAC performance each quarter
- establish a “Digital Accessibility Charter”, or modify the existing Accessibility Charter. Include a publicly-tracked target and commitment from each CE, to reduce average accessibility issue counts by ~20% each year.
- generate media awareness of compliance failure.
- get the CE of Whaikaha to chair an all-CEs meeting, supported by PSC, to address accessibility performance data once per year.
- modify the government procurement system to centralise the management of all web/digital development, and use this mechanism to ensure automated and manual accessibility testing is embedded into the procurement lifecycle.
So… what levers do we use?
The above list is pretty huge. One of the ideas is perhaps a bit too eggy.
But, do you know what the exciting thing is?
The solution to the web accessibility problem in NZ is probably sitting in that list somewhere. We just have to pick the right combination of levers and implement them simultaneously.
Proposed levers for Layer 3
- Reports: Ask Treasury to make year-on-year CWAC performance data a mandatory element of agency annual reports
- Badges: Modify the Web Usability Standard to require compliance grade badges on all website footers
- Notify: Implement multiple communications activities to broadcast CWAC data
- Procurement: Centralise web/app development procurement processes with the GCDO, ensuring accessibility assurance is embedded throughout the procurement lifecycle.
I’ve selected this set of 4 levers, because they’re all pretty simple to implement, and will likely have a high ROI. Except maybe the procurement lever — centralising all procurement of digital development could be a pain, but I think the GCDO is already going down this path — so it could actually be easy.
Layer 3, Lever 1: Reports
This lever aims to ensure all agencies include year-on-year CWAC data in their annual report documents.
A requirement must also exist to establish per-agency targets. Each agency should specify what level of CWAC compliance they expect to reach by the next annual report, with a brief summary of how they intend to meet the target.
Annual reports are a pre-existing governmental accountability system — it makes sense to simply plug in to pre-existing structures where possible. Annual reports are particularly valuable, as they create a linkage with Parliament — the ultimate entity of accountability in NZ Govt.
Intended impacts:
- CEs, DCEs, and comms staff are forced to approve the publication of their CWAC data, raising awareness of failure in the organisation
- Annual report content is likely to influence CE performance conversations with the Public Service Commission
- Ministers responsible for each agency (hopefully) read the annual reports, improving Ministerial oversight and accountability
- Annual reports are considered by Parliamentary select committees — it enables Parliament to hold each agency to account for its accessibility performance, particularly where the CE is being scrutinised by a select committee.
- The media is more likely to pick up on accessibility issues in government if it is in annual reports — this increases public accountability through the fourth estate.
I have orchestrated a small test to see if parts of this strategy will work. I have pulled some strings to ensure the GCDO will face Parliamentary scrutiny on DIA’s CWAC data soon. This will allow us to observe how Parliamentary scrutiny can be used to hold executives to account for their agency’s accessibility failures. It should hopefully send a signal through DIA’s hierarchy that Parliament is aware of their accessibility failures.
Layer 3, Lever 2: Badges
This lever aims to make each web team painfully aware of their accessibility failures by integrating a visible score into all their web pages. Web teams often sweep their accessibility failures under the rug — subjecting disabled people to unfair experiences that non-disabled people don’t notice. Badges on each website makes the problem visible to all.
The GCDO could develop a script that injects a small badge in each government website’s footer.
The script will query a central server to fetch the latest Web Standards compliance score for each website.
It could be a compliance grade like: A, B, C, F.
- A is reserved for websites with 0 CWAC-detected issues.
- B is for ~1 issue per page on average
- C is for ~2 issues per page on average
- F is for >=3 issues per page on average.
Intended impact:
- Web teams will be sad if their websites have a bad score on their badge. They will likely work to improve it.
- Web teams will face constant motivation to keep high scores, avoid regressions.
- Web team managers will feel pressure to improve and maintain badge scores.
- Senior leaders will also feel that pressure.
- Ministers may take notice of the badge and ask their CE to improve it.
- The media may report on poor performing, critical sites.
Layer 3, Lever 3: Notify
This lever is basically just intensifying communications activities to ensure everyone who needs to know about CWAC data, is kept informed.
Things like:
- Emailing all web teams quarterly with their CWAC performance data
- Embedding CWAC performance scores into a Government Digital Accessibility Forum presentation
- DIA making social media posts about the CWAC data, with performance insights
- Emailing all public sector CEs with their CWAC scores.
Layer 3, Lever 4: Procurement
This lever is an attempt to beef up procurement processes by centralising them. It’s actually not really related to CWAC data, but… thought I’d just throw this out there anyway.
Often web accessibility is not met by web suppliers. It’s caused by many agencies not realising they must include accessibility as a requirement for web development. Or, they don’t know how to check that what they’ve been supplied, is actually compliant with the standards.
By giving the GCDO more centralised control over digital procurement, minimum standards can be better enforced by a central procurement team.
DIA already administers Marketplace, a quasi-centralised procurement system. It could build on this infrastructure to help maintain high standards with suppliers.
Conclusion
Going back to that idea about disabled people throwing 1 egg at public service chief executives for each accessibility barrier identified in the data… Well the funny thing about that idea is, if implemented, it’d probably work pretty damn well. Perhaps we should give it a go and ask the CEs if they’d be keen to participate.
We have an awesome accessibility measurement system.
We just have to turn up the pressure a bit.
I will be working over the coming months to try and get the accountability system implemented.
Stay tuned 😎
— Callum