Security awareness: measure the report rate, not the click rate
Almost every awareness programme steers by one number: what percentage clicks the fake mail. That number drops, flattens out around five per cent and then says nothing more. This is about the number that does keep moving.
The click rate measures how good your exercise was, not how resilient your organisation is. It falls to a floor of a few per cent and stays there, whatever else you do.
The report rate measures something else: how many people pass a suspicious message on, and how fast. That number decides how much time you have when it is real.
An exercise that ends in a list of names works against you. Someone afraid of being wrong reports nothing, and then you lose the very signal you are steering by.
What it is, and what it is not
Security awareness is not a forty-minute e-learning everyone rushes through in December. That is the shape it is usually bought in, and it is also why it is so often experienced as a box to tick.
What it is actually about is narrower and more concrete: does someone recognise a message that is off, and do they know what to do with it. That second part is almost always forgotten. People who do spot it but do not know where to take it do nothing, and from where you sit that is indistinguishable from someone who never noticed.
The subject is also broader than phishing alone: what you do with data, what you do and do not confirm over the phone, what happens when someone loses a laptop. But phishing is where it starts, because that is where most of it comes in.
Why the click rate is the wrong number
Almost every programme reports the click rate, and in the first year it looks good: from twenty per cent to eight, to five. After that nothing happens.
That is not a plateau in resilience but in the exercise. The number mostly tells you how recognisable your fake mail was. Make it harder and the percentage rises again; make it easier and it falls. You are measuring your own test design.
There is an unpleasant assumption underneath it too: that zero is achievable. It is not. Someone processing fifty invoices a day will click one eventually, and a defence that leans on that never happening is not a defence but a hope.
The report rate does measure something that moves. How many recipients pass the message on, and how long that takes. That number tells you how much time sits between the first click and the moment someone intervenes, and that time is the whole difference between an incident and a morning.
The two are also connected in a way the click rate cannot show: in an organisation where reporting is normal, the person who just clicked reports it themselves. That is the fastest report there is, and you only get it when there is no penalty attached.
What an exercise should and should not do
Reporting has to be one action. A button in the mail client, not an address to remember and not a form on the intranet. Every extra step costs reports, and the people who drop out are exactly the ones who hesitated, so the reports you need most.
Someone has to look at them. A report button with nothing behind it works once. Anyone who reports three times and never hears whether it was anything does not report a fourth. This is the quiet way such a programme bleeds out, and it shows in the figures before it shows in the people.
No lists of names. The temptation is real, because the reporting makes it easy. But anyone afraid of ending up on such a list stops reporting that they might have got something wrong, and with that you lose precisely the signal the programme runs on.
Exercise more often and more briefly. One campaign a year teaches nobody anything; a short exercise each quarter keeps the subject alive without becoming an irritation.
What we do here, and what we do not
What we do. Set up and run the platform, for us that is Cofense: the rehearsed messages, the report button in the mail client, and the route a report takes to someone who can act on it. We make sure that chain stands and that the figures above come out of it.
What we do not do. The course material is the vendor’s and not ours; we are not a training provider. Personnel policy is yours: we do not decide what happens to someone who clicks, and we strongly advise against tying that to figures. And we issue no declaration that your organisation is "awareness-compliant": no such thing exists, and where a standard asks for awareness, your own records demonstrate it, not our word.
Questions we get about this
The four that come back most often.
Does awareness training actually work?
Partly, and that is not a dismissal. Training lowers the number of clicks but never to zero, so a setup that leans on nobody clicking will come unstuck eventually. What training does do is raise the chance that someone reports it, and that is where the return sits. See also our piece on phishing, which covers the technical side: the controls that survive a click.
How often should we run an exercise?
Rather short every quarter than large once a year. An annual campaign is an event that passes; a short exercise each quarter keeps the subject present. Vary the difficulty while you are at it, or you end up measuring only whether people recognise your house style of fake mail.
Should we confront people who click?
We advise against it and we do not decide it; that is personnel policy and it is yours. What we can say is what it costs: as soon as clicking has consequences the report rate drops, and the person who just clicked is the one whose report you need most. A repeat clicker is a reason for a conversation about their work, not for a measure.
Do ISO 27001 or NIS2 require this?
Yes, awareness appears in both. What they ask is that it happens and that you can demonstrate it, not a particular percentage. Your own records are the evidence; we supply the figures they consist of. We certify nobody; that is done by an accredited certification body.
Further reading
The technical side, and where this lands in a standard.
Shall we look at what you measure today?
Tell us what is running now and which figures come out of it. We will say whether you are steering by the click rate or by something that gets you further, and what it takes to see that second one.
Practical IT knowledge in your inbox
New guides on management, security and the workplace, written by the people doing the work. No sales talk, and you can unsubscribe in one click.