I’ve started tracking the outcomes out of the Polaris–>Vega LX Starter email notices process more closely in an attempt to pin down rashes of failures we’ve been seeing. As part of that, I’ve started to measure (for the first time ever) the overall ‘daily success rate’ of our email notices. Even on the days I see the problem I’m tracking, the rate is 98% or above.
I’ve been trying to figure out what meaningful targets are for notices that are “working fine.” We have automatic renewals turned on in Polaris, and something like 75% of all the notices we send out are almost due/autorenew notices. If every single hold notice were to fail one one day, our overall delivery rate would still probably be in the 98% bracket, so the top-line number feels like it’s not all that meaningful. But at the same time, tracking several different categories of notices each day feels like overkill.
Can anyone share how they determine if their notices are working “well enough”? What measurements do you use?