I think part of the cause for concern (unfounded or not) is the Electoral commission's wording is if you think you are on the roll you can cast a special, but it will only count if you are [present tense] on the roll, which on the surface doesn't seem to cover people who used to be on.
elections.nz