Seriously, tell them. You can be gentle about it, but they deserve to know why you won't be following them there.