They wouldn't do it on a whim, and if it did happen, it would likely only be a temporary effect. [Which is probably putting it very lightly, but it's an attempt to reassure him. Somehow.] I've lost my memories before... but they've all come back, as well.