In terms of utilitarian ethics (and legal/practical point of view) I'd say he needs to be punished anyway. It sets a dangerous precedent if you allow people to escape punishment if they seem to have suffered memory loss. We can't really distinguish between actual memory loss and feigned memory loss that I'm aware of. The bigger picture begs for the individual to be punished regardless.
It's pretty clear that letting people off the hook gets especially dangerous if the loss is due to a psychological condition and not physical damage. There is also the chance that the memory loss is reversible, which means they're only rendered "temporarily benign."
To me, the purpose of prison isn't just reformation. The presence of that impending punishment sets up this ordered world we live in. The average person can live with their mind at ease knowing that if someone wrongs them they will pay for their crimes, the deterrent effect isn't just a tangible benefit, but it creates peace of mind. Every time we add an exception we chip away at that.
I guess I'm OK with an individual being punished who doesn't understand why because they've lost their memory as long as it's not unduly cruel, they're educated on what happened with all of the available evidence during their incarceration, and they undergo the same rehabilitation processes as anyone does -- plus they should be open to parole, given a little special treatment, and let out early if their story is corroborated by their actions and behavior during their imprisonment.
Then again, I really believe in rehabilitation and think a lot of our punishments for criminals are far too harsh. 20 years in prison for a crime of passion just doesn't add up to justice for me; what good is it doing anyone if it's clear the person isn't going to kill again?