Originally Posted by whitemischief
If anyone can give me a hint as to where I should look to try and eradicate this attacker forever, I'd be grateful.
Apparently it is not in a file that is replaced by the WordPress installation. That still leaves a lot of files. You have to go through each text file (as opposed to image and other binary files). There is no reliable trick to this. You just have to look through each file (except images). Make sure you also inspect your hidden files like .htaccess.
So, the hint would be to look in every file you haven't looked in yet.