How do I delete all files for a game?

After deleting a game from XMB, then reinstalling it still starts me at the same point. Are there hidden saved files somewhere or something? I looked for the NPUA after deleting from XMB and it's not there.

More specifically:
I was playing Infamous Festival of Blood, tried to change appearance settings and now the game is stuck in an unplayable mode. I deleted the game from XMB, reinstalled it and it still starts me at the same place I was. How do I delete all the files associated with it so I can install and start from the beginning? I tried to find the NPUA folder in the GAME folder but it was not there.

I suppose while I'm at it, how would I save the save data for games I want to move to an external HD so when I want to play them again I haven't lost all my progress?