You can't use factory service mode to downgrade if you're on 3.56 or later. In fact, if you put it in FSM if it's on that range of firmwares, it will get stuck on it and no way to recover it without downgrading.
As for downgrading, it's done with a flasher (currently requires soldering, although solderless kits are supposed to come out soon), you directly rewrite your PS3's flash, there are tuts on how to do this in the net, too lazy to look for them though.
PS3 Slim CECH-3004A 160GB
PS2 Slim SCPH-70004 - FMCB 1.8b