Are you using the same version of SMS for both tests?
The Sony CDVDMAN module (Loaded by the PS2 on processor reboots) can only handle 8+3 filenames, and the CDFS module (Loaded by SMS) should extend on that limitation to support LFNs.
The only thing I can think of is that SMS didn't load CDFS, or somehow CDFS isn't working right (Other than the other probability that you used an older version of SMS).
We need more information. What's your console's model, the type of disc you that SMS is on, and the type of disc that your media files are on.
Also, what is the version of SMS that you are using?
SCPH-10000 S. MINOKAMO v1.01 (Defunct)
SCPH-10000 S. KISARAZU v1.00 (faulty)
SCPH-15000 S. KOHDA (With warranty seal!) :D
DESR-5100 S. EMCS