The chip in Roku does decode various codecs, too bad they would not fix MKV format.
USB Media Formats
Video: MP4 (H.264) MKV (H.264)
Audio: AAC, MP3
Image: JPG, PNG
I've never seen that problem. Do they have any sort of DRM? Can you send me a problem file so I can take at look at what's going on on the Roku (maybe it's throwing some kind of error message)? I guess you could always convert it to a WAV file and then back to MP3 with Audacity.
There's no limit as far as I know. I have well over 2,000 and at least one person has over 8,000. I'm guessing you'd run into problems if they were all in one folder/directory, so hopefully you have some sort of organizational system.