I upgraded from Roku stick to Roku Ultra for the main reason to have a USB port to play my own media. I have also downloaded Roku Media Player for this purpose. Everything works well, except pictures taken by Sony camera as vertical are displayed as horizontal on Roku Media Player, which apparently does not read JPG file properties. Therefore, I recommend for Roku to fix this bug in the next software update.
Nevertheless, I found a temporary fix by editing the image in Irfan View and adding black horizontal canvas on both sides of the image, so Roku thinks that the picture is horizontal and displays it properly.