Unfortunately, there is no way to add additional codec support to any Roku. Your only options are to either re-encode your videos using H.264 or H.265, or use a DLNA server to transcode and stream the videos to your player. I do the latter, using Serviio as my DLNA server. There's also Plex, which can act as a DLNA server or use its native Roku app. And there are many other possible choices that support all the usual operating systems. . The disadvantage of running a DLNA server is the need to keep a separate computer (or NAS) running on your network. However, it has a major advantage in that you can now stream your media to any device on your network, and support more than one stream at a time, depending on your server hardware's capabilities. Since I leave all of my computers on 24/7 anyway, there's no down side for my operation.
Dan Ultra (4640), Ultra (4670), Ultra (4800), Premiere (3920), Insignia 720p Roku TV, Sharp 4K Roku TV, Nvidia Shield, Windows 10 Pro x64 running Serviio and Plex on a wired Gigabit network.
I actually run a NAS with Emby, Plex, and miniDLNA. I bought this Roku hoping I won't have to have a bunch of equipment to do my streaming. I already use a PS3 and Sony tiny Blu-Ray box. The Sony box plays those files. I had high hopes for the Roku tho.
I will re-code using h264 and use miniDLNA as a pass-trough, works wonders.