So, you have an old tv without hdmi input, yes you will need a converter, not just plug or cable converter they don't work. You need a box with circuitry to properly convert from one signal to two. Like this one:
TNP HDMI to DVI Converter with Audio Out - HDMI to DVI Video Audio Adapter Sound Splitter to 3.5mm A...
Dvi is only video, you need a separate audio line which the above converter gives you.
As for your newer Roku that replaced your old Roku, it still has av outputs? That makes it at least 5 or 6 years old, its channels are going to stop working soon too, I would upgrade to newest Roku you can, so you get ALL channels going forward. All the Roku needs is hdmi output, which links to hdmi input on the converter, the video is sent to tv by dvi, and the audio by breakout cable 3.5mm to rca.
Edmund
Roku Community Streaming Expert
Help others find this answer and click "Accept as Solution."
If you appreciate my answer, maybe give me a Kudo.
I am not a Roku employee.