I have had this issue on multiple Roku remotes over the years. For some reason the wifi voice remotes with TV controls drain the batteries much faster than they should. It is annoying for sure, what I started doing is simply removing the batteries from the remotes when I am done with the Roku, then putting them back in when I want to use the Roku again.
It's very annoying having to do this but it does help a lot. For some reason even when not in use these remotes are draining the batteries on us, the best option is don't give it that chance when you aren't using it.
Leave the batteries out and store them in the casing that pops off so they don't roll off the table, that is what I do. Roku seems unwilling or unable to fix this issue with any type of update and they certainly don't seem to be in any hurry to design new remotes that actually function properly.