Newer Roku remotes use WIFI instead of infrared to communicate with the player. When you power your Roku from a USB port on the back of your TV, it only provides power while the TV is on. When you turn your TV off, power is cut to the Roku, and this causes the remote to constantly "search" for the Roku unit. Some TV models allow you to keep power to the USB ports while the TV is off. You should enable that feature if you want to maximize battery life. In the absence of that feature, you can power the Roku with a wall adapter instead.
The tradeoff to this is that your Roku stays on constantly, which is by design. Some argue that like your smart speaker, phone, and other voice operated devices the Roku voice remote is always listening. Some say leaving the unit powered all the time shortens the life of the Roku unit. You have to decide what works best for your situation.
