There is a CEC feature (1-touch play) within Roku that allows any button press on the Roku remote (other than the Power and Volume buttons) to automatically switch to the HDMI port that Roku is assigned and wake the device. If you are using the Roku remote to try and change channels on your TV this may be the issue.
Otherwise, if you are using your TV remote (not Roku remote) and the above is occuring, then there may just be a conflict between the remotes.
Try disabling Roku's 1-touch play in your settings and see if that helps. Settings/System/Control other Devices/1-touch play [Uncheck to Disable]
Roku Community Streaming Expert
Just another Roku user... I am not a Roku employee.
Insignia RokuTV, Ultra 4660x, Premiere+ 3921, Express 4k+ 3941