Using cable remote switches back to ROKU box - how to disable

Roku and the cable box are plugged into separate HDMI sources. I am able to switch between them, however when I use the remote control for the cable, it activates ROKU again 

The only way to stop this is to completely unplug the Roku when I want to use the cable 

How do I disable this and have separate remotes and sources?

Cheers!