@jdevine03, the TV control part of the remote is completely separate from the Roku control part, so it's possible for TV controls to work but not the Roku controls or vice versa. Have you tried swapping remotes to see if the problem lies in the remote or the stick? You'll have to re-pair the remotes if you do that which can be a headache in itself.
The TV controls work via IR (infrared) so you must point the remote at the TV unlike the Roku controls where you can point the remote anywhere. I would say try new batteries for the one on the upstairs TV, but I would think the Roku controls would stop working before the TV controls. You can verify whether the TV controls are working by pointing the remote at a digital camera like your phone's and pressing a TV control button on the remote. You should see a flash of light on the camera screen. You can see the battery level under the Remotes section of the Settings. It may or may not be accurate, but the stronger the batteries the brighter the flash of light should be on the camera screen.
Roku Community Streaming Expert
Help others find this answer and click "Accept as Solution."
If you appreciate my answer, maybe give me a Kudo.
I am not a Roku employee.