Plug what in? My Sony TV is connected to my Yamaha AV Amp via an audio optical connection. The volume needs to control the Amp, not the TV. I find it just absolutely ridiculous that Roku hasn't been able to control an AV amp, but every cheap universal remote and Amazon firestick remote can do it. If the Firestick+ wasn't such slow flakey product, I'd keep it just to consolidate remotes.
The Roku remotes can only control the Roku devices and the television's volumes but not accessories as they are unprogrammable.
The best solution is to use a universal streaming remote if you are trying to consolidate into one remote. (even the $10 ones will let you control the tv, the roku, and your accessories like the AV amps).
Just dont throw your Roku remote away though, sometimes you need it to set Roku back up before pairing to a new remote.
EDIT: If you enable the HDMI CEC on your TV and has ARC support, the Roku remote may actually work the volume of the amp similar to the RokuTVs. Going to be in the TV settings though if using a Roku streaming device as opposed to the RokuTV that has that feature in its settings.