Now that I think of it, I believe I actually have the Roku Premier so I think my remote is different than the Ultra.
I just bought mine a month or so ago and it has no mic icon. I've also tried using voice commands with Google Home to turn up the volume. The Roku does recognize the command but it still does nothing. (And yes, my software is up to date.)
So, volume control is apparently non-existent for the Roku Premier.
I agree that they should figure out how to control volume. I have an old Sharp/Roku TV and it had volume control (mute vol+ or vol-) and I could even turn the TV on/off. All showed up on the screen on my iPhone. We "upgraded" all of our TVs to Roku Ultra and hardwired them. Now we have fewer features. Bummer to take a step backwards.