Now that I think of it, I believe I actually have the Roku Premier so I think my remote is different than the Ultra.
I just bought mine a month or so ago and it has no mic icon. I've also tried using voice commands with Google Home to turn up the volume. The Roku does recognize the command but it still does nothing. (And yes, my software is up to date.)
So, volume control is apparently non-existent for the Roku Premier.