Both of my Roku Ultras have exactly this same behavior.
Out of the blue one day, both Ultra units start at level 50 for volume. If I turn up the volume via the Roku remote, the audio level does NOT increase, although the onscreen volume level display does look as though the volume is being increased. I turned the volume level to maximum on my tv and headphones, just to be able to hear the streaming audio at all.
I have TWO Roku 4 Ultra units (models 4800X, 4640X) and they BOTH developed this low volume problem at the same time. Previously, both Roku Ultra units worked as expected, with volume being very loud when turned up to max level.
I haven't changed hook-up configuration, connected hardware, Roku models, headphones or any other variable.
Each of the 2 Rokus are hooked-up to a different brand of tv (one Vizio, one TCL).
Both Rokus use the same internet connection, as they (and their many Roku predecessors) always have.
Neither Roku has "volume leveling" turned on. Restarting the units had no effect.
Both Rokus have the same software version (10.5.0) and Build (4208), with no pending updates available.
Based on these facts, I strongly suspect a software problem.
I contacted Roku Support and have been told to try things that make no sense to me, such as "try another HDMI cable". Really? You think that BOTH HDMI cables failed on BOTH Roku Ultras on the same day?!?
Would love to hear from anyone else who has this problem - especially if you've found a solution.