Those buttons are about the tiniest buttons I've ever seen. If you can't get it unstuck just remove it. It shouldn't affect anything - you just won't have a mute button.
I did just try a test with my remote and I can hold in the mute button and my Ultra still responds to the Roku control buttons, so it may not be your problem. My remote is using IR for TV control (not CEC) if that makes a difference. Assuming the button is stuck and the batteries aren't dead, you should see the IR transmitter always illuminating when you point the remote at your phone's camera.
Roku Community Streaming Expert
Help others find this answer and click "Accept as Solution."
If you appreciate my answer, maybe give me a Kudo.
I am not a Roku employee.