Closed captioning is off but captions appear anyway

Is this happening across the board on all channels, or just on some particular ones that you tend to watch?

The Roku default method of accessing your overall Closed Captioning options from the Roku home screen is via Roku menu path "Settings > Accessibility" and use the "Subtitles & Captioning" section on the right of the screen.

While viewing a program, press the * key on your Roku remote and examining the options presented which can vary depending on what that channel supports. You may have to look in Accessibility.

But more and more channels are not using the Roku default, and CC can only be set while viewing the program. These channels substitute their own method of access -- I'm guessing this is so they will have consistent controls with versions of their app running on platforms other than Roku. Often these options are accessed by pressing the up and/or down cursor control keys while viewing the program to pop the channel's own control options on the screen.