Roku Ultra white light always stays on?

Roku devices don't turn off, so yes, it's always drawing power, but it's minimal when it's idle (and pretty low when it's not).  The power button on the remote is strictly for the TV.  Depending on the Roku model there may be a "power saving" setting under Settings->System->Power.  My testing shows this saves no more than a Watt or two and can cause more problems than it's worth.

The light should go off when either the screensaver activates or power saving mode kicks in, but it's become less and less reliable as an indication of anything with each Roku update.  You'll find plenty of threads on the issue.

Confirmed this works - uncheck the power saving feature and the white LED in the front of the unit will go off after a short pause when the TV is powered off.

Hey Jeff you can skip the tape, I also just bought two newer Roku Ultra LTs and though there isn't a specific LED setting (I'd checked thoroughly), the trick of unchecking the 20 minute power saving feature did in fact cause the LED to go out after powering off the TV (and a brief pause).