How important is the HD "Safe Zone"

I am working on a game for Roku. Right now I am designing it for HDTVs using 1280x720 as my area to work in. Then I remembered that Roku suggests keeping important text and such within a "Safe Zone". I really would much prefer to utilize the entire screen. I have the gameplay in a 960x720 area in the middle of the screen and I would like to use 160 pixels width on each side for some text. It is not a lot of space but I can make it work. However if I develop according the the safe zone guidelines I would only get to use 96 of those pixels.

So my main question is. How many people will actually be affected if I have content right up to the edge of the screen