There is no way for you to specify the buffer time. However, if you know what the highest bitrate in your HLS variant list is, you could set it to that. This is currently required as a workaround for a Roku bug if the highest bitrate is less than 1.5 Mbps.
--Kevin