I can only speculate and give you things to try out.
1) Be sure that you are differentiating between time executing brightscript and time waiting for images to download. It looks like you have already done this by making your custom component and empty group.
2) remove the onChange from the fields. The number of observer callbacks invoked can cause overhead, and especially so if you have a particularly large number of BRS objects or SG components in memory.
3) RAF needs memory for video cache and will probably affect downloaded images which may be purged during ad playback.
4) You should put a print statement in all of your functions so that you can see in real time what is happening. a timestamp in these print statements would be helpful too.
5) Be aware of the total system resource and what other threads may be doing. After RAF ad break ends, if you have a Task thread running or main brightscript thread doing work then it will take away from SG UI thread CPU resources.