roku brightscript benchmark suite...

ok below is my first Roku BrightScript program... the benchmark suite harness and first benchmark, sieve

in the days to come i will be porting/adding to the suite around 40 other benchmarks which I got from
http://shootout.alioth.debian.org/
http://dada.perl.it/shootout/
http://www.bagley.org/~doug/shootout/
(and other websites)

sub sieve(n)
	flags = CreateObject("roArray", 8192 + 1, true)

	count = 0

	while n > 0 
		n = n - 1
		
		count = 0

		for i = 2 to 8192
			flags[i] = true
		end for

		for i = 2 to 8192
			if (flags[i]) 
				' remove all multiples of prime: i
				for k = i + i to 8192 step i 
					flags[k] = false
				end for

				count = count + 1
			end if
		end for
	end while
end sub


sub RunBenchmarks()
	benchmarks = {sieve:100}
	
	screen = CreateObject("roParagraphScreen")
	port = CreateObject("roMessagePort")
	screen.SetMessagePort(port)
	
	screen.SetTitle("Benchmark Suite")

	screen.AddButton(1, "Close")
	
	screen.Show()

	dialog = CreateObject("roOneLineDialog")
	dialog.SetTitle("Running benchmarks ...")
	dialog.ShowBusyAnimation()
	dialog.Show()
  
	for each benchmark in benchmarks
		value = benchmarks[benchmark]

		dialog.SetTitle("Running benchmark " + benchmark + "(" + Stri(value) + ") ...")
		
		st = UpTime(0)
		Eval(benchmark + "(" + Stri(value) + ")")
		et = UpTime(0)
		
		screen.AddHeaderText(benchmark + "(" + Stri(value) + "): " + Str(et-st))
	end for
	
	dialog.Close()

	closed = false
	
	' event loop
	while true
		msg = wait(0, port) ' wait for an event

		if type(msg) = "roParagraphScreenEvent"
			if msg.isButtonPressed()
				buttonIndex = msg.GetIndex()

				if buttonIndex = 1
					exit while
				end if
			else if msg.isScreenClosed()
				exit while
			end if
		end if
	end while

	screen.Close()
end sub


sub DisplayMainScreen()
	screen = CreateObject("roParagraphScreen")
	port = CreateObject("roMessagePort")
	screen.SetMessagePort(port)
	
	screen.SetTitle("Benchmark Suite")

	screen.AddButton(1, "Start")
	screen.AddButton(2, "Close")

	screen.Show()

	' event loop
	while true
		msg = wait(0, port) ' wait for an event

		if type(msg) = "roParagraphScreenEvent"
			if msg.isButtonPressed()
				buttonIndex = msg.GetIndex()

				if buttonIndex = 1
					RunBenchmarks()
				else
					exit while
				end if
			else if msg.isScreenClosed()
				exit while
			end if
		end if
	end while

	screen.Close()
	' anytime all screens within a channel are closed, the channel will exit
end sub

sub Main()
	DisplayMainScreen()
end sub

ok today i expanded my benchmark suite by adding numerous benchmarks to test integer, float, double, boolean and string operators. I even found a bug 😉

Hopefully, tomorrow I hope to add benchmarks for individual BrightScript builtin functions.

Then I still have like 40 comprehensive benchmarks to port to BrightScript...

Happy computing...

sub bench(n)
end sub

sub sieve(n)
	flags = CreateObject("roArray", 8192 + 1, true)

	count = 0

	while n > 0 
		n = n - 1
		
		count = 0

		for i = 2 to 8192
			flags[i] = true
		end for

		for i = 2 to 8192
			if (flags[i]) 
				' remove all multiples of prime: i
				for k = i + i to 8192 step i 
					flags[k] = false
				end for

				count = count + 1
			end if
		end for
	end while
end sub

sub integerAssignmentOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = i%
	end for
end sub

sub integerArithmeticAdditionOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = r% + i%
	end for
end sub

sub integerArithmeticSubtractOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = r% - i%
	end for
end sub

sub integerArithmeticMultiplicationOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = r% * i%
	end for
end sub

sub integerArithmeticDivisionOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = r% / i%
	end for
end sub

sub integerArithmeticExponentiationOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = r% ^ i%
	end for
end sub

sub integerUnaryPlusOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = +i%
	end for
end sub

sub integerUnaryNegationOperator(n%)
	r% = 1%

	for i% = 1% to n%
		r% = -i%
	end for
end sub

sub integerComparisonLessThanOperator(n%)
	r% = 1%

	for i% = 1% to n%
		b = r% < i%
	end for
end sub

sub integerComparisonGreaterThanOperator(n%)
	r% = 1%

	for i% = 1% to n%
		b = r% > i%
	end for
end sub

sub integerComparisonLessThanOrEqualToOperator(n%)
	r% = 1%

	for i% = 1% to n%
		b = r% <= i%
	end for
end sub

sub integerComparisonGreaterThanOrEqualToOperator(n%)
	r% = 1%

	for i% = 1% to n%
		b = r% >= i%
	end for
end sub

sub integerComparisonEqualToOperator(n%)
	r% = 1%

	for i% = 1% to n%
		b = r% = i%
	end for
end sub

sub integerComparisonNotEqualToOperator(n%)
	r% = 1%

	for i% = 1% to n%
		b = r% <> i%
	end for
end sub

sub integerBitwiseAndOperator(n%)
	r% = 1%
	
	for i% = 1% to n%
		r% = r% and i%
	end for
end sub

sub integerBitwiseOrOperator(n%)
	r% = 1%
	
	for i% = 1% to n%
		r% = r% or i%
	end for
end sub

sub integerBitwiseNegationOperator(n%)
	for i% = 1% to n%
		r% = not i%
	end for
end sub

sub floatAssignmentOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = i!
	end for
end sub

sub floatArithmeticAdditionOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = r! + i!
	end for
end sub

sub floatArithmeticSubtractOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = r! - i!
	end for
end sub

sub floatArithmeticMultiplicationOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = r! * i!
	end for
end sub

sub floatArithmeticDivisionOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = r! / i!
	end for
end sub

sub floatArithmeticExponentiationOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = r! ^ i!
	end for
end sub

sub floatUnaryPlusOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = +i!
	end for
end sub

sub floatUnaryNegationOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		r! = -i!
	end for
end sub

sub floatComparisonLessThanOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		b = r! < i!
	end for
end sub

sub floatComparisonGreaterThanOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		b = r! > i!
	end for
end sub

sub floatComparisonLessThanOrEqualToOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		b = r! <= i!
	end for
end sub

sub floatComparisonGreaterThanOrEqualToOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		b = r! >= i!
	end for
end sub

sub floatComparisonEqualToOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		b = r! = i!
	end for
end sub

sub floatComparisonNotEqualToOperator(n!)
	r! = 1.0!

	for i! = 1.0! to n!
		b = r! <> i!
	end for
end sub

sub floatBitwiseAndOperator(n!)
	r! = 1.0!
	
	for i! = 1.0! to n!
		r! = r! and i!
	end for
end sub

sub floatBitwiseOrOperator(n!)
	r! = 1.0!
	
	for i! = 1.0! to n!
		r! = r! or i!
	end for
end sub

sub floatBitwiseNegationOperator(n!)
	for i! = 1.0! to n!
		r! = not i!
	end for
end sub

sub doubleAssignmentOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = i#
	end for
end sub

sub doubleArithmeticAdditionOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = r# + i#
	end for
end sub

sub doubleArithmeticSubtractOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = r# - i#
	end for
end sub

sub doubleArithmeticMultiplicationOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = r# * i#
	end for
end sub

sub doubleArithmeticDivisionOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = r# / i#
	end for
end sub

sub doubleArithmeticExponentiationOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = r# ^ i#
	end for
end sub

sub doubleUnaryPlusOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = +i#
	end for
end sub

sub doubleUnaryNegationOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		r# = -i#
	end for
end sub

sub doubleComparisonLessThanOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		b = r# < i#
	end for
end sub

sub doubleComparisonGreaterThanOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		b = r# > i#
	end for
end sub

sub doubleComparisonLessThanOrEqualToOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		b = r# <= i#
	end for
end sub

sub doubleComparisonGreaterThanOrEqualToOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		b = r# >= i#
	end for
end sub

sub doubleComparisonEqualToOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		b = r# = i#
	end for
end sub

sub doubleComparisonNotEqualToOperator(n#)
	r# = 1.0#

	for i# = 1.0# to n#
		b = r# <> i#
	end for
end sub

sub doubleBitwiseAndOperator(n#)
	r# = 1.0#
	
	for i# = 1.0# to n#
		r# = r# and i#
	end for
end sub

sub doubleBitwiseOrOperator(n#)
	r# = 1.0#
	
	for i# = 1.0# to n#
		r# = r# or i#
	end for
end sub

sub doubleBitwiseNegationOperator(n#)
	for i# = 1.0# to n#
		r# = not i#
	end for
end sub

sub logicalAndOperator(n)
	for i = 1 to n
		b = i > 5 and i < 7
	end for
end sub

sub logicalOrOperator(n)
	for i = 1 to n
		b = i < 5 or i > 7
	end for
end sub

sub booleanAssignmentOperator(n)
	r = false

	for i = 1 to n
		r = true
	end for
end sub

sub booleanLogicalNegationOperator(n)
	r = false

	for i = 1 to n
		r = not r
	end for
end sub

sub stringConcatenationOperator(n)
	r$ = ""

	for i = 1 to n
		r$ = r$ + "."
	end for
end sub

sub stringComparisonLessThanOperator(n)
	r$ = "45"

	for i% = 1% to n%
		b = r$ < Stri(i%)
	end for
end sub

sub stringComparisonGreaterThanOperator(n)
	r$ = "45"

	for i% = 1% to n%
		b = r$ > Stri(i%)
	end for
end sub

sub stringComparisonLessThanOrEqualToOperator(n)
	r$ = "45"

	for i% = 1% to n%
		b = r$ <= Stri(i%)
	end for
end sub

sub stringComparisonGreaterThanOrEqualToOperator(n)
	r$ = "45"

	for i% = 1% to n%
		b = r$ >= Stri(i%)
	end for
end sub

sub stringComparisonEqualToOperator(n)
	r$ = "45"

	for i% = 1% to n%
		b = r$ = Stri(i%)
	end for
end sub

sub stringComparisonNotEqualToOperator(n)
	r$ = "45"

	for i% = 1% to n%
		b = r$ <> Stri(i%)
	end for
end sub

sub referenceComparisonEqualToOperator(n)
	roa = CreateObject("roAssociativeArray")
	rol = CreateObject("roList")

	for i% = 1% to n%
		b = roa = rol
	end for
end sub

sub referenceComparisonNotEqualToOperator(n)
	roa = CreateObject("roAssociativeArray")
	rol = CreateObject("roList")

	for i = 1 to n
		b = roa <> rol
	end for
end sub

function RunBenchmarks()
	benchmarks = [
		["bench",1],
		["sieve",100],
		["integerAssignmentOperator",100],
		["integerArithmeticAdditionOperator",100],
		["integerArithmeticSubtractOperator",100],
		["integerArithmeticMultiplicationOperator",100],
		["integerArithmeticDivisionOperator",100],
		["integerArithmeticExponentiationOperator",100],
		["integerUnaryPlusOperator",100],
		["integerUnaryNegationOperator",100],
		["integerComparisonLessThanOperator",100],
		["integerComparisonGreaterThanOperator",100],
		["integerComparisonLessThanOrEqualToOperator",100],
		["integerComparisonGreaterThanOrEqualToOperator",100],
		["integerComparisonEqualToOperator",100],
		["integerComparisonNotEqualToOperator",100],
		["integerBitwiseAndOperator",100],
		["integerBitwiseOrOperator",100],
		["integerBitwiseNegationOperator",100],
		["floatAssignmentOperator",100],
		["floatArithmeticAdditionOperator",100],
		["floatArithmeticSubtractOperator",100],
		["floatArithmeticMultiplicationOperator",100],
		["floatArithmeticDivisionOperator",100],
		["floatArithmeticExponentiationOperator",100],
		["floatUnaryPlusOperator",100],
		["floatUnaryNegationOperator",100],
		["floatComparisonLessThanOperator",100],
		["floatComparisonGreaterThanOperator",100],
		["floatComparisonLessThanOrEqualToOperator",100],
		["floatComparisonGreaterThanOrEqualToOperator",100],
		["floatComparisonEqualToOperator",100],
		["floatComparisonNotEqualToOperator",100],
		["floatBitwiseAndOperator",100],
		["floatBitwiseOrOperator",100],
		["floatBitwiseNegationOperator",100],
		["doubleAssignmentOperator",100],
		["doubleArithmeticAdditionOperator",100],
		["doubleArithmeticSubtractOperator",100],
		["doubleArithmeticMultiplicationOperator",100],
		["doubleArithmeticDivisionOperator",100],
		["doubleArithmeticExponentiationOperator",100],
		["doubleUnaryPlusOperator",100],
		["doubleUnaryNegationOperator",100],
		["doubleComparisonLessThanOperator",100],
		["doubleComparisonGreaterThanOperator",100],
		["doubleComparisonLessThanOrEqualToOperator",100],
		["doubleComparisonGreaterThanOrEqualToOperator",100],
		["doubleComparisonEqualToOperator",100],
		["doubleComparisonNotEqualToOperator",100],
		["doubleBitwiseAndOperator",100],
		["doubleBitwiseOrOperator",100],
		["doubleBitwiseNegationOperator",100],
		["logicalAndOperator",100],
		["logicalOrOperator",100],
		["booleanAssignmentOperator",100],
		["booleanLogicalNegationOperator",100],
		["stringConcatenationOperator",100],
		["stringComparisonLessThanOperator",100],
		["stringComparisonGreaterThanOperator",100],
		["stringComparisonLessThanOrEqualToOperator",100],
		["stringComparisonGreaterThanOrEqualToOperator",100],
		["stringComparisonEqualToOperator",100],
		["stringComparisonNotEqualToOperator",100],
		["referenceComparisonEqualToOperator",100],
		["referenceComparisonNotEqualToOperator",100]
	]
	
	dialog = CreateObject("roOneLineDialog")
	port = CreateObject("roMessagePort")
	dialog.SetMessagePort(port)
	
	dialog.SetTitle("Running ...")
	
	dialog.ShowBusyAnimation()
	
	dialog.Show()
	
	print "Running Benchmark Suite"
	print

	tt = 0
	
	text = ""
	
	for each benchmark in benchmarks
		print "Running Benchmark: " + benchmark[0] + "(" + Stri(benchmark[1]) + ") ..."
		
		st = UpTime(0)
		Eval(benchmark[0] + "(" + Stri(benchmark[1]) + ")")
		et = UpTime(0)

		t = et-st
		
		tt = tt + t
		
		print "Elapsed Time: " + benchmark[0] + "(" + Stri(benchmark[1]) + "): " + Str(t) + " s"
		print
	end for
	
	print "Total Elapsed Time: " + str(tt) " s"
	print
	
	dialog.Close()
	
	return tt
end function

sub DisplayRunScreen()
	screen = CreateObject("roParagraphScreen")
	port = CreateObject("roMessagePort")
	screen.SetMessagePort(port)
	
	screen.SetTitle("Benchmark Suite")
	
	screen.AddButton(1, "Close")
	
	screen.Show()

	tt = RunBenchmarks()

	screen.AddHeaderText("Elapsed Time: " + Str(tt) + "s")
	
	' event loop
	while true
		msg = wait(0, port) ' wait for an event

		if type(msg) = "roParagraphScreenEvent"
			if msg.isButtonPressed()
				buttonIndex = msg.GetIndex()

				if buttonIndex = 1
					exit while
				end if
			end if
			if msg.isScreenClosed()
				exit while
			end if
		end if
	end while

	screen.Close()
end sub

sub DisplayMainScreen()
	screen = CreateObject("roParagraphScreen")
	port = CreateObject("roMessagePort")
	screen.SetMessagePort(port)
	
	screen.SetTitle("Benchmark Suite")
	
	screen.AddButton(1, "Start")
	screen.AddButton(2, "Close")
	
	screen.Show()

	' event loop
	while true
		msg = wait(0, port) ' wait for an event

		if type(msg) = "roParagraphScreenEvent"
			if msg.isButtonPressed()
				buttonIndex = msg.GetIndex()

				if buttonIndex = 1
					DisplayRunScreen()
				else if buttonIndex = 2
					exit while
				end if
			end if
			if msg.isScreenClosed()
				exit while
			end if
		end if
	end while

	screen.Close()
end sub

sub Main()
	DisplayMainScreen()
end sub

Geoff (Gandk-Geoff) over here actually wrote a benchmark framework so we could determine the cost of low level operations in BrightScript. From what I remember it's fairly comprehensive, but it's targeted at low level brightscript ops, not algorithms. Stuff like what's the difference between accessing a variable and an array item or associative array item (quite a bit, is the answer).

I'll see if he's interested in making it public.

From what I remember, typing your variables actually results in a (near negligible) performance hit, associative array and list/array access is pretty horrendous (it you are trying to accomplish ops in a tight loop, e.g. trying to achieve 60fps in a came), and there's a few other gotchas.

I always wanted to save all the results and post them up with a version number to track process, but never got around to it.

"kbenson" wrote:
associative array and list/array access is pretty horrendous (it you are trying to accomplish ops in a tight loop, e.g. trying to achieve 60fps in a came)

I've recently been doing some of my own benchmarking for this same exact reason. I was shocked at the performance difference between:

point = {
    x: 100
    y: 100
}
[...loop...]
    newX = point.x + i
[...loop...]

and

x = 100
y = 100
[...loop...]
    newX = x + i
[...loop...]

That may explain why the 2D interfaces take ints instead of rects...

wow very nice looks like a lot of though and effort went into creating your benchmark suite thank you we will all now be able to benefit from your efforts that is very kind gesture best regards