Roku Developer Program

Join our online forum to talk to Roku developers and fellow channel creators. Ask questions, share tips with the community, and find helpful resources.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Roku Guru

rokudev/unit-testing-framework: an imaginary solution

There is this "Unit Testing Framework" by RokuCo, which apparently has been available for almost 5 months now. I had deeper look yesterday and to summarize my conclusion, it is "an imaginary solution to a real need"^. No wonder it has never been officially announced. 

First, I won't grade it on good/bad scale - it is beyond the categories of "good" and "bad" - it simply does not work. It would be like discussing "gas mileage" on a car that won't start - well i don't know what's the MPG, since it hasn't driven even a mile yet. I discovered that critical functions are missing - e.g. FW_isAssociativeArray, FW_isАrray. The July 2016 delivery was apparently a "drop and run" event - if somebody from the Co had bothered downloading and testing the repository, they'd know it won't run.

Now there are good news, in that at least somebody did care after all - mostly @SiekermanTechnology, who took the lib for a spin in November and logged the 11 issues (including the missing files, #2 and #7), which have been blissfully ignored so far (hello?! Quo vadis, RokuCo "evangelists" and "developer experience managers"?). Luckily, he found a way to recover the missing files from elsewhere - and so did @cdthompson.

I am no big fan of XUnit kind of frameworks - they seem to me too much "form" and not enough "substance". Were i a "100% QA engineer", i might think different. However, i value the ability to verify functionality down to function level. TDD^^ style of work also has a very solid point!

I read the source of one of the files - fw_baseTestSuite.brs - so consider the following a peer-review notes:

  • BTS__AssertArrayContainsOnly() can't possibly work! Really? You think a function can dynamically be called like `("FW_Is" + typeStr)(item)`? That's adorable! Where do you think we are - Narnia? Tcl? Apparently never tested.

  • BTS__EqValues is unusable. Shows lack of understanding of the numeric types (hint: besides roFloat and roInt, there are also Integer, Float, Double, roDouble, LongInteger...) - try and see the following fail: (1, 1.0), (1, box(1)), (1, 1&), (1e, 1d). It also won't work with String vs roString, try ("12", "1"+"2"). Very sloppy, harmful code actually. Also, it would crash when called with incomparable types, say BTS__EqValues(global, global) or BTS__EqValues(myPort, myPort)

  • No point having AssertArrayCount(arr, cnt, msg), since can always do assert(arr.count() = cnt, msg) instead. Ditto AssertArrayNotCount()

  • BTS__AssertEmpty(), BTS__AssertNotEmpty do not fail when called with wrong type arguments (they should)

  • AssertEmpty(), AssertNoEmpty() make no sense: nobody said "you know what would be great - if there were 1 function that can check both arrays and string for 'emptiness' - but no other types". Instead just use assert(var="", msg) and assert(var.count()=0, msg)

  • BTS__EqAssocArray and BTS__EqArray don't check they were indeed passed AA and array, breaking tradition

(^) yes, i meant "imaginary" and not "imaginative" - my English is not that bad. It couldn't have been imaginative, since it's just attempt at hammering something JUnit/NUnit-alike for B/S - something that was done 7 years ago by Mark Roddy, see viewtopic.php?f=34&t=25722. That's right, seven years - beginning of 2010!
PS. and now that i had quick glace at BrsTest, well... i notice portions of code&comments that unit-testing-framework has taken from there without giving due copyright credit. Talk about being ungrateful to open source... technically that's illegal?

(^^) "Test-Driven Development", in which you write first the expectations how a functionality should work - and then "all you have to do" is fill out the function/method with code that makes the test cases run successfully. That keeps you honest about what the fn does - and also flags you when to stop (if there is no failing case, we are done). Plus it stays for later to catch refactoring when it back-fires (AKA regression testing).
0 Kudos
Roku Guru

Re: rokudev/unit-testing-framework: an imaginary solution

Ha - there is a great example of how not to use eval():
"fw_testRunner.brs#L208" wrote:

                    eval("testSuite=" + functionName + "()")

Basically the equivalent of BASIC's "ON ERROR RESUME NEXT" or `try { ... } catch(ignore) { /* nada */ }` - call it and never bother to check if there was an error. I noticed that and chuckled at it - but later saw that somebody suffered it in real life too, issue #9
0 Kudos