Roku Developer Program

Developers and content creators—a complete solution for growing an audience directly.
Showing results for 
Search instead for 
Did you mean: 
Level 8

rokudev/unit-testing-framework: an imaginary solution

There is this "Unit Testing Framework" by RokuCo, which apparently has been available for almost 5 months now. I had deeper look yesterday and to summarize my conclusion, it is "an imaginary solution to a real need"^. No wonder it has never been officially announced. 

First, I won't grade it on good/bad scale - it is beyond the categories of "good" and "bad" - it simply does not work. It would be like discussing "gas mileage" on a car that won't start - well i don't know what's the MPG, since it hasn't driven even a mile yet. I discovered that critical functions are missing - e.g. FW_isAssociativeArray, FW_isАrray. The July 2016 delivery was apparently a "drop and run" event - if somebody from the Co had bothered downloading and testing the repository, they'd know it won't run.

Now there are good news, in that at least somebody did care after all - mostly @SiekermanTechnology, who took the lib for a spin in November and logged the 11 issues (including the missing files, #2 and #7), which have been blissfully ignored so far (hello?! Quo vadis, RokuCo "evangelists" and "developer experience managers"?). Luckily, he found a way to recover the missing files from elsewhere - and so did @cdthompson.

I am no big fan of XUnit kind of frameworks - they seem to me too much "form" and not enough "substance". Were i a "100% QA engineer", i might think different. However, i value the ability to verify functionality down to function level. TDD^^ style of work also has a very solid point!

I read the source of one of the files - fw_baseTestSuite.brs - so consider the following a peer-review notes:

  • BTS__AssertArrayContainsOnly() can't possibly work! Really? You think a function can dynamically be called like `("FW_Is" + typeStr)(item)`? That's adorable! Where do you think we are - Narnia? Tcl? Apparently never tested.

  • BTS__EqValues is unusable. Shows lack of understanding of the numeric types (hint: besides roFloat and roInt, there are also Integer, Float, Double, roDouble, LongInteger...) - try and see the following fail: (1, 1.0), (1, box(1)), (1, 1&), (1e, 1d). It also won't work with String vs roString, try ("12", "1"+"2"). Very sloppy, harmful code actually. Also, it would crash when called with incomparable types, say BTS__EqValues(global, global) or BTS__EqValues(myPort, myPort)

  • No point having AssertArrayCount(arr, cnt, msg), since can always do assert(arr.count() = cnt, msg) instead. Ditto AssertArrayNotCount()

  • BTS__AssertEmpty(), BTS__AssertNotEmpty do not fail when called with wrong type arguments (they should)

  • AssertEmpty(), AssertNoEmpty() make no sense: nobody said "you know what would be great - if there were 1 function that can check both arrays and string for 'emptiness' - but no other types". Instead just use assert(var="", msg) and assert(var.count()=0, msg)

  • BTS__EqAssocArray and BTS__EqArray don't check they were indeed passed AA and array, breaking tradition

(^) yes, i meant "imaginary" and not "imaginative" - my English is not that bad. It couldn't have been imaginative, since it's just attempt at hammering something JUnit/NUnit-alike for B/S - something that was done 7 years ago by Mark Roddy, see viewtopic.php?f=34&t=25722. That's right, seven years - beginning of 2010!
PS. and now that i had quick glace at BrsTest, well... i notice portions of code&comments that unit-testing-framework has taken from there without giving due copyright credit. Talk about being ungrateful to open source... technically that's illegal?

(^^) "Test-Driven Development", in which you write first the expectations how a functionality should work - and then "all you have to do" is fill out the function/method with code that makes the test cases run successfully. That keeps you honest about what the fn does - and also flags you when to stop (if there is no failing case, we are done). Plus it stays for later to catch refactoring when it back-fires (AKA regression testing).
0 Kudos
1 Reply
Level 8

Re: rokudev/unit-testing-framework: an imaginary solution

Ha - there is a great example of how not to use eval():
"fw_testRunner.brs#L208" wrote:

                    eval("testSuite=" + functionName + "()")

Basically the equivalent of BASIC's "ON ERROR RESUME NEXT" or `try { ... } catch(ignore) { /* nada */ }` - call it and never bother to check if there was an error. I noticed that and chuckled at it - but later saw that somebody suffered it in real life too, issue #9
0 Kudos