The Black Box is an ingenious idea for teaching DOE via a hands-on exercise – far easier than other approaches like catapults, trebuchets, paper helicopters, or golfing toys (been there and done that as you can see via the links). In less than half an hour I experimented on the upper left ‘sextant’ of the Black Box. Originally I’d planned to get help from my son Hank, but I discovered it was easy enough just to do myself. I think it’s a blast!
What’s great about doing an actual (not simulated) experiment is running into practical issues of having to do pre-experimental range-finding, dealing with measurement issues (two different scales on the ruler, where to measure too, how hard to push down, etc) and so forth. Other aspects are more subtle, such as the difficulty when running an experiment to not look at the prior result of a replicated run and cheat on making each one match. For example, I swear that I did not cheat on the repeats, but maybe I did unconsciously, because so many agreed exactly. Also, I realized when talking with Jim afterwards that I misread the 64ths scale as 60ths! Doh!!! (For the record, I corrected the numbers.)
I set up a 2^2 (two-level factorial) with 3 center points in a fully-replicated, blocked design — see results attached. Just for fun, I tried analyzing the first block — very educational — it reminded me not to try analyzing an unreplicated 2^2! (Four runs provide nothing for statistical testing unless one makes the dangerous assumption that the two-factor interaction (2FI) effect must be a measure of experimental error.) As shown by the 3D surface, my 2FI model fell short of the center points (notice how they all ‘lollipop’ up) – thus the ANOVA revealed significant curvature (p = 0.0001)as evidenced by the center points .
I gave Jim back his Black Box before I could probe its mysteries any further by augmenting my initial experiment design into a response surface method (RSM), for example by simply checking the centers of the edges of the square region. Jim says that he hopes to go into production with his Black Box by year end. At that time he will offer us one to evaluate for our training. Then my battle with the Black Box can be continued.