For most people this will probably be an incredibly boring post. You’ve been warned. For others, like me, who are very concerned with language in science, and precision of language, this is the post for you.
I’ve always described our review criteria on Balefire Labs as “objective.” But lately I’ve been noticing that a lot of other edtech review sites describe their criteria as objective. And I don’t think their criteria are all that objective. So I wanted to dig into that here and ask for your help in thinking it through.
To me, an objective criterion is one that is clearly defined in specific terms. The terms are specified sufficiently that two or more people can agree on whether or not the terms of the definition are met. So, for example, an objective criterion for an educational app might be “includes feedback for correct answers.” Subjective criteria, conversely, might be “fun and engaging.”
But I’ve been noticing that some edtech review sites, instead of using the term “objective” to mean the opposite of subjective, are using it to mean unbiased. For me, that is a completely separate thing having to do with whether or not a reviewer has an interest, monetary or otherwise, in seeing an app do well in the marketplace.
So here’s my dilemma that I need your help with and I’m going to get very nerdy and pedantic here. In science, we have another term that I could use instead of “objective,” given the clear differences in how people are using it. That term is “operational definition.” An operational definition defines something according to a specific process or set of validation tests that determine its presence or quantity. One Princeton University faculty member’s website explains its use thus: One might use definitions that rely on operations in order to avoid the troubles associated with attempting to define things in terms of some intrinsic essence. (Fun and engaging, anyone?)
Here is some more good stuff on operational definitions from that same website: Properties described in this manner must be sufficiently accessible, so that persons other than the definer may independently measure or test for them at will. The binary version [of the process] produces either the result that the object exists, or that it doesn’t, in the experiential field to which it is applied.
Okay. So you can probably see where I am with this. The way that we, at Balefire Labs, approach our reviewing is with operational definitions for our criteria. Essentially, we take a science-based approach to reviewing and we strive for our criteria to be unambiguous in their application by us and in their consumption by customers. Why do we think that’s important? Because we are striving to tell customers what they are getting in each app that comprises its instructional quality (based on best practices from educational research), not our subjective opinion of what is fun, engaging, or similar (unbiased though it may be).
Not to mention the fact that what is fun and engaging is pretty idiosyncratic and related to kids’ skill levels. But that’s another post.
I had been thinking that referring to our criteria as “objective” would simplify the marketing process so that we wouldn’t have to burden our customers with learning about operational definitions and why they’re important. But now I’m in a quandry about that because I think it’s confusing when people use the same word, in this case “objective,” to mean multiple things. Instead, should I take on the task of using “operationally-defined criteria?”
Need your opinions and advice on this. Have any of you had experience educating the public on operational definitions? And how has it gone? Thanks in advance for the help!