Thursday, March 09, 2006

Failure to Falsify

It is spring break here, and one of my primary tasks was to catch up on refereeing backlog.
Yes, I've been a bad boy. No, if your papers review was umpteen weeks late odds are pretty high it was not actually me, but maybe it was. Only The Editor and His Scientific Editors know fer sure. And me.

But, the concerted effort brought to mind a comment I got when I were but a lad.
A not-unkind referee's report noted that my wonderful model was parametrised, with several essentially free parameters.
Now, this is a common practise in theory papers of all sorts, I hear even String Theorists are prone to such things...
Ideally this tests a theory, in so far as observations will tell us which choice of parameters is true, and this may then constrain the underlying physics (which ought to be parameter free, if we only understood it right, righ?).
But, there is a not-uncommon problem associated with this, which is that with enough theoretical parameters, and little enough data, you can fit anything. So observations provide no actual information to constrain the theory. The theory is unfalsifiable.

Now, the branching ratio for blue straggler formation channels in dense stellar systems is unlikely to lead to theocracy, decadal sectarian wars, or even harsh blog comments... but, the point remained: the "theory" was unfalsifiable and strictly viewed from the right paradigmic angle, Not Science.
Suitably abashed I thought hard, and modified the paper to explain how possible observations could test, and falsify, broad classes of competing scenarios (and about now, "we" are getting round to actually doing the observations, gotta love that Hubble, but I digress). Yay.

So... here I am plowing through papers to comment on, I press the "Send" buttons and lean back and reflect.
Dammit! I think almost all the model scenarios I just refereed are essentially unfalsifiable, the parametric uncertainties are large enough that observations will never rule them out as formulated. That is not to say the theory is out-and-out wrong, rather the phenomena may be dominated by other physical effects that were approximated over or out, and hence these particular effects may be irrelevant. Only observations can really determine if that is the case, but the degrees of theoretical freedom are large enough that you can still argue for the presence of the effect in some relevant scenarios even with negative results for any arbitary set of actual observations - short of observing the whole universe for all time, I suppose.

I hate it when that happens.

Fortunately I got more papers to referee now, so I can go overcompensate.
Hah, just kidding.


Post a Comment

<< Home