I like what Jerry Martinson has to say here. Comments? Critiques?
http://www.futurepundit.com/mt/mt-altcomments.cgi?entry_id=4589
Jerry Martinson said at September 18, 2007 12:43 AM:
I've never really understood how public research is funded in any particular area. I don't know but I'm guessing it has a lot to do with the political connections of the principle investigators. To me, it seems that the result is a messy patchwork of studies that have limited statistical power to discover the unknown. When I read medical research papers, I'm kind of shocked at the culture of the experimental design. There's a great emphasis on a few "canned" types of experimental design that worship at the alter of Gauss. This culture emphasizes formality of a particular kind of investigation, at the expense of trying to discover the unexpected. It seems like there is little coordinated planning on how to record information in such a way as to make synthetic "meta-analysis" actually automated or convenient, much less useful and accurate. I worry that the culture of professors guiding underlings who need to show original research to complete academic requirements in a certain way is inhibiting genuine discovery.
I'm not aware of much philosophical work on the art experimental design for discovering the unknown that has influenced medical research. There is quite a bit of work in electrical engineering and in certain parts of OR-centric industrial engineering that have done quite a bit of work on the problem in general however. Consider the hidden Markov models or Kalman filtering extensions that electrical engineers have to learn in school. I remember in college where fresh after learning about Kalman filters, Markov models, and Bayesian methods in the EE classes, I then attended a very interesting lecture from George Box where it seemed like all these extremely powerful ideas that EE's are trained in could then be adapted to systematic empirical discovery of financial and medical models. The ideas seemed much more powerful than Black-Scholes or the simplified statistical methods that are taught in the life-sciences.
It would be interesting if one could develop a well-accepted method of making complex system models using this type of framework that mimic the complicated models we already know about in biological systems. We could then genericize the models and then we could then test various kinds of investigation and experimental designs to see how rapidly they converge on the correct hidden model. From the knowledge built up from that, places like the NIH or the HHMI could then make more intelligent funding decisions and set better standards for keeping and recording information.
Another thing about studies is that the published results create too much prose centered around discussion of the "significant" finding with the underlying valuable data archived in some arcane format that is inaccessible or incompatible with other works. Better standards for how the (unplublished) records are kept, organized, and made accessible could lead to much better derivative data mining efforts. Guidance from experimental design research could suggest which "schemas" would be best for derivative research rather than being a fool's errand for some SQL expert who knows little about the data he's organizing.
Comments