Why is scientific progress incremental? Clearly, the construction of increasingly unified theories in physics and elsewhere is an example incremental compression of experimental data, of the description of our world.
On the other hand, we know that the compression problem, the problem of finding the shortest program given data is optimally solved by Levin search. However, this search procedure is not incremental, but an all-or-none search where all programs are executed in a dovetailed fashion until a programs is found that generates the required sequence.
One explanation could be that our compression scheme in science is not optimal. However, progress in science is fairly fast and seems to be much faster than Levin search. Imagine, for example, the present quantum field theory incoded as a bitstring, and permutate all strings of that length. It certainly would take a gazillion years even with our fastest comupters.
What resolves that paradox then? This line of thought seem to show that the data that we are typically dealing with, when we try to explain our world, is drawn from a subset of all possible sequences, a subset, for which a much faster compression scheme exists than Levin search. I suspect that those are sequences with high logical depth.