Category Archives: compression

Emergence of attention mechanisms during compression

It just dawned on me. When we want to compress, we have to do it in one or the other incremental fashion, arriving in description lengths of intermediate length during the process. Also, there will be different paths that we … Continue reading

Posted in attention, compression | Leave a comment

The merits of indefinite regress

The whole field of machine learning, and artificial intelligence in general, is plagued by a particular problem: the well known curse of dimensionality. In a nutshell, this curse means that whenever we try to increase the dimension of our search … Continue reading

Posted in compression, Uncategorized | 7 Comments