Deeplearning

The last few weeks, the autoresearch repository by Andrew Karparthy has made some waves. Everybody seemed to be hyped for LLMs doing deep learning research, while I had a look at the README and thought: “Well, that sounds like hyperparameter optimization with extra steps.” Below you can see the progress plot Karparthy published as part of the repo. The LLM runs 83 experiments over eight hours and successfully reduces the validation metric by around 0.0282 bits per byte. ...

“Another article comparing types of autoencoders?”, you may think. “There are already so many of them!”, you may think. “How does he know what I am thinking?!”, you may think. While the first two statements are certainly appropriate reactions - and the third a bit paranoid - let me explain my reasons for this article. There are indeed articles comparing some autoencoders to each other (e.g. [1], [2], [3]), I found them lacking something. Most only compare a hand full of types and/or only scratch the surface of what autoencoders can do. Often you see only reconstructed samples, generated samples, or latent space visualization but nothing about downstream tasks. I wanted to know if a stacked autoencoder is better than a sparse one for anomaly detection or if a variational autoencoder learns better features for classification than a vector-quantized one. Inspired by this repository I found, I took it into my own hands, and thus this blog post came into existence. ...