When I have an idea, I try to make a point of actually implementing it to see what happens. This is a collection of those experiments including both the successes and the failures.
Polynomial Residual Layer
A modification of residual connections to see how it affects a neural network’s approximation power.
Google Colab Notebook
Reduced Cross Entropy
A modified Cross-Entropy loss function.