A Survey on recent activation functions with emphasis on oscillating activation functions

Abstract

The neural network learning and generalizing mechanisms of the brain have always intrigued scientists and researchers. Therefore, with the aim to mimic these “physiological workings” of the brain, the concept of using layers of artificial neurons (nodes) to make experienced predictions, accurate decisions, etc (Artificial neural network (ANN)) was introduced [ 5]. This paper briefly discusses few activation functions that have been used frequently over the years. In addition to that, the recent propositions regarding oscillating activation functions have been dealt with.