Evaluation of Oscillatory Activation Functions on Retinal Fundus Multi-disease Image Dataset (RFMiD)

Abstract

Recent studies in biological neurons have suggested existence of oscillatory activation functions, capable of solving the XOR problem individually. The mathematical representations of these functions have shown promising results on vision tasks when compared to the commonly used activations such as Swish, Mish and varaints of PReLU. Although many open source datasets provide good insights on their performances, specific use cases of these functions in architectures to solve domain specific problem with their relevant metrics is not covered. Medical image datasets often provide a valuable convention as the scope for error in the results is expected to be low, and they have specific criterions which needs to be improved as per the problem at hand. The common neural network backbones could be modified for hyperparameterizing the activation functions used in them to measure the capacity in which these new oscillating activation functions can be used in these datasets. In our task, we evaluate the performance of a family of activation functions on Retinal Fundus Multi-disease Image Dataset (RFMiD), which is a multi-label dataset for 46 conditions, using VGG, DesNet, MobileNet and EfficientNet. We also observe the performances on the subsets of these labels for better benchmarking. Based on our empirical evaluation we see that leveraging oscillatory activation functions allows a performance gain which showcases x percent improvement over best performing montonic non-linear activation functions like ReLU, Swish, Mish and could potentially enable generalised models for retinal screening that can help in classifying a number of sight threatening pathologies.