Cryptocurrency Price Estimation Using Hyperparameterized Oscillatory Activation Functions in LSTM Networks

Abstract

Activation functions are critical components of neural networks, helping the model learn highly-intricate dependencies, trends, and patterns. Non-linear activation functions allow the model to behave as a functional approximator, learning complex decision boundaries and multi-dimensional patterns in the data. Activation functions can be combined with one another to learn better representations with the objective of improving gradient flow, performance metrics reducing training time and computational cost. Recent work on oscillatory activation functions\cite{noel2021growing}\cite{noel2021biologically} showcased their ability to perform competitively on image classification tasks using a compact architecture. Our work proposes the utilization of these oscillatory activation functions for predicting the volume-weighted average of Bitcoin on the G-Research Cryptocurrency Dataset. We utilize a popular LSTM architecture for this task achieving competitive results when compared to popular activation functions formally used.