Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses

Abstract

In this paper I explore the robustness of the Multi-Task Deep Neural Networks (MT-DNN) againstnon-targeted adversarial attacks across Natural Language Understanding (NLU) tasks as well assome possible ways to defend against them. Liu et al., have shown that the Multi-Task Deep NeuralNetwork, due to the regularization effect produced when training as a result of it’s cross task data, ismore robust than a vanilla BERT model trained only on one task (1.1%-1.5% absolute difference).I then show that although the MT-DNN has generalized better, making it easily transferable acrossdomains and tasks, it can still be compromised as after only 2 attacks (1-character and 2-character)the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail tasks. Finally I propose a domainadaptable defense which restores the model’s accuracy (36.75% and 25.94% respectively) as opposedto a general purpose defense or an off-the-shelf spell checker.