A combination of autoregressors and autoencoders using XLNet for sentiment analysis

Abstract

In this paper sentiment analysis has been performed in order to evaluate the performance of XLNet on this particular task. XLNet is rather a ground-breaking network on language understanding which uses the perks of both autoregressive models and autoencoders. While BERT uses autoencoders and Transformers use autoregression, XLNet combines the aforementioned networks’ attributes in order to achieve higher performance in many NLP tasks, such as sentiment analysis, question answering, reading comprehension, natural language understanding etc. In this work we evaluate the XLNet model in several sentiment classification tasks in terms of accuracy and efficiency. The XLNet reaches state of the art results and outperforms BERT which is the previous state of the art model on natural language processing.

This was an assignment for the course of Deep learning in PhD program of National Technical Unicersity of Athens

• Team composed of 3 persons
• Runs have been made on HPC-ARIS through batch scripts
• Course grade 10/10 (excellent)
• Full report formatted as a paper in here
• Code for 2 sentiment analysis tasks out of 3 (implemented by the author of this repo) in here
• Data available here