Review article

GPT (Generative Pre-Trained Transformer)— A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

Gokul YenduriSchool of Computer Science and Engineering, VIT-AP University, Amaravati, Andhra Pradesh, IndiaM. RamalingamSchool of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, IndiaG. Chemmalar SelviSchool of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, IndiaY. SupriyaSchool of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, IndiaGautam SrivastavaDepartment of Mathematics and Computer Science, Brandon University, Brandon, MB, CanadaPraveen Kumar Reddy MaddikuntaSchool of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, IndiaG. Deepti RajSchool of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, IndiaRutvij H. JhaveriDepartment of Computer Science and Engineering, School of Technology, Pandit Deendayal Energy University, Gandhinagar, IndiaB. PrabadeviSchool of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, IndiaWeizheng WangDepartment of Computer Science, City University of Hong Kong, Hong Kong, SAR, ChinaAthanasios V. VasilakosCenter for AI Research (CAIR), University of Agder (UiA), Grimstad, NorwayThippa Reddy GadekalluCenter of Research Impact and Outcome, Chitkara University, Rajpura, Punjab, India

2024en

ABI

Abstract

The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward the development of machines that can understand and communicate using language in a manner that closely resembles that of humans. GPT is based on the transformer architecture, a deep neural network designed for natural language processing tasks. Due to their impressive performance on natural language processing tasks and ability to effectively converse, GPT have gained significant popularity among researchers and industrial communities, making them one of the most widely used and effective models in natural language processing and related fields, which motivated to conduct this review. This review provides a detailed overview of the GPT, including its architecture, working process, training procedures, enabling technologies, and its impact on various applications. In this review, we also explored the potential challenges and limitations of a GPT. Furthermore, we discuss potential solutions and future directions. Overall, this paper aims to provide a comprehensive understanding of GPT, its enabling technologies, their impact on various applications, emerging challenges, and potential solutions.

Identifiers

DOI: 10.1109/access.2024.3389497

Citations and references

Cited by 20 references

Metrics — AkademScholar