This study builds upon the DetectGPT research paper by extending the application of the papers algorithm to new datasets domains, including Medical Research, E-Commerce Product Reviews, and Book Summaries.
Additionally, overall transparency was enhanced by thoroughly documenting the original algorithm, sharing insights on execution, and explaining how associated hyperparameters were fine-tuned.
Research • Deep Learning • NLP • Data Preprocessing
Dataset Characteristics: Out of the three datasets, the Amazon Reviews dataset performed particularly poorly. From analysis, this is likely due to the significantly short, simple and generic nature of this domain, meaning any text of this nature will make AI detection much less accurate.
Prompt Engineering: With the advent of advanced prompt engineering, it’s evident that conventional AI text detection methods will be weak in this domain. This is another point of future exploration.
Reporting the accuracy of the model when trained with datasets on new dataset domains.
