This study builds upon the DetectGPT research paper by extending the application of the papers algorithm to new datasets domains, including Medical Research, E-Commerce Product Reviews, and Book Summaries.
Additionally, overall transparency was enhanced by thoroughly documenting the original algorithm, sharing insights on execution, and explaining how associated hyperparameters were fine-tuned.
Research • Deep Learning • NLP • Data Preprocessing
Dataset Characteristics: This study looks at what kinds of text are hardest to detect. Short, simple, generic text (such as casual product reviews) tends to perform worse. The weaker results on the Amazon Reviews dataset back this up, showing that detection accuracy drops on this type of content.
Methodology: The method compares original text to several perturbed versions of it. AI-generated text tends to degrade more under these perturbations than human-written text does. This difference is what the detection is based on.
Model: The study uses GPT-2 to generate perturbations. Results could differ with newer models.
Reporting the accuracy of the model when trained with datasets on new dataset domains.
