Evaluating deepfake detection models: A comprehensive framework for comparison across diverse datasets
Loading...
Supplementary material
Other Title
Authors
Mangotra, Babita Dasoar
Author ORCID Profiles (clickable)
Degree
Master of Applied Technologies (Computing)
Grantor
Unitec, Te Pūkenga – New Zealand Institute of Skills and Technology
Date
2025
Supervisors
Varastehpour, Soheil
Shakiba, Masoud
Shakiba, Masoud
Type
Masters Thesis
Ngā Upoko Tukutuku (Māori subject headings)
Keyword
deepfakes
detection
modelling
image manipulation
fake videos
detection
modelling
image manipulation
fake videos
ANZSRC Field of Research Code (2020)
Citation
Mangotra, B.D. (2025). Evaluating deepfake detection models: A comprehensive framework for comparison across diverse datasets (Unpublished document submitted in partial fulfilment of the requirements for the degree of Master of Applied Technologies (Computing)). Unitec, Te Pūkenga - New Zealand Institute of Skills and Technology
https://hdl.handle.net/10652/6799
Abstract
Rapid advancements in deepfake technology pose serious threats to public confidence in digital material, cybersecurity, and the ability to authenticate media. Deepfake movies have gotten progressively realistic with new artificial intelligence developments, especially in Generative ANs and autoencoders, which makes detection more difficult. By creating and assessing ML models for recognizing deep fakes and using several datasets to improve generalizability and robustness, our work seeks to solve these difficulties.
Celeb-DF V2, FaceForensics++, CIPLib, Deepfake Detection Challenge Dataset, as well as Fake Celeb AV-V1.2 are among the datasets this work uses. Every dataset is chosen depending on their variance in video alteration techniques, resolution changes, and source authenticity to guarantee a thorough assessment of machine learning models among several deepfake generating approaches. Steps in data preparation augmentation, normalisation, feature extraction are used to improve model training's efficacy.
Then we investigate several deepfake detection models: Forensics Convolutional Neural Networks, Vision Transformers, Masked Autoencoders, Two-Stream Neural Network as well as Common Fake Feature Network in a comparative manner. Every model is evaluated in relation to computing efficiency, generalizing across datasets, and efficacy in reducing detection mistakes. Performance data including validation and training accuracy, loss functions, and error rates including equal error rate, half total error rate, false acceptance rate, and false rejection rate is examined using a standard assessment framework.
While Vision Transformers and Two-Stream Neural Network models show great detection ability, experimental results show that the Common Fake Feature Network model routinely beats others, obtaining almost zero error rates and up to 100% validation accuracy on demanding datasets including Fake Celeb AV-V1.2 and Deepfake Detection Challenge Dataset. Common Fake Feature Network exceptional success can be ascribed to its capacity to use sophisticated feature extraction and classification approaches to capture fine-grained alterations in deepfake films.
The results of this work underline the need of constantly developing machine learning methods to properly fight digital disinformation since they show the need of managing detection accuracy with error minimizing. Furthermore, underlined by this study is the crucial need of strong deep fake identification systems in digital integrity, cybersecurity, and media forensics. This work adds important new perspectives on the continuous battle against deepfake dangers by building a disciplined evaluation approach and contrasting several SotA detecting techniques. Future research will concentrate on increasing model adaptability, computing efficiency, and integration of present time deep fake detection for pragmatic application in digital security systems.
Publisher
Permanent link
Link to ePress publication
DOI
Copyright holder
Author
Copyright notice
All rights reserved