Design and development of face recognition-based security system using expression game as liveness detection
DOI:
https://doi.org/10.31763/businta.v8i2.756Keywords:
Face Recognition, Liveness Detection, Biometric Security, Security System, Face ExpressionAbstract
Face recognition as a security system has undergone significant developments, but challenges in live detection are still a major issue in preventing fraud. Liveness detection is a method that helps face recognition security more resistant to fraud. This research aims to address this issue by developing an innovative security system that integrates face recognition with a facial expression game, enhancing live detection and user engagement. The primary objectives are to ensure seamless integration, maintain a fun and challenging user experience, and demonstrate practical applicability. We applied a Waterfall method in our research to ensure a straightforward approach. We successfully applied this system for the door lock-unlock mechanism, simulating a restricted area. YuNet, a face detection model runs in the web interface and controls the NodeMCU to either lock or unlock the door. The study concluded 95% success rate from the participants in making facial expressions: Smile, Normal, and Sad. However, expressing Sadness within the 3-second timeframe posed some difficulties. The average duration for completing the mini-game was approximately 16.31 seconds from the start. The integration of a facial expression game as a liveness detection required careful design to balance security and user engagement that is fun to experience. This research underscores the significance of addressing current challenges in biometric security by integrating an interactive element into the live detection process. The developed system contributes to the field by enhancing the robustness and user experience of face recognition security systems, demonstrating potential for broader application in restricted access scenarios.
References
M. Wang and W. Deng, “Deep face recognition: A survey,” Neurocomputing, vol. 429, pp. 215–244, 2021, doi: 10.1016/j.neucom.2020.10.081.
A. P. Wibawa, W. A. Yudha Pratama, A. N. Handayani, and A. Ghosh, “Convolutional Neural Network (CNN) to determine the character of wayang kulit,” Int. J. Vis. Perform. Arts, vol. 3, no. 1, pp. 1–8, Jun. 2021, doi: 10.31763/viperarts.v3i1.373.
S. B. R. Prasad and B. S. Chandana, “Mobilenetv3: a deep learning technique for human face expressions identification,” Int. J. Inf. Technol., vol. 15, no. 6, pp. 3229–3243, 2023, doi: 10.1007/s41870-023-01380-x.
M. Ngan, P. Grother, and K. Hanaoka, “Ongoing Face Recognition Vendor Test (FRVT) part 6A:,” National Institute of Standards and Technology, Gaithersburg, MD, Jul. 2020. doi: 10.6028/NIST.IR.8311.
S. Khairnar, S. Gite, K. Kotecha, and S. D. Thepade, “Face Liveness Detection Using Artificial Intelligence Techniques: A Systematic Literature Review and Future Directions,” Big Data Cogn. Comput., vol. 7, no. 1, p. 37, Feb. 2023, doi: 10.3390/bdcc7010037.
A. P. Wibawa et al., “Decoding and preserving Indonesia’s iconic Keris via A CNN-based classification,” Telemat. Informatics Reports, vol. 13, p. 100120, 2024, doi: 10.1016/j.teler.2024.100120.
R. Koshy and A. Mahmood, “Enhanced Deep Learning Architectures for Face Liveness Detection for Static and Video Sequences,” Entropy, vol. 22, no. 10, p. 1186, Oct. 2020, doi: 10.3390/e22101186.
W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recognition: A literature survey,” ACM Comput. Surv., vol. 35, no. 4, pp. 399–458, 2003, doi: 10.1145/954339.954342.
D. Srivastava, P. Shukla, and A. K. Sahani, “Face Verification System with Liveness Detection,” in 2021 IEEE 18th India Council International Conference (INDICON), 2021, pp. 1–5, doi: 10.1109/INDICON52576.2021.9691561.
V. V Zolotarev, S. O. Knyazuk, and E. A. Maro, “Liveness Detection Mechanisms to Enhance Robustness of Authentication Methods in Game-Based Educational Services,” in 2020 International Conference Quality Management, Transport and Information Security, Information Technologies (IT&QM&IS), 2020, pp. 566–570, doi: 10.1109/ITQMIS51053.2020.9322889.
M. R. Dronky, W. Khalifa, and M. Roushdy, “A Review on Iris Liveness Detection Techniques,” in 2019 Ninth International Conference on Intelligent Computing and Information Systems (ICICIS), 2019, pp. 48–59, doi: 10.1109/ICICIS46948.2019.9014719.
N. Rogmann and M. Krieg, “Liveness Detection in Biometrics,” in 2015 International Conference of the Biometrics Special Interest Group (BIOSIG), 2015, pp. 1–14, doi: 10.1109/BIOSIG.2015.7314611.
H. A. Rosyid, M. Palmerlee, and K. Chen, “Deploying learning materials to game content for serious education game development: A case study,” Entertain. Comput., vol. 26, pp. 1–9, May 2018, doi: 10.1016/j.entcom.2018.01.001.
H. A. Rosyid, A. Y. Pangestu, and M. I. Akbar, “Can Diegetic User Interface Improves Immersion in Role-Playing Games?,” in 2021 7th International Conference on Electrical, Electronics and Information Engineering (ICEEIE), 2021, pp. 200–204, doi: 10.1109/ICEEIE52663.2021.9616732.
S. M. Khan, Waterfall Model Used in Software Development Reference: Software Requirements Engineering Waterfall Model. pp. 1-3, 2023. [Online]. Available at: https://www.researchgate.net/profile/Sardar-Mudassar-Khan-2/publication/371902449.
U. S. Senarath, “Waterfall methodology, prototyping and agile development,” Tech. Rep., pp. 1–16, 2021, [Online]. Available at: https://www.researchgate.net/profile/Udesh-S-Senarath/publication/353324450.
W. Wu, H. Peng, and S. Yu, “YuNet: A Tiny Millisecond-level Face Detector,” Mach. Intell. Res., vol. 20, no. 5, pp. 656–665, 2023, doi: 10.1007/s11633-023-1423-y.
Make It CA, “NodeMCU ESP8266: Details and Specifications,” Website, 2024.
S. Barai, D. Biswas, and B. Sau, “Estimate distance measurement using NodeMCU ESP8266 based on RSSI technique,” in 2017 IEEE Conference on Antenna Measurements & Applications (CAMA), 2017, pp. 170–173, doi: 10.1109/CAMA.2017.8273392.
M. M. Hasan, M. S. U. Yusuf, T. I. Rohan, and S. Roy, “Efficient two stage approach to detect face liveness : Motion based and Deep learning based,” in 2019 4th International Conference on Electrical Information and Communication Technology (EICT), 2019, pp. 1–6, doi: 10.1109/EICT48899.2019.9068813.
H. A. Rosyid, S. Patmanthara, and I. R. Cahyudi, Game development. Ahlimedia Book, 2021.
I. Patidar, K. S. Modh, and C. Chattopadhyay, “Artificially Intelligent Game Framework Based on Facial Expression Recognition BT - Computer Vision, Pattern Recognition, Image Processing, and Graphics,” 2020, pp. 312–321, doi: 10.1007/978-981-15-8697-2_29.
Z. Aghababaeyan, M. Abdellatif, L. Briand, S. R., and M. Bagherzadeh, “Black-Box Testing of Deep Neural Networks through Test Case Diversity,” IEEE Trans. Softw. Eng., vol. 49, no. 5, pp. 3182–3204, 2023, doi: 10.1109/TSE.2023.3243522.
I. Bhatti, J. A. Siddiqi, A. Moiz, and Z. A. Memon, “Towards Ad hoc Testing Technique Effectiveness in Software Testing Life Cycle,” in 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), 2019, pp. 1–6, doi: 10.1109/ICOMET.2019.8673390.
E. Alégroth, M. Nass, and H. H. Olsson, “JAutomate: A Tool for System- and Acceptance-test Automation,” in 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, 2013, pp. 439–446, doi: 10.1109/ICST.2013.61.
A. Sadaj, M. Ochodek, S. Kopczyńska, and J. Nawrocki, “Maintainability of Automatic Acceptance Tests for Web Applications—A Case Study Comparing Two Approaches to Organizing Code of Test Cases BT - SOFSEM 2020: Theory and Practice of Computer Science,” 2020, pp. 454–466, doi: 10.1007/978-3-030-38919-2_37.
D. Garlan and M. Shaw, “An introduction to software architecture,” in Advances in software engineering and knowledge engineering, World Scientific, 1993, pp. 1–39, doi: 10.1142/9789812798039_0001.
O. E. Amestica, P. E. Melin, C. R. Duran-Faundez, and G. R. Lagos, “An Experimental Comparison of Arduino IDE Compatible Platforms for Digital Control and Data Acquisition Applications,” in 2019 IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), 2019, pp. 1–6, doi: 10.1109/CHILECON47746.2019.8986865.
M. Grinberg, Flask web development, 2nd ed. “ O’Reilly Media, Inc.,” p. 30, 2018. [Online]. Available at: https://www.oreilly.com/library/view/flask-web-development/9781491991725/.
J. Hunt, “Flask Web Services,” in Advanced Guide to Python 3 Programming, Springer, 2023, pp. 575–581, doi: 10.1007/978-3-031-40336-1_51.
K. P., D. M., L. B. M., and G. N., “Design and Development of NodeMCU Based Smart IoT Door System,” in 2023 4th International Conference on Smart Electronics and Communication (ICOSEC), 2023, pp. 398–403, doi: 10.1109/ICOSEC58147.2023.10275975.
Y. Sun, X. Wang, and X. Tang, “Deep learning face representation from predicting 10,000 classes,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1891–1898, doi: 10.1109/CVPR.2014.244.
S. Li and W. Deng, “Deep facial expression recognition: A survey,” IEEE Trans. Affect. Comput., vol. 13, no. 3, pp. 1195–1215, 2020, doi: 10.1109/TAFFC.2020.2981446.
V. M. Patel, R. Chellappa, D. Chandra, and B. Barbello, “Continuous user authentication on mobile devices: Recent progress and remaining challenges,” IEEE Signal Process. Mag., vol. 33, no. 4, pp. 49–61, 2016, doi: 10.1109/MSP.2016.2555335.
S. Bhattacharya, G. S. Nainala, P. Das, and A. Routray, “Smart attendance monitoring system (SAMS): a face recognition based attendance system for classroom environment,” in 2018 IEEE 18th international conference on advanced learning technologies (ICALT), 2018, pp. 358–360, doi: 10.1109/ICALT.2018.00090.
C. Rathgeb, A. Dantcheva, and C. Busch, “Impact and detection of facial beautification in face recognition: An overview,” IEEE Access, vol. 7, pp. 152667–152678, 2019, doi: 10.1109/ACCESS.2019.2948526.
S. Rao, Y. Huang, K. Cui, and Y. Li, “Anti-spoofing face recognition using a metasurface-based snapshot hyperspectral image sensor,” Optica, vol. 9, no. 11, pp. 1253–1259, 2022, doi: 10.1364/OPTICA.469653.
J. Akati and M. Conrad, “Anti-Tailgating Solution Using Biometric Authentication, Motion Sensors and Image Recognition,” in 2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), 2021, pp. 825–830, doi: 10.1109/DASC-PICom-CBDCom-CyberSciTech52372.2021.00137.
M. O. Oloyede, G. P. Hancke, and N. Kapileswar, “Evaluating the effect of occlusion in face recognition systems,” in 2017 IEEE AFRICON, 2017, pp. 1547–1551, doi: 10.1109/AFRCON.2017.8095712
