BeCAPTCHA: Bot Detection based on Behavioral Biometrics
The heterogeneous flow of data generated during the interaction with the devices can be used to model human behaviour when interacting with the technology and improve bot detection algorithms. For this, we propose a CAPTCHA method based on the analysis of the information obtained during a normal HCI tasks. We evaluate the method by generating fake samples synthesized with Generative Adversarial Neural Networks and handcrafted methods. Our results suggest the potential of mouse dynamics and mobile sensors to characterize the human behaviour and develop a new generation of CAPTCHAs. The experiments are evaluated with teo databases: 1) HuMIdb (Human Mobile Interaction database), a novel multimodal mobile database collected for this work that comprises 14 mobile sensors acquired from 600 users; 2) BeCAPTCHA-Mouse database which comprises more than 10K synthetic and real mouse trayectories.
edBB: Biometrics and Behavior for Assessing Remote Education
We present a platform for student monitoring in remote education consisting of a collection of sensors and software that capture biometric and behavioral data. We define a collection of tasks to acquire behavioral data that can be useful for facing the existing challenges in automatic student monitoring during remote evaluation. Additionally, we release an initial database including data from 38 different users completing these tasks with a set of basic sensors: webcam, microphone, mouse, and keyboard; and also from more advanced sensors: NIR camera, smartwatch, additional RGB cameras, and an EEG band. Information from the computer (e.g. system logs, MAC, IP, or web browsing history) is also stored. During each acquisition session each user completed three different types of tasks generating data of different nature: mouse and keystroke dynamics, face data, and audio data among others. The tasks have been designed with two main goals in mind: i) analyse the capacity of such biometric and behavioral data for detecting anomalies during remote evaluation, and ii) study the capability of these data, i.e. EEG, ECG, or NIR video, for estimating other information about the users such as their attention level, the presence of stress, or their pulse rate
eHealth: Characterization of the Handwriting Skills
as a Biomarker for Parkinson’s Disease
Parkinson’s disease (PD) is a neurodegenerative disorder that occurs due to loss of dopamine, a neurotransmitter that helps in regulating muscle movements. The disease is chronic and progressive, and affects multiple areas of the central nervous system. PD is characterized by alterations of the motor system such as bradykinesia, resting tremor, muscular rigidity, and posture. Handwriting analysis offers the possibility to assess and monitor those motor skills of PD patients. Different abnormal behaviors in handwriting are observed in PD patients. For instance, micrographia occurs in 5% of the patients before other motor symptoms appear, and about 30% of the handwriting worsening cases are reported after the medical diagnosis. Handwriting tasks have significant advantages: they are simple, less intrusive, natural, do not need specialized infrastructure and can be administered remotely
Selected publications on HCI topic:
A. Acien, A. Morales, J. Fierrez, R. Vera-Rodriguez, O. Delgado-Mohatar, “BeCAPTCHA: Behavioral Bot Detection using Touchscreen and Mobile Sensors benchmarked on HuMIdb“, Engineering Applications of Artificial Intelligence, (in press) 2021. [pdf]
Luis F. Gomez-Gomez, A. Morales, J. Fierrez, J. R. Orozco-Arroyave, “Exploring Facial Expressions and Affective Domains for Parkinson Detection,” arXiv:2012.06563. [pdf]
J. Hernandez-Ortega, R. Daza, A. Morales, J. Fierrez and R. Tolosana, “Heart Rate Estimation from Face Videos for Student Assessment: Experiments on edBB,” Proc. of IEEE Computers, Software, and Applications Conference (COMPSAC), Madrid, Spain, July 2020. [pdf]
J. Hernandez-Ortega, R. Daza, A. Morales, J. Fierrez, J. Ortega-Garcia, “edBB: Biometrics and Behavior for Assessing Remote Education,” Proc. of AAAI Workshop on Artificial Intelligence for Education (AI4EDU), New York, USA, 2020. [pdf][GitHub]
A. Acien, A. Morales, J. Fierrez, R. Vera-Rodriguez, I. Bartolome, “BeCAPTCHA: Detecting Human Behavior in Smartphone Interaction using Multiple Inbuilt Sensors,” Proc. of AAAI Workshop on Artificial for Cyber Security (AICS), New York, USA, 2020. [pdf]
A. Acien, A. Morales, J. Fierrez, R. Vera-Rodriguez, “BeCAPTCHA-Mouse: Synthetic Mouse Trajectories and Improved Bot Detection,” arXiv:2005.00890, 2020. [pdf][GitHub]
A. Acien, A. Morales, J. Fierrez, R. Vera Rodriguez and J. Hernandez-Ortega, “Active Detection of Age Groups Based on Touch Interaction“, IET Biometrics, vol. 8, n. 1, pp. 101-108, January 2019. [pdf]
R. Castrillon, A. Acien, J. Orozco-Arroyave, A. Morales, J. Vargas, R.Vera-Rodrıguez, J. Fierrez, J. Ortega-Garcia and A. Villegas, “Characterization of the Handwriting Skills as a Biomarker for Parkinson Disease“, Proc. of IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) – Human Health Monitoring Based on Computer Vision, Lille, France, April 2019.
J. Hernandez-Ortega, J. Fierrez, A. Morales, D. Diaz, “A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos,” Proc. of IEEE Intl. Workshop on Medical Computing (MediComp), Madrid, Spain, 2020. [pdf]
J Hernandez-Ortega, S Nagae, J Fierrez, A Morales, “Quality-Based Pulse Estimation from NIR Face Video with Application to Driver Monitoring,” Proc. of Iberian Conference on Pattern Recognition and Image Analysis, Madrid, Spain, pp. 108-119, 2019.
R. Vera-Rodriguez, R. Tolosana, J. Hernandez-Ortega, A. Acien, A. Morales, J. Fierrez and J. Ortega-Garcia, “Modeling the Complexity of Signature and Touch-Screen Biometrics using the Lognormality Principle“, Rejean Plamondon and Angelo Marcelli and Miguel A. Ferrer (Eds.) in The Lognormality Principle and its Applications, World Scientific, 2019.
J. Hernandez-Ortega, J. Fierrez, A. Morales and P. Tome, “Time Analysis of Pulse-based Face Anti-Spoofing in Visible and NIR“, in Proc. IEEE Conf. on Computer Vision and Pattern Recognition Workshops, Salt Lake City, USA, July 2018.
A. Morales, F. M. Costela, R. Tolosana and R. L. Woods, “Saccade Landing Point Prediction: A Novel Approach based on Recurrent Neural Networks“, in Proc. of International Conference on Machine Learning Technologies (ICMLT), Jinan, China, May 2018 (Best Presentation Award).
J. Hernandez-Ortega, A. Morales, J. Fierrez and A. Acien, “Detecting Age Groups using Touch Interaction based on Neuromotor Characteristics“, IET Electronics Letters, vol. 20, n. 53, pp. 1349–1350, September 2017.
A. Morales, J. Fierrez, R. Vera-Rodriguez and J. Ortega-Garcia, “Biometric Student Authentication for e-Learning Platforms“, in Proc. Congreso Internacional sobre Aprendizaje, Innovacion y Competitividad (CINAIC), pp. 371-376, Madrid (Spain), October 2015.
Aythami Morales 