Ongoing Projects

BIOPROCTORING – Plataforma de Reconocimiento Biométrico y Análisis de Comportamiento durante Evaluaciones Online (2022-2024)

Objectives:

• Design and delepment of a multimodal biometric and behavioral plataform for online courses. The platform will include face recognition technologies, keystroke dynamic recognition and behavior models o improve the thrustworthy and security during online evaluations.

INSPIRA-CM – Identificación de Mecanismos, Biomarcadores e Intervenciones mediante abordajes Computacionales (2023-2027)

Objectives:

• Design of computational models for the generation of synthetic biomedical data. Within this objective, techniques will be developed to automatically generate realistic and balanced data sets. Advanced machine learning techniques (e.g., adversarial generative architectures and/or deep reinforcement learning) supported by expert knowledge of the consortium will be used. The goal is to be able to understand the natural processes involved in data related to EPOC and AOS, and then model those processes through automatic algorithms and generate new data with properties similar to real ones..
• Development of image analysis tools in infrared spectrum to support biomarker detection in EPOC and AOS. The use of Computer Vision technologies is proposed for the enhancement and extraction of patterns related to brown adipose tissue images. The developments will be supported by the great advances in this field, dominated in recent years by algorithms such as Deep Convolutional Networks and Transformers more recently. Initially, it is proposed to adapt state-of-the-art models to the infrared domain, for which auxiliary data will be used that allow pre-adaptation and later specific adjustment to adipose tissue images based on the expert knowledge of the consortium.

HumanCAIC: Enhanced Behavioral Biometrics for Human-Centric AI in Context (2022-2024)

Objectives:

• New Human-Centric AI developments based on improved biometrics and behavior understanding. HumanCAIC will advance the state-of-the-art in automatic behavior understanding by: i) exploiting the latest advanced on multimodal fusion of heterogeneous sources of information; ii) incorporating human feedback and human intervention to guide the learning process of automatic decision-making algorithms; and iii) adding context features to improve the domain adaptation of general models to specific applications.
• Developments on privacy-preserving, transparent, and discrimination-aware machine learning technologies. HumanCAIC proposes novel machine learning strategies designed to incorporate Human-Centric requirements as learning objectives of the training processes of behavioral models.
• Development of novel trustworthy Human-AI interfaces designed to reduce the gap between the social science and the AI developments.
• We will evaluate the developments of HumanCAIC in two case studies focused on: i) digital education; and ii) digital health.

BBforTAI: Biometrics and Behavior for Unbiased and Trustworthy AI with Applications (2022-2024)

Objectives:

1. The development of new methods for measuring and combating biases in multimodal AI frameworks. In this sense, the aim is to extend existing bias analysis and explainability studies in AI, as well as to develop bias prevention methods that are general enough to be applied to different data-driven learning architectures regardless of the nature of the data.
2. Develop new methods for improving the trust in multimodal AI systems, by integrating the developments in objective 1 with recent advances in secure and privacy-preserving AI.
3. Develop core technologies for incorporating the main advances in the previous points into the value chain of practical application areas of social importance. In this regard, we propose four case studies based on some pillars of the welfare society: (i) e-learning platforms; (ii) multimodal biometrics for e-health; (iii) equal opportunities in the access to the labour market; and (iv) media and content analytics.
4. The technical cooperation with ELSA and social/human experts to generate new knowledge on the behavior of people in the contexts portraited in BBforTAI. The project aims to contribute to legal developments, standards, and best practices of use of AI systems, as well as analyzing human behavior patterns in the four cases studied in the previous point.

Selected Publications:

• I. Serna, A. Morales, J. Fierrez and N. Obradovich, “Sensitive Loss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learning”, Artificial Intelligence, April 2022. [PDF] [DOI] [Dataset] [Tech]
• R. Daza, D. DeAlcala, A. Morales, R. Tolosana, R. Cobos and J. Fierrez, “ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning”, in AAAI Workshop on Artificial Intelligence for Education (AI4EDU), Vancouver, Canada, February 2022. [PDF] [DOI] [Dataset]
• I. Serna, D. DeAlcala, A. Morales, J. Fierrez and J. Ortega-Garcia, “IFBiD: Inference-Free Bias Detection”, in AAAI Workshop on Artificial Intelligence Safety (SafeAI), CEUR, vol. 3087, Vancouver, Canada, February 2022. [PDF] [DOI] [Dataset] [Code]

TRESPASS: Training in Secure and Privacy-preserving Biometrics (2020-2024)

Objectives:

• To combat rising security challenges, the global market for biometric technologies is growing at a fast pace. It includes all processes used to recognise, authenticate and identify persons based on biological and/or behavioural characteristics.
• The EU-funded TReSPAsS-ETN project will deliver a new type of security protection (through generalised presentation attack detection (PAD) technologies) and privacy preservation (through computationally feasible encryption solutions).
• The TReSPAsS-ETN Marie Sklodowska-Curie early training network will couple specific technical and transferable skills training including entrepreneurship, innovation, creativity, management and communications with secondments to industry.

Selected publications:

• J. Hernandez-Ortega, J. Fierrez, L. F. Gomez, A. Morales, J. L. Gonzalez-de-Suso and F. Zamora-Martinez, “FaceQvec: Vector Quality Assessment for Face Biometrics based on ISO Compliance”, in IEEE/CVF Winter Conf. on Applications of Computer Vision Workshops (WACVw), Waikoloa, HI, USA, January 2022. [PDF] [DOI] [Code] [Tech]
• L. F. Gomez, A. Morales, J. R. Orozco-Arroyave, R. Daza and J. Fierrez, “Improving Parkinson Detection using Dynamic Features from Evoked Expressions in Video”, in IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRw), June 2021, pp. 1562-1570. [PDF] [DOI]
• O. Delgado-Mohatar, R. Tolosana, J. Fierrez and A. Morales, “Blockchain in the Internet of Things: Architectures and Implementation”, in IEEE Conf. on Computers, Software, and Applications (COMPSAC), Madrid, Spain, July 2020. [PDF] [DOI]

Web: https://www.trespass-etn.eu/

AI4Food:  Inteligencia Artificial para la Prevención de Enfermedades Crónicas a través de una Nutrición Personalizada (2021-2024)

Objectives:

• AI4Food will develop a series of enabling technologies to process, analyze and exploit a large number of biometric signals indicative of individual habits, phenotypic and molecular data.
• AI4Food will integrate all this information and develop new machine learning algorithms to generate a paradigm shift in the field of nutritional counselling.
• AI4Food technology will allow a more objective and effective assessment of the individual nutritional status, helping experts to propose changes towards healthier eating habits from general solutions to personalized solutions that are more effective and sustained over time for the prevention of chronic diseases.
• AI4Food will also advance knowledge on three questions using these new technologies: 1) WHICH are the sensor dependent and sensor independent biomarkers that work best for nutritional modelling of human behavior and habits? 2) WHEN, that is, under what circumstances (e.g., user habits, signal quality, context, phenotypic and molecular data), and 3) HOW can we best leverage those signals and context information to improve nutritional recommendations.