At the AIT Austrian Institute of Technology, Media Intelligence tools are being developed to support humans in better identifying DeepFakes and manipulated or generated media content.
The research field “Media Intelligence” in the context of computer vision is concerned with the development of algorithms and technologies that make it possible to analyze and interpret multimedia content such as images, videos, audio files and associated texts, and to make the information and results obtained understandable.
The use of computer vision techniques in this field is developing methods for automatically recognizing and classifying objects and faces in images and videos, for example. In addition, these methods are used to evaluate the authenticity of multimedia content based on various indicators (e.g. manipulation of image/sound, detection of deepfakes or synthetic media).
In addition to the analysis of audiovisual characteristics, methods of Natural Language Processing (NLP) and Explainable AI (XAI) are also used. These enable the automatic classification and evaluation of text-based content that is linked to multimedia content. For example, image and video subtitles, social media posts, comments or metadata can be analyzed to draw conclusions about the plausibility, origin and possible manipulation or contradictions of content. By combining computer vision, NLP and XAI, multimodal misinformation, for example, can be better recognized, understood and contextualized.
In addition, methods for context enrichment are used to compare media content with additional information (e.g. place of recording, temporal inconsistencies, source information, etc.). This enables a well-founded check for disinformation or targeted manipulation.
The modular and scalable design of the methods makes it possible to process large data collections efficiently. This is particularly relevant for applications in the fields of digital forensics, combating disinformation, terrorism prevention, open source intelligence (OSINT) and supports the analysis of security-critical media content by law enforcement and security authorities.
Goals
- Development of algorithms for the automatic analysis and interpretation of multimedia content
- Assessing the authenticity of multimedia content (e.g. detecting manipulation and deepfakes)
- Using NLP and XAI techniques to make results more understandable
- Identifying misinformation through multimodal analysis
- Context enrichment for better verification of media content
- Efficient processing of large data collections through modular and scalable methods
Application Domains
- digital media forensics
- Combating disinformation
- Terrorism prevention
- Open-source intelligence (OSINT)
- Law enforcement and security authorities
- Analysis of security-critical media content
Links
