While AI and here most often machine learning (ML) becomes more common as a tool for various security applications where data must be analysed, clustered or attributed, the security of the applied algorithms is often limited. Various research results in the past years show shortcomings of trained neural nets like robustness against targeted attacks. The risk of privacy loss is also discussed in the context of machine learning – potential leakage from trained neural networks as well as de-anonymization in complex data sets with the help of machine learning.

This leads to a lack of trust and acceptance by the public: ML is perceived as a risk and a threat, a mechanism to screen large parts of everyday life beyond control. Under these circumstances, using the potential of ML for security solutions and other sensitive applications becomes challenging.

In IT security the usage of ML is already established in multiple domains today. SPAM detection is a well-known example where support vector machines try to distinguish wanted from unwanted emails. Author attribution combines natural language forensics and machine learning. Deep learning helps identifying illicit images and improved malware detection as well as network intrusion detection.

Today ML algorithms and trained nets become the target of attacks. Various approaches try to mislead or influence ML-based decisions, requiring countermeasures of IT security protecting the core assets of ML.

ML also becomes a tool of attackers. IT security needs to be prepared for attacks able to adapt more quickly to complex security measures; just like intrusion detection systems aim to identify complex attacks with the help of ML.

Adversarial machine learning will become more common in IT security. Whenever a security challenge can be described on the one hand with a relatively simple concept and on the other hand can be addressed by machine learning, the other side, be it defender or attacker, will use adversarial machine learning to efficiently identify weaknesses in the strategy of the other partner and deploy specialized attacks or defenses against it.

SenPAI addresses the subject of security and ML from two perspectives: The first perspective is improving the security of ML algorithms and systems based on ML. This does not include standard system security, which is a generic requirement for all IT systems. The focus is on security challenges that are especially and in some cases exclusively relevant to ML. The term "security" has to be seen in a broad sense here, as issues like privacy leakage or transparency of decisions shall also be addressed.

The second perspective is application-centric. As the National Center of applied Cybersecurity Research ATHENE has a focus on applied security solutions, SenPAI aims to develope and evaluate new security applications based on ML. These applications can and shall also utilize the security mechanisms developed in the technology-centric research projects and give a feedback on their usability and performance.

The application-centric projects may raise new challenges for the technology-centric projects. These projects will also potentially be Big Data projects due to their handling of complex data and their aim to derive efficiently information relevant for security matters from it.

The technology-centric projects will focus on research publications and PhD theses. Tools will be developed and implemented as well. The application-centric projects will focus on building demonstrators and discussing them with the public, governmental organizations and the industry.

In recent years, AI has experienced rapid growth in performance and fields of application. However, there are still many challenges that have not been addressed by exhaustive solutions. Within SenPAI, the following aspects of AI and security are potential research goals: