On April 14 2021, I gave an invited talk at the UVA Human and Machine Intelligence Seminar:
Scope of problems our tools aim to tackle
Classifiers based on machine learning algorithms have shown promising results for many security tasks including malware classification and network intrusion detection, but classic machine learning algorithms are not designed to operate in the presence of adversaries. Intelligent and adaptive adversaries may actively manipulate the information they present in attempts to evade a trained classifier, leading to a competition between the designers of learning systems and attackers who wish to evade them. This project is developing automated techniques for predicting how well classifiers will resist the evasions of adversaries, along with general methods to automatically harden machine-learning classifiers against adversarial evasion attacks.
At the junction between machine learning and computer security, this project involves toolboxes for five main task as shown in the following table. Our system aims to allow a classifier designer to understand how the classification performance of a model degrades under evasion attacks, enabling better-informed and more secure design choices. The framework is general and scalable, and takes advantage of the latest advances in machine learning and computer security.
We categorize the topics into a list of subtasks and list our selected works in the following table:
|1||Evade NLP Machine Learning||[TextAttack: A Framework for Adversarial Attacks in Natural Language Processing]||EMNLP2020||GitHub|
|2||Evade Machine Learning||[Automatically Evading Classifiers, Case Study on PDF Malware Classifiers]||NDSS16||GitHub|
|3||Evade NLP Machine Learning||[Black-box Generation of Adversarial Text Sequences to Fool Deep Learning Classifiers]||DeepSecureWkp18||GitHub|
|4||Detect Adversarial Attacks||[Feature Squeezing- Detecting Adversarial Examples in Deep Neural Networks]||NDSS18||GitHub|
|5||Defense against Adversarial Attacks||[DeepCloak- Masking Deep Neural Network Models for Robustness against Adversarial Samples]||ICLRwkp17||GitHub|
|6||Visualize Adversarial Attacks||[Adversarial-Playground- A Visualization Suite for Adversarial Samples]||VizSec17||GitHub|
|7||Theorems of Adversarial Examples||[A Theoretical Framework for Robustness of (Deep) Classifiers Against Adversarial Samples]||ICLRw17|
|8||Trustworthy via Interpretation||[Deep Motif Dashboard]||ICLRw2017|
Thanks for reading!
Title: Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples
Title: Reevaluating Adversarial Examples in Natural Language
On April 23 2019, I gave an invited talk at the ARO Invitational Workshop on Foundations of Autonomous Adaptive Cyber Systems
On December 21 @ 12noon, I gave a distinguished webinar talk in the Fall 2018 webinar series of the Institute for Information Infrastructure Protection (I3P)...
We are releasing EvadeML-Zoo: A Benchmarking and Visualization Tool for Adversarial Examples (with 8 pretrained deep models+ 9 state-of-art attacks).
Jack’s DeepMotif paper (Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks ) have received the “best paper awar...
Tool Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks
Paper ICLR17 Workshop
Paper ICLR17 workshop