Thu 24 Jun 2021 01:35 - 01:40 at PLDI-A - Talks 2A: Machine Learning
We present DeepT, a novel method for certifying Transformer networks based on abstract interpretation. The key idea behind DeepT is our new Multi-norm Zonotope abstract domain, an extension of the classical Zonotope designed to handle $\ell^1$ and $\ell^2$-norm bound perturbations. We introduce all Multi-norm Zonotope abstract transformers necessary to handle these complex networks, including the challenging softmax function and dot product. Our evaluation shows that DeepT can certify average robustness radii that are $28\times$ larger than the state-of-the-art, while scaling favorably. Further, for the first time, we certify Transformers against synonym attacks on long sequences of words, where each word can be replaced by any synonym. DeepT achieves a high certification success rate on sequences of words where enumeration-based verification would take 2 to 3 orders of magnitude more time.
Wed 23 JunDisplayed time zone: Eastern Time (US & Canada) change
13:30 - 14:05 | |||
13:30 5mTalk | Learning to Find Naming Issues with Big Code and Small Supervision PLDI DOI | ||
13:35 5mTalk | Fast and Precise Certification of Transformers PLDI Gregory Bonaert ETH Zurich, Dimitar I. Dimitrov ETH Zurich, Maximilian Baader ETH Zurich, Martin Vechev ETH Zurich DOI | ||
13:40 5mTalk | Web Question Answering with Neurosymbolic Program Synthesis PLDI Qiaochu Chen University of Texas at Austin, USA, Aaron Lamoreaux University of Texas at Austin, Xinyu Wang University of Michigan, Greg Durrett University of Texas at Austin, USA, Osbert Bastani University of Pennsylvania, Işil Dillig University of Texas at Austin DOI | ||
13:45 5mTalk | Robustness Certification with Generative Models PLDI Matthew Mirman ETH Zurich, Alexander Hägele ETH Zurich, Timon Gehr ETH Zurich, Pavol Bielik ETH Zurich, Martin Vechev ETH Zurich Link to publication DOI | ||
13:50 5mTalk | DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion PLDI Wei Niu College of William & Mary, Jiexiong Guan College of William & Mary, Yanzhi Wang Northeastern University, Gagan Agrawal Augusta University, Bin Ren College of William & Mary DOI | ||
13:55 5mTalk | Vectorized Secure Evaluation of Decision Forests PLDI Raghav Malik Purdue University, Vidush Singhal Purdue University, Benjamin Gottfried Purdue University, Milind Kulkarni Purdue University DOI Pre-print | ||
14:00 5mTalk | AKG: Automatic Kernel Generation for Neural Processing Units using Polyhedral Transformations PLDI Jie Zhao State Key Laboratory of Mathematical Engineering and Advanced Computing, Bojie Li Huawei Technologies, Wang Nie Huawei Technologies, Zhen Geng Huawei Technologies, Renwei Zhang Huawei Technologies, Xiong Gao Huawei Technologies, Bin Cheng Huawei Technologies, Chen Wu Huawei, Yun Cheng Huawei Technologies, Zheng Li Huawei Technologies, Peng Di Huawei Technologies, Kun Zhang Huawei Technologies, Xuefeng Jin Huawei Technologies DOI |
Thu 24 JunDisplayed time zone: Eastern Time (US & Canada) change
01:30 - 02:05 | |||
01:30 5mTalk | Learning to Find Naming Issues with Big Code and Small Supervision PLDI DOI | ||
01:35 5mTalk | Fast and Precise Certification of Transformers PLDI Gregory Bonaert ETH Zurich, Dimitar I. Dimitrov ETH Zurich, Maximilian Baader ETH Zurich, Martin Vechev ETH Zurich DOI | ||
01:40 5mTalk | Web Question Answering with Neurosymbolic Program Synthesis PLDI Qiaochu Chen University of Texas at Austin, USA, Aaron Lamoreaux University of Texas at Austin, Xinyu Wang University of Michigan, Greg Durrett University of Texas at Austin, USA, Osbert Bastani University of Pennsylvania, Işil Dillig University of Texas at Austin DOI | ||
01:45 5mTalk | Robustness Certification with Generative Models PLDI Matthew Mirman ETH Zurich, Alexander Hägele ETH Zurich, Timon Gehr ETH Zurich, Pavol Bielik ETH Zurich, Martin Vechev ETH Zurich Link to publication DOI | ||
01:50 5mTalk | DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion PLDI Wei Niu College of William & Mary, Jiexiong Guan College of William & Mary, Yanzhi Wang Northeastern University, Gagan Agrawal Augusta University, Bin Ren College of William & Mary DOI | ||
01:55 5mTalk | Vectorized Secure Evaluation of Decision Forests PLDI Raghav Malik Purdue University, Vidush Singhal Purdue University, Benjamin Gottfried Purdue University, Milind Kulkarni Purdue University DOI Pre-print | ||
02:00 5mTalk | AKG: Automatic Kernel Generation for Neural Processing Units using Polyhedral Transformations PLDI Jie Zhao State Key Laboratory of Mathematical Engineering and Advanced Computing, Bojie Li Huawei Technologies, Wang Nie Huawei Technologies, Zhen Geng Huawei Technologies, Renwei Zhang Huawei Technologies, Xiong Gao Huawei Technologies, Bin Cheng Huawei Technologies, Chen Wu Huawei, Yun Cheng Huawei Technologies, Zheng Li Huawei Technologies, Peng Di Huawei Technologies, Kun Zhang Huawei Technologies, Xuefeng Jin Huawei Technologies DOI |