Write a Blog >>
PLDI 2021
Sun 20 - Sat 26 June 2021 Virtual Conference

This program is tentative and subject to change.

Wed 23 Jun 2021 13:45 - 13:50 at PLDI-A - Talks 2A: Machine Learning
Thu 24 Jun 2021 01:45 - 01:50 at PLDI-A - Talks 2A: Machine Learning

Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices. To achieve high accuracy, DNN models have become increasingly deep with hundreds or even thousands of operator layers, leading to high memory and computational requirements for inference. Operator fusion (or kernel/layer fusion) is key optimization in many state-of-the-art DNN execution frameworks, such as TensorFlow, TVM, and MNN, that aim to improve the efficiency of the DNN inference. However, these frameworks usually adopt fusion approaches based on certain patterns that are too restrictive to cover the diversity of operators and layer connections, especially those seen in many extremely deep models. Polyhedral-based loop fusion techniques, on the other hand, work on a low-level view of the computation without operator-level information, and can also miss potential fusion opportunities. To address this challenge, this paper proposes a novel and extensive loop fusion framework called DNNFusion. The basic idea of this work is to work at an operator view of DNNs, but expand fusion opportunities by developing a classification of both individual operators and their combinations. In addition, DNNFusion includes 1) a novel mathematical-property-based graph rewriting framework to reduce evaluation costs and facilitate subsequent operator fusion, 2) an integrated fusion plan generation that leverages the high-level analysis and accurate light-weight profiling, and 3) additional optimizations during fusion code generation. DNNFusion is extensively evaluated on 15 DNN models with varied types of tasks, model sizes, and layer counts. The evaluation results demonstrate that DNNFusion finds up to $8.8 \times$ higher fusion opportunities, outperforms four state-of-the-art DNN execution frameworks with $9.3\times$ speedup. The memory requirement reduction and speedups can enable the execution of many of the target models on mobile devices and even make them part of a real-time application.

This program is tentative and subject to change.

Conference Day
Wed 23 Jun

Displayed time zone: Eastern Time (US & Canada) change

13:30 - 14:05
Talks 2A: Machine LearningPLDI at PLDI-A +12h
13:30
5m
Talk
Learning to Find Naming Issues with Big Code and Small Supervision
PLDI
Jingxuan HeETH Zurich, Cheng-Chun LeeEPFL, Veselin RaychevDeepCode, Martin VechevETH Zurich
DOI
13:35
5m
Talk
Web Question Answering with Neurosymbolic Program Synthesis
PLDI
Qiaochu ChenUniversity of Texas at Austin, USA, Aaron LamoreauxUniversity of Texas at Austin, Xinyu WangUniversity of Michigan, Greg DurrettUniversity of Texas at Austin, USA, Osbert BastaniUniversity of Pennsylvania, Isil DilligUniversity of Texas at Austin
DOI
13:40
5m
Talk
AKG: Automatic Kernel Generation for Neural Processing Units using Polyhedral Transformations
PLDI
Jie ZhaoState Key Laboratory of Mathematical Engineering and Advanced Computing, Bojie LiHuawei Technologies, Wang NieHuawei Technologies, Zhen GengHuawei Technologies, Renwei ZhangHuawei Technologies, Xiong GaoHuawei Technologies, Bin ChengHuawei Technologies, Chen WuHuawei, Yun ChengHuawei Technologies, Zheng LiHuawei Technologies, Peng DiHuawei Technologies, Kun ZhangHuawei Technologies, Xuefeng JinHuawei Technologies
DOI
13:45
5m
Talk
DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion
PLDI
Wei NiuCollege of William & Mary, Jiexiong GuanCollege of William & Mary, Yanzhi WangNortheastern University, Gagan AgrawalAugusta University, Bin RenCollege of William & Mary
DOI
13:50
5m
Talk
Robustness Certification with Generative Models
PLDI
Matthew MirmanETH Zurich, Alexander HägeleETH Zurich, Timon GehrETH Zurich, Pavol BielikETH Zurich, Martin VechevETH Zurich
Link to publication DOI
13:55
5m
Talk
Vectorized Secure Evaluation of Decision Forests
PLDI
Raghav MalikPurdue University, Vidush SinghalPurdue University, Benjamin GottfriedPurdue University, Milind KulkarniPurdue University
DOI Pre-print
14:00
5m
Talk
Fast and Precise Certification of Transformers
PLDI
Gregory BonaertETH Zurich, Dimitar I. DimitrovETH Zurich, Maximilian BaaderETH Zurich, Martin VechevETH Zurich
DOI

Conference Day
Thu 24 Jun

Displayed time zone: Eastern Time (US & Canada) change

01:30 - 02:05
Talks 2A: Machine LearningPLDI at PLDI-A
01:30
5m
Talk
Learning to Find Naming Issues with Big Code and Small Supervision
PLDI
Jingxuan HeETH Zurich, Cheng-Chun LeeEPFL, Veselin RaychevDeepCode, Martin VechevETH Zurich
DOI
01:35
5m
Talk
Web Question Answering with Neurosymbolic Program Synthesis
PLDI
Qiaochu ChenUniversity of Texas at Austin, USA, Aaron LamoreauxUniversity of Texas at Austin, Xinyu WangUniversity of Michigan, Greg DurrettUniversity of Texas at Austin, USA, Osbert BastaniUniversity of Pennsylvania, Isil DilligUniversity of Texas at Austin
DOI
01:40
5m
Talk
AKG: Automatic Kernel Generation for Neural Processing Units using Polyhedral Transformations
PLDI
Jie ZhaoState Key Laboratory of Mathematical Engineering and Advanced Computing, Bojie LiHuawei Technologies, Wang NieHuawei Technologies, Zhen GengHuawei Technologies, Renwei ZhangHuawei Technologies, Xiong GaoHuawei Technologies, Bin ChengHuawei Technologies, Chen WuHuawei, Yun ChengHuawei Technologies, Zheng LiHuawei Technologies, Peng DiHuawei Technologies, Kun ZhangHuawei Technologies, Xuefeng JinHuawei Technologies
DOI
01:45
5m
Talk
DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion
PLDI
Wei NiuCollege of William & Mary, Jiexiong GuanCollege of William & Mary, Yanzhi WangNortheastern University, Gagan AgrawalAugusta University, Bin RenCollege of William & Mary
DOI
01:50
5m
Talk
Robustness Certification with Generative Models
PLDI
Matthew MirmanETH Zurich, Alexander HägeleETH Zurich, Timon GehrETH Zurich, Pavol BielikETH Zurich, Martin VechevETH Zurich
Link to publication DOI
01:55
5m
Talk
Vectorized Secure Evaluation of Decision Forests
PLDI
Raghav MalikPurdue University, Vidush SinghalPurdue University, Benjamin GottfriedPurdue University, Milind KulkarniPurdue University
DOI Pre-print
02:00
5m
Talk
Fast and Precise Certification of Transformers
PLDI
Gregory BonaertETH Zurich, Dimitar I. DimitrovETH Zurich, Maximilian BaaderETH Zurich, Martin VechevETH Zurich
DOI