Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing
Fri 25 Jun 2021 21:15 - 21:20 at PLDI-B - Talks 5B: Defect Detection and Repair
JavaScript (JS) is a popular, platform-independent programming language. To ensure the interoperability of JS programs across different platforms, the implementation of a JS engine should conform to the ECMAScript standard. However, doing so is challenging as there are many subtle definitions of API behaviors, and the definitions keep evolving.
We present COMFORT, a new compiler fuzzing framework for detecting JS engine bugs and behaviors that deviate from the ECMAScript standard. COMFORT leverages the recent advance in deep learning-based language models to automatically generate JS test code. As a departure from prior fuzzers, COMFORT utilizes the well-structured ECMAScript specifications to automatically generate test data along with the test programs to expose bugs that could be overlooked by the developers or manually written test cases. COMFORT then applies differential testing methodologies on the generated test cases to expose standard conformance bugs. We apply COMFORT to ten mainstream JS engines. In 200 hours of automated concurrent testing runs, we discover bugs in all tested JS engines. We had identified 158 unique JS engine bugs, of which 129 have been verified, and 115 have already been fixed by the developers. Furthermore, 21 of the COMFORT-generated test cases have been added to Test262, the official ECMAScript conformance test suite.
Fri 25 JunDisplayed time zone: Eastern Time (US & Canada) change
09:00 - 09:40 | |||
09:00 5mTalk | Test-Case Reduction and Deduplication Almost for Free with Transformation-Based Compiler Testing PLDI Alastair F. Donaldson Imperial College London, Paul Thomson Google, Vasyl Teliman National Technical University of Ukraine, Stefano Milizia Imperial College London, André Perez Maselco Federal University of ABC, Antoni Karpiński Warsaw University of Technology DOI | ||
09:05 5mTalk | Execution Reconstruction: Harnessing Failure Reoccurrences for Failure Reproduction PLDI Gefei Zuo University of Michigan, Jiacheng Ma University of Michigan, Andrew Quinn University of Michigan, Pramod Bhatotia TU Munich, Pedro Fonseca Purdue University, Baris Kasikci University of Michigan DOI | ||
09:10 5mTalk | Concolic Program Repair PLDI Ridwan Salihin Shariffdeen National University of Singapore, Yannic Noller National University of Singapore, Lars Grunske Humboldt University of Berlin, Abhik Roychoudhury National University of Singapore DOI Pre-print | ||
09:15 5mTalk | Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing PLDI Guixin Ye Northwest University, Zhanyong Tang Northwest University, Shin Hwei Tan Southern University of Science and Technology, Dingyi Fang Northwest University, Xiaoyang Sun University of Leeds, Lizhong Bian Alipay, Songfang Huang Alibaba DAMO Academy, Haibo Wang University of Leeds, Zheng Wang University of Leeds, UK DOI | ||
09:20 5mTalk | Path-Sensitive Sparse Analysis without Path Conditions PLDI Qingkai Shi Hong Kong University of Science and Technology, Peisen Yao Hong Kong University of Science and Technology, Rongxin Wu Xiamen University, Charles Zhang Hong Kong University of Science and Technology DOI | ||
09:25 5mTalk | Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring PLDI Kia Rahmani Purdue University, Kartik Nagar IIT Madras, Benjamin Delaware Purdue University, Suresh Jagannathan Purdue University DOI | ||
09:30 5mTalk | SyRust: Automatic Testing of Rust Libraries with Semantic-Aware Program Synthesis PLDI Yoshiki Takashima Carnegie Mellon University, Ruben Martins Carnegie Mellon University, Limin Jia Carnegie Mellon University, Corina S. Păsăreanu Carnegie Mellon University DOI | ||
09:35 5mTalk | When Threads Meet Events: Efficient and Precise Static Race Detection with Origins PLDI Bozhen Liu Texas A&M University, Peiming Liu Texas A&M University, Yanze Li Texas A&M University, Chia-Che Tsai Texas A&M University, Dilma Da Silva Texas A&M University, Jeff Huang Texas A&M University DOI | ||
21:00 - 21:40 | |||
21:00 5mTalk | Test-Case Reduction and Deduplication Almost for Free with Transformation-Based Compiler Testing PLDI Alastair F. Donaldson Imperial College London, Paul Thomson Google, Vasyl Teliman National Technical University of Ukraine, Stefano Milizia Imperial College London, André Perez Maselco Federal University of ABC, Antoni Karpiński Warsaw University of Technology DOI | ||
21:05 5mTalk | Execution Reconstruction: Harnessing Failure Reoccurrences for Failure Reproduction PLDI Gefei Zuo University of Michigan, Jiacheng Ma University of Michigan, Andrew Quinn University of Michigan, Pramod Bhatotia TU Munich, Pedro Fonseca Purdue University, Baris Kasikci University of Michigan DOI | ||
21:10 5mTalk | Concolic Program Repair PLDI Ridwan Salihin Shariffdeen National University of Singapore, Yannic Noller National University of Singapore, Lars Grunske Humboldt University of Berlin, Abhik Roychoudhury National University of Singapore DOI Pre-print | ||
21:15 5mTalk | Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing PLDI Guixin Ye Northwest University, Zhanyong Tang Northwest University, Shin Hwei Tan Southern University of Science and Technology, Dingyi Fang Northwest University, Xiaoyang Sun University of Leeds, Lizhong Bian Alipay, Songfang Huang Alibaba DAMO Academy, Haibo Wang University of Leeds, Zheng Wang University of Leeds, UK DOI | ||
21:20 5mTalk | Path-Sensitive Sparse Analysis without Path Conditions PLDI Qingkai Shi Hong Kong University of Science and Technology, Peisen Yao Hong Kong University of Science and Technology, Rongxin Wu Xiamen University, Charles Zhang Hong Kong University of Science and Technology DOI | ||
21:25 5mTalk | Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring PLDI Kia Rahmani Purdue University, Kartik Nagar IIT Madras, Benjamin Delaware Purdue University, Suresh Jagannathan Purdue University DOI | ||
21:30 5mTalk | SyRust: Automatic Testing of Rust Libraries with Semantic-Aware Program Synthesis PLDI Yoshiki Takashima Carnegie Mellon University, Ruben Martins Carnegie Mellon University, Limin Jia Carnegie Mellon University, Corina S. Păsăreanu Carnegie Mellon University DOI | ||
21:35 5mTalk | When Threads Meet Events: Efficient and Precise Static Race Detection with Origins PLDI Bozhen Liu Texas A&M University, Peiming Liu Texas A&M University, Yanze Li Texas A&M University, Chia-Che Tsai Texas A&M University, Dilma Da Silva Texas A&M University, Jeff Huang Texas A&M University DOI | ||