PLDI Research Papers

PLDI is a premier forum for programming language research, broadly construed, including design, implementation, theory, applications, and performance. PLDI seeks outstanding research that extends and/or applies programming-language concepts to advance the field of computing. Novel system designs, thorough empirical work, well-motivated theoretical results, and new application areas are all welcome emphases in strong PLDI submissions.

Congratulations to the authors of our Distinguished Papers!

Distinguished Papers

Accepted Papers

Call for Papers

PLDI is a premier forum for programming language research, broadly construed, including design, implementation, theory, applications, and performance. PLDI seeks outstanding research that extends and/or applies programming-language concepts to advance the field of computing. Novel system designs, thorough empirical work, well-motivated theoretical results, and new application areas are all welcome emphases in strong PLDI submissions.

Evaluation Criteria and Process

Reviewers will evaluate each contribution for its accuracy, significance, originality, and clarity. Submissions should be organized to communicate clearly to a broad programming-language audience as well as to experts on the paper’s topics. Papers should identify what has been accomplished and how it relates to previous work.

Authors of empirical papers are encouraged to consider the seven categories of the SIGPLAN Empirical Evaluation Guidelines when preparing their submissions.

Deadlines and formatting requirements, detailed below, will be strictly enforced, with extremely rare extenuating circumstances considered at the discretion of the Program Chair.

In almost all cases, reviews will be performed by a subset of the Program Committee (PC). Authors will have the opportunity to respond to initial reviews to correct and clarify technical concerns. The PC will make final accept/reject decisions.

Authors may contact only the Program Chair about submitted papers during and after the review process. Contacting PC members about submitted paper(s) is an ethical violation and may be grounds for summary rejection.

Double-Blind Reviewing

PLDI uses double-blind reviewing. This means that author names and affiliations must be omitted from the submission. Additionally, if the submission refers to prior work done by the authors, that reference should be made in third person. These are firm submission requirements. Any supplementary material must also be anonymized.

The FAQ on Double-Blind Reviewing clarifies the policy for the most common scenarios. But there are many gray areas and trade-offs. If you have any doubts about how to interpret the double-blind rules, please contact the Program Chair. Overestimate the need to contact the Program Chair for complex cases that are not fully covered by the FAQ.

Submission Site Information

The submission site is https://pldi2021.hotcrp.com.

Authors can submit multiple times prior to the (firm!) deadline. Only the last submission will be reviewed. There is no abstract deadline. The submission site requires entering author names and affiliations, relevant topics, and potential conflicts. Addition or removal of authors after the submission deadline will need to be approved by the Program Chair (as this kind of change potentially undermines the goal of eliminating conflicts during paper assignment).

The submission deadline is 11:59PM November 20, 2020 anywhere on earth: https://en.wikipedia.org/wiki/Anywhere_on_Earth

Declaring Conflicts

When submitting the paper, you will need to declare potential conflicts. Conflicts should be declared between an adviser and an advisee (e.g., Ph.D., post-doc). Other conflicts include institutional conflicts, financial conflicts of interest, friends or relatives, or any recent co-authors on papers and proposals (last 2 years).

Please do not declare spurious conflicts: such incorrect conflicts are especially harmful if the aim is to exclude potential reviewers, so spurious conflicts can be grounds for rejection. If you are unsure about a conflict, please consult the Program Chair.

Formatting Requirements

Papers should be formatted according to the two-column ACM proceedings format. Each paper should have no more than 12 pages, excluding bibliography, in 10pt font. There is no limit on the page count for references. Each reference must list all authors of the paper (do not use et al). The citations should be in numeric style, e.g., [52]. Submissions should be in PDF format and printable on US Letter and A4 sized paper. These requirements are all the same as in the previous year.

Papers that exceed the length requirement or deviate from the expected format will be rejected.

Make sure that figures and tables are legible, even after the paper is printed in gray-scale.

Appendices should not be part of the paper, but should be submitted as supplementary material. Supplementary material should also be anonymized, as described below. These requirements are also the same as last year.

As explained in more detail at http://www.sigplan.org/Resources/Author, LaTeX users should use the sigplan subformat of the acmart format by downloading pldi2021.zip. Word users should use the acmart template for Word. Please note the following:

• The acmart-sigplanproc-template.tex template included in the above zip file has the correct defaults for PLDI 2021 submissions. Specifically, the first line should be \documentclass[sigplan,10pt,review,anonymous]{acmart}\settopmatter{printfolios=true,printccs=false,printacmref=false}. The default citation style is numeric.
• Do not mess with the class file or settings to try to sneak in additional space. (Conversely, you may toggle the printccs and printacmref flags if you wish, but these changes will consume space.)
• Do not use the PACMPL files or format; PLDI is not using them. However, the template files were designed to make migrating a paper from one format to the other as simple as possible.

Supplementary Material

Authors are free to provide supplementary material if that material supports the claims in the paper. Such material may include proofs, experimental results, and/or data sets. This material should be uploaded at the same time as the submission. Reviewers are not required to examine the supplementary material but may refer to it if they would like to find further evidence supporting the claims in the paper.

Plagiarism and Concurrent Work

Papers must describe unpublished work that is not currently submitted for publication elsewhere as described by the SIGPLAN Republication Policy: http://www.sigplan.org/Resources/Policies/Republication/. Authors should also be aware of the ACM Policy on Plagiarism: https://www.acm.org/publications/policies/plagiarism-overview. Concurrent submissions to other conferences, workshops, journals, or similar venues of publication are disallowed. Prior work must, as always, be cited and referred to in the third person even if it is the authors’ work, so as to preserve author anonymity. If you have further questions, contact the Program Chair.

Artifact Evaluation for Accepted Papers

The authors of accepted PLDI papers will be invited to submit supporting materials to the Artifact Evaluation process. Artifact Evaluation is run by a separate committee whose task is to assess how well the artifacts support the work described in the papers. This submission is voluntary but encouraged and will not influence the final decision regarding the papers. Papers that go through the Artifact Evaluation process successfully will receive a badge printed on the papers themselves. Authors of accepted papers are encouraged to make these materials publicly available upon publication of the proceedings, by including them as “source materials” in the ACM Digital Library.

Accepted Papers

Accepted papers will be made available (once the conference starts and for one month following) via 1-click download from the ACM Digital Library.

PLDI welcomes all authors, regardless of nationality. If authors are unable despite reasonable effort to obtain visas to travel to the conference, we will make arrangements to enable remote participation or presentation by another attendee on behalf of the authors.

Publication Date

AUTHORS TAKE NOTE: The official publication date is the date the proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of your conference. The official publication date affects the deadline for any patent filings related to published work. (For those rare conferences whose proceedings are published in the ACM Digital Library after the conference is over, the official publication date remains the first day of the conference.)

Acknowledgments

This call-for-papers is an adaptation and evolution of content from previous instances of PLDI. We are grateful to prior organizers for their work, which is reused here.

General

Q: Why are you using double-blind reviewing?

A: Studies have shown that a reviewer’s attitude toward a submission may be affected, even unconsciously, by the identity of the authors. We want reviewers to be able to approach each submission without any such, possibly involuntary, pre-judgment. Many computer science conferences have embraced double-blind reviewing. PLDI has used it for several years now and doing so is stipulated in the Practices of PLDI.

Q: Do you really think blinding actually works? I suspect reviewers can often guess who the authors are anyway.

A: It is rare for authorship to be guessed correctly, even by expert reviewers, as detailed in this study.

Q: Couldn’t blind submission create an injustice where a paper is inappropriately rejected based upon supposedly-prior work which was actually by the same authors and not previously published?

A: Reviewers are held accountable for their positions and are required to identify any supposed prior work that they believe undermines the novelty of the paper. Any assertion that “this has been done before” by reviewers should be supported with concrete information. The author response mechanism exists in part to hold reviewers accountable for claims that may be incorrect.

For authors

Q: What exactly do I have to do to anonymize my paper?

A: Use common sense. Your job is not to make your identity undiscoverable but simply to make it possible for reviewers to evaluate your submission without having to know who you are. The specific guidelines stated in the call for papers are simple: omit authors’ names from your title page, and when you cite your own work, refer to it in the third person. For example, if your name is Smith and you have worked on amphibious type systems, instead of saying “We extend our earlier work on statically typed toads [Smith 2004],” you might say “We extend Smith’s [2004] earlier work on statically typed toads.” Also, be sure not to include any acknowledgements that would give away your identity. In general, you should aim to reduce the risk of accidental unblinding. For example, if your paper is the first to describe a system with a well-known name or codename, or you use a personally-identifiable naming convention for your work, then use a different name for your submission (which you may indicate has been changed for the purposes of double-blind reviewing). You should also avoid revealing the institutional affiliation of authors or at which the work was performed.

Q: I would like to provide supplementary material for consideration, e.g., the code of my implementation or proofs of theorems. How do I do this?

A (and also see the next question): On the submission site there will be an option to submit supplementary material along with your main paper. This supplementary material should also be anonymized; it may be viewed by reviewers during the review period, so it should adhere to the same double-blind guidelines.

Q: My submission is based on code available in a public repository. How do I deal with this?

A: Making your code publicly available is not incompatible with double-blind reviewing. You should do the following. First, cite the code in your paper, but remove the actual URL and, instead say “link to repository removed for double-blind review” or similar. Second, if, when writing your author response, you believe reviewer access to your code would help, say so in your author response (without providing the URL), and send the URL to the Program Chair.

Q: I am building on my own past work on the WizWoz system. Do I need to rename this system in my paper for purposes of anonymity, so as to remove the implied connection between my authorship of past work on this system and my present submission?

A: Maybe. The core question is really whether the system is one that, once identified, automatically identifies the author(s) and/or the institution. If the system is widely available, and especially if it has a substantial body of contributors and has been out for a while, then these conditions may not hold (e.g., LLVM or HotSpot), because there would be considerable doubt about authorship. By contrast, a paper on a modification to a proprietary system (e.g., Visual C++, or a research project that has not open-sourced its code) implicitly reveals the identity of the authors or their institution. If naming your system essentially reveals your identity (or institution), then anonymize it. In your submission, point out that the system name has been anonymized. If you have any doubts, please contact the Program Chair.

Q: I am submitting a paper that extends my own work that previously appeared at a workshop. Should I anonymize any reference to that prior work?

A: No. But we recommend you do not use the same title for your PLDI submission, so that it is clearly distinguished from the prior paper. In general, there is rarely a good reason to anonymize a citation. One possibility is for work that is tightly related to the present submission and is also under review. When in doubt, contact the Program Chair.

Q: Am I allowed to post my (non-blinded) paper on my web page? Can I advertise the unblinded version of my paper on mailing lists or send it to colleagues? Can I give a talk about my work while it is under review? How do I handle social media? What about arXiv?

A: We have developed guidelines, described here, to help everyone navigate in the same way the tension between the normal communication of scientific results, which double-blind reviewing should not impede, and actions that essentially force potential reviewers to learn the identity of the authors for a submission. Roughly speaking, you may (of course!) discuss work under submission, but you should not broadly advertise your work through media that is likely to reach your reviewers. We acknowledge there are gray areas and trade-offs; we cannot describe every possible scenario.

Things you may do:

• Put your submission on your home page.
• Discuss your work with anyone who is not on the review committees, or with people on the committees with whom you already have a conflict.
• Present your work at professional meetings, job interviews, etc.
• Submit work previously discussed at an informal workshop, previously posted on arXiv or a similar site, previously submitted to a conference not using double-blind reviewing, etc.

Things you should not do:

• Contact members of the review committees about your work, or deliberately present your work where you expect them to be.
• Publicize your work on major mailing lists used by the community (because potential reviewers likely read these lists).
• Publicize your work on social media if wide public [re-]propagation is common (e.g., Twitter) and therefore likely to reach potential reviewers. For example, on Facebook, a post with a broad privacy setting (public or all friends) saying, “Whew, PLDI paper in, time to sleep” is okay, but one describing the work or giving its title is not appropriate. Alternately, a post to a group including only the colleagues at your institution is fine.

Reviewers will not be asked to recuse themselves from reviewing your paper unless they feel you have gone out of your way to advertise your authorship information to them. If you are unsure about what constitutes “going out of your way”, please contact the Program Chair.

Q: Will the fact that PLDI is double-blind have an impact on handling conflicts-of-interest?

A: Double-blind reviewing does not change the principle that reviewers should not review papers with which they have a conflict of interest, even if they do not immediately know who the authors are. Authors declare conflicts-of-interest when submitting their papers using the guidelines in the call-for-papers. Papers will not be assigned to reviewers who have a conflict.

For reviewers

Q: What should I do if I learn the authors’ identity? What should I do if a prospective PLDI author contacts me and asks to visit my institution?

A: If you feel that the authors’ actions are largely aimed at ensuring that potential reviewers know their identity, contact the Program Chair. Otherwise, you should not treat double-blind reviewing differently from other reviewing. In particular, refrain from seeking out information on the authors’ identity, but if you discover it accidentally this will not automatically disqualify you as a reviewer. Use your best judgment.

Q: The authors have provided a URL to supplemental material. I would like to see the material but I worry they will snoop my IP address and learn my identity. What should I do?

A: Contact the Program Chair, who will download the material on your behalf and make it available to you.

Q: If I am assigned a paper for which I feel I am not an expert, how do I seek an outside review?

A: PC members should do their own reviews, not delegate them to someone else. If doing so is problematic for some papers, e.g., you don’t feel completely qualified, then consider the following options. First, submit a review for your paper that is as careful as possible, outlining areas where you think your knowledge is lacking. Assuming we have sufficient expert reviews, that could be the end of it: non-expert reviews are valuable too, since conference attendees are by-and-large not experts for any given paper. Second, the review form provides a mechanism for suggesting additional expert reviewers to the PC Chair, who may contact them if additional expertise is needed. Please do not contact outside reviewers yourself. As a last resort, if you feel like your review would be extremely uninformed and you’d rather not even submit a first cut, contact the Program Chair.

Q: How do we handle potential conflicts of interest since I cannot see the author names?

A: The conference review system will ask that you identify conflicts of interest when you get an account on the submission system. Feel free to also identify additional authors whose papers you feel you could not review fairly for reasons other than those given (e.g., strong personal friendship).

Q: How should I avoid learning the authors’ identity if I am using web-search in the process of performing my review?

A: You should make a good-faith effort not to find the authors’ identity during the review period, but if you inadvertently do so, this does not disqualify you from reviewing the paper. As part of the good-faith effort, do not use search engines with terms like the paper’s title or the name of a new system being discussed. If you need to search for related work you believe exists, do so after completing a preliminary review of the paper.

These guidelines are an evolution of guidelines originally created by Michael Hicks for POPL 2012, slightly modified for PLDI 2012 by Frank Tip, shortened by Keshav Pingali for PLDI 2014, modified slightly by Steve Blackburn for PLDI 2015, and then edited by Emery Berger for PLDI 2016, Dan Grossman for PLDI 2018, Kathleen Fisher for PLDI 2019, and finally by Emina Torlak for PLDI 2020.