The Largest Case Study of Code Reviews—Ever


clearly in favor of the optimistic interpretation. There were many
cases where reviewers argued with the preparatory comments and where reviewers
clearly were looking at other parts of the code and thinking about ramifications
elsewhere in the code base.

So, our conclusion is that author preparation is indeed a Good Thing and that it saves time overall during the review because reviewers are not having to call out "obvious" defects.

Results in summary
To summarize all our results, including some things not discussed here:

    • Lightweight-style reviews are effective and efficient
    • Review fewer than 200-400 LOC at a time
    • Aim for an inspection rate of less than 300-500 LOC/hour
    • Take enough time for a proper, slow review, but not more than 60-90 minutes
    • Author preparation is good
    • Expect defect rates around 15/hour, higher only when <150 LOC under review
    • Left to their own devices, reviewers' inspection rates will vary widely,  even with similar authors, reviewers, files, and review size

For a full explanation and evidence for all these results, refer to  Best Kept Secrets of Peer Code Review .

Cisco® and MeetingPlace® are registered trademarks of Cisco Systems Inc.. These names and the information herein are reproduced with permission.

About the author

Jason Cohen's picture Jason Cohen

Jason Cohen is the author of Best Kept Secrets of Peer Code Review, the definitive work on the new process of Lightweight peer review with 7,000 copies in circulation. Jason conducted and published the largest case study of peer code review at Cisco Systems®—2,500 reviews over 10 months with a real, delivered product. Jason is also the founder of Smart Bear Inc., makers of Code Collaborator that assists developers and managers with lightweight peer code review processes. Jason has spoken at many conferences, trade shows, and ACM/IEEE society meetings across the country.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!