I find the concept of code review has a tarnished reputation as of late. As the industry has grown, what was once an organic and valuable process has been turned into a chore. Requiring official approval from other engineers before making a code change is ubiquitous at this point, but that makes it a roadblock and puts pressure on it to be speedy. In the effort make code review even quicker I see a lot of folks reaching for LLMs.
From one angle the LLM makes sense as a tool for code review. It will happily ingest a whole bunch of text and filter down for known patterns to warn on, and it can at least simulate the sort of back and forth discussion within both strict rules and loose guidelines. There’s a lot of projects out there that use it as a sort of “first line” for review, and even a few I’ve come across that will accept it as the only review needed to merge in a change.
I think that using LLMs as a primary reviewer misses out on one of the most important benefits of code review. Sometimes it can help a lot just to see the code from another person’s point of view. When you write up some piece of code you know how everything fits together. The naming, the control flow, the design tradeoffs will (hopefully) be clear to you, but may not be so obvious to someone who’s going to come through later. Getting to see someone else’s first take, what catches their eye right off the bat, what confuses them or or just seems off, is a valuable piece of feedback.
LLMs can be helpful for analyzing code (the updates from the security researchers certainly seem concering as of late) and can apply and explain fairly simple formulaic guidance (e.g. “don’t add new routes to v1, because it’s deprecated”), but their tone and their viewpoint still to me feels very generic. It’s as if I’m getting the average answer off of stack overflow without the color of a full comment section. I can ask it to go into detail on some code segment, but in guiding it through the codebase I am teaching it my mental model which it will then repeat right back to me. Instead of a new perspective I just get an echo chamber of my own design.
In my personal work, if I ask you to review my code, it is because I want to know what you think of it. Use an LLM to help you pick it apart if you like, but I am not in need of any quick canned answers. Your review can be terse or cheeky or even wrong, so long as the voice is yours.