LLM Code Review Challenge

In this non scientific contest, I checked the code review prowess of three (3) excellent LLM's. The contestants are:

1. OpenAI GPT 4.1
2. Claude Sonnet 3.7
3. DeepSeek v3

Method

LLM canvass provided by https://app.understand.tech

Attach code file (Appendix A)  as input to a single prompt below.

“Review attached java code. Find issues and provide a report. Summarize issues first, followed by detail fix.”Sit back and watch results stream in.

Summary

All contestants did an excellent job. The table below is not meant to pick a winner. Rather, it highlights that if we combine the result set from multiple LLM, we will have better coverage over a single LLM. 

There are at least 10 issues listed in the table below. Let’s see how the contestants did.

Summary Response

Here we review full response received from the LLM canvass.

GPT 4.1

Summary of Issues

1. Multiple Public Classes in One File: Java does not permit two public classes in a single file.
2. Transfer Lacks Recipient Logging: Recipient's transaction history is not updated when funds are received.
3. Insecure Exposure of Internal List: getTransactionHistory() returns a modifiable reference.
4. Silent Failures in Transactions: Errors like invalid deposit/transfer aren't always communicated to the user.
5. No Null Recipient Check in Transfer: Transfer could throw a NullPointerException.
6. Misleading Transaction Logging Order: Current transfer log may be confusing due to withdrawal sequencing.
7. No toString() Override for Debug: Makes debugging less convenient.
8. Thread Safety Not Addressed: Unsuitable for concurrent use in real-world apps.
9. Transaction History Can Be Forged Externally: No attempt at immutability or access restriction.
10. Edge Case: Zero or Negative Transactions: Not all methods robustly handle this.

Claude Sonnet 3.7

Summary of Issues

1. Multiple public classes in one file: Both BankAccount and Main classes are public in the same file.
2. Filename/class name mismatch: File is named banker.java but contains BankAccount class.
3. Insufficient input validation: Constructor allows negative initial balance.
4. Poor error handling: Missing proper exceptions for invalid operations.
5. Security vulnerability: Direct access to transaction history list.
6. Lack of synchronization: No thread safety for concurrent operations.
7. Duplicate transaction recording: Transfer method adds redundant transaction records.
8. Double precision issues: Using double for financial calculations.
9. Missing documentation: Limited JavaDoc comments.
10. No timestamp for transactions: Transaction records lack date/time information.

DeepSeek v3

Summary of Issues:

1. Input Validation Missing: No checks for null/empty strings or negative balances
2. Floating-Point Precision: Using double for financial calculations can cause rounding errors
3. Thread Safety: No synchronization for concurrent access
4. Transaction History Exposure: Getter returns mutable internal state
5. Incomplete Transaction Logging: Missing timestamps and balance updates
6. Error Handling: Basic console output instead of proper exceptions
7. Transfer Logging: Only logs on sender side, not recipient
8. Code Organization: Main class should be separate
9. Immutable Fields: Account number/holder should be final
10. Missing Documentation: No JavaDoc comments

Appendix A ( Banker.java)

// Banker.java
// This file contains intentional bugs and areas for improvement
// Task: Identify and fix the issues in this bank account implementation

import java.util.ArrayList;
import java.util.List;

public class BankAccount {
    private String accountNumber;
    private String accountHolder;
    private double balance;
    private List<String> transactionHistory;

    public BankAccount(String accountNumber, String accountHolder, double initialBalance) {
        this.accountNumber = accountNumber;
        this.accountHolder = accountHolder;
        this.balance = initialBalance;
        this.transactionHistory = new ArrayList<>();
        transactionHistory.add("Account created with initial balance: " + initialBalance);
    }

    public void deposit(double amount) {
        if (amount > 0) {
            balance += amount;
            transactionHistory.add("Deposited: " + amount);
        }
    }

    public void withdraw(double amount) {
        if (amount > 0 && amount <= balance) {
            balance -= amount;
            transactionHistory.add("Withdrew: " + amount);
        } else {
            System.out.println("Invalid withdrawal amount");
        }
    }

    public void transfer(BankAccount recipient, double amount) {
        if (amount > 0 && amount <= balance) {
            this.withdraw(amount);
            recipient.deposit(amount);
            transactionHistory.add("Transferred: " + amount + " to account " + recipient.getAccountNumber());
        }
    }

    public void printTransactionHistory() {
        System.out.println("Transaction History for Account: " + accountNumber);
        for (String transaction : transactionHistory) {
            System.out.println(transaction);
        }
    }

    // Getters
    public String getAccountNumber() {
        return accountNumber;
    }

    public String getAccountHolder() {
        return accountHolder;
    }

    public double getBalance() {
        return balance;
    }

    public List<String> getTransactionHistory() {
        return transactionHistory;
    }
}

// Main class to demonstrate the BankAccount functionality
public class Main {
    public static void main(String[] args) {
        BankAccount account1 = new BankAccount("123456", "John Doe", 1000.0);
        BankAccount account2 = new BankAccount("789012", "Jane Smith", 500.0);

        account1.deposit(200);
        account1.withdraw(100);
        account1.transfer(account2, 300);

        account1.printTransactionHistory();
        account2.printTransactionHistory();

        System.out.println("Account 1 balance: " + account1.getBalance());
        System.out.println("Account 2 balance: " + account2.getBalance());
    }
}