Improving Unit Tests for Consistent Code Quality

Don’t Focus Too Much On Writing More Tests Too Soon 📌 Prioritize Quality over Quantity - Make sure the tests you have (and this can even be just a single test) are useful, well-written and trustworthy. Make them part of your build pipeline. Make sure you know who needs to act when the test(s) should fail. Make sure you know who should write the next test. 📌 Test Coverage Analysis: Regularly assess the coverage of your tests to ensure they adequately exercise all parts of the codebase. Tools like code coverage analysis can help identify areas where additional testing is needed. 📌 Code Reviews for Tests: Just like code changes, tests should undergo thorough code reviews to ensure their quality and effectiveness. This helps catch any issues or oversights in the testing logic before they are integrated into the codebase. 📌 Parameterized and Data-Driven Tests: Incorporate parameterized and data-driven testing techniques to increase the versatility and comprehensiveness of your tests. This allows you to test a wider range of scenarios with minimal additional effort. 📌 Test Stability Monitoring: Monitor the stability of your tests over time to detect any flakiness or reliability issues. Continuous monitoring can help identify and address any recurring problems, ensuring the ongoing trustworthiness of your test suite. 📌 Test Environment Isolation: Ensure that tests are run in isolated environments to minimize interference from external factors. This helps maintain consistency and reliability in test results, regardless of changes in the development or deployment environment. 📌 Test Result Reporting: Implement robust reporting mechanisms for test results, including detailed logs and notifications. This enables quick identification and resolution of any failures, improving the responsiveness and reliability of the testing process. 📌 Regression Testing: Integrate regression testing into your workflow to detect unintended side effects of code changes. Automated regression tests help ensure that existing functionality remains intact as the codebase evolves, enhancing overall trust in the system. 📌 Periodic Review and Refinement: Regularly review and refine your testing strategy based on feedback and lessons learned from previous testing cycles. This iterative approach helps continually improve the effectiveness and trustworthiness of your testing process.

Many developers write ineffective unit tests because they misunderstand what is supposed to be a "unit". A common misconception is that it's supposed to represent a unit of implementation, such as a method or a function. Therefore, a common practice is to write tests for each method or function and mock everything else. While such an approach has some benefits, it has some really big downsides. You will end up spending way too much time writing tests, which will be difficult to maintain. You will also not be able to use your tests to verify that the behaviour hasn't changed after refactoring, as changing implementation details will force you to change the tests. A much more effective approach is to treat a unit of behaviour as the "unit" in the context of unit tests. This way, you write your tests as close to the public API as possible (e.g. at the level of the public interface that enables access into your library or module). You would also keep mocking to the minimum and use as many real dependencies as possible. This way, you will still implicitly test all your implementation details, as they will still be invoked. However, now you have much fewer tests, those tests are easy to maintain, and you will be able to use these tests to verify that your refactoring effort didn't result in any side effects.

Here’s my step-by-step action plan whenever I work with a client to help them get a new automation project started. Maybe it’s useful to you, too. 0. Write a single, meaningful, efficient test. I don’t care if it’s a unit test, an integration test, an E2E test or whatever, as long as it is reliable, quick and produces information that is valuable. 1. Run that test a few times locally so you can reasonably assume that the test is reliable and repeatable. 2. Bring the test under version control. 3. Add the test to an existing pipeline or build a pipeline specifically for the execution of the test. Have it run on every commit or PR, or (not preferred) every night, depending on your collaboration strategy. 4. Trigger the pipeline a few times to make sure your test runs as reliably on the build agent as it does locally. 5. Improve the test code if and where needed. Run the test locally AND through the pipeline after every change you make to get feedback on the impact of your code change. This feedback loop should still be VERY short, as we’re still working with a single test (or a very small group of tests, at the most). 6. Consider adding a linter for your test code. This is an optional step, but one I do recommend. At some point, you’ll probably want to enforce a common coding style anyway, and introducing a linter early on is way less painful. Consider being pretty strict. Warnings are nice and gentle, but easy to ignore. Errors, not so much. 7. Only after you’ve completed all the previous steps you can start adding more tests. All these new tests will now be linted, put under version control and be run locally and on a build agent, because you made that part of the process early on, thereby setting yourself up for success in the long term. 8. Make refactoring and optimizing your test code part of the process. Practices like (A)TDD have this step built in for a reason. 9. Once you’ve added a few more tests, start running them in parallel. Again, you want to start doing this early on, because it’s much harder to introduce parallelisation after you’ve already written hundreds of tests. 10 - ∞ Rinse and repeat. Forget about ‘building a test automation framework’. That ‘framework’ will emerge pretty much by itself as long as you stick to the process I outlined here and don’t skip the continuous refactoring.

Your code coverage is 80%? Congrats… but your tests might still be useless. For years, code coverage has been the go-to metric for testing. The idea? The more lines of code covered, the better your tests. But here’s the problem: 🚨 90% code coverage does not mean high-quality tests. 🚨 0% code coverage doesn’t necessarily mean bad code, just that if you break it you won't have a clue. I’ve seen teams aim for high coverage, only to realize their tests weren’t actually catching or preventing real bugs. So what should we really measure? ✅ Do your tests actually catch and prevent bugs? ✅ Do they cover real-world use cases—both happy paths and edge cases? ✅ Can your tests detect unexpected mutations or regressions? Here is our attempt to define a new way to measure the tests quality of the tests themselves: 🔥 EQS (Early Quality Score) 🔥 Instead of just checking how much of your code is covered, EQS factors in test quality with three key dimensions: Code Coverage – What % of your code is tested? Mutation Score – How well do your tests detect real code changes? Scope Coverage – What percentage of your public methods have unit tests and 100% coverage? This takes test quality to the next level, answering the real question: Are my tests actually protecting my code? We’ve been using EQS internally at Early, and the insights are game-changing. It helps us evaluate our technology for high-quality test generation, spot gaps, and improve test effectiveness. What are your thoughts? Do you have other ideas to measure the quality of the tests themselves?

Generating code faster is only valuable if you can validate every change with confidence. Software engineering has never really been about writing code. Coding is often the easy part. Testing is harder, and many teams struggle with it. As tools make it easier to generate code quickly, that gap widens. If you can produce changes faster than you can validate them, you eventually create more code than you can safely operate. Which begs the question: What does good testing actually look like? 🔍 What Good Looks Like One of the biggest challenges I see is that teams struggle to understand what “good” testing means and never define it. Pipelines are often built early in a project, when the team is small, and they rarely keep pace with the system and organization as they grow. My starting principle is simple: - At pull request time, you should have strong confidence that the change will not break the service or platform being modified. - Within a day of merging, you should have strong confidence that the change hasn’t broken the full customer journey that the platform supports. 🔁 On Pull Request For backend platforms, I like to see three levels of automated testing before merging. Code Tests (Unit Tests) This level is the foundation. Unit tests validate internal logic, error handling, and edge cases. Techniques such as fuzz testing and benchmarking also reveal issues early. As the test pyramid tells us, this is where the majority of testing and logic validation should take place. Service-Level Functional Tests Too many teams stop at unit tests for pull requests.
Functional tests should also be run in CI for every pull request. Services should be tested in isolation with functional tests. Dependencies can be mocked, but things like databases should ideally run for real (Dockerized). This is where API contracts are validated and regressions can be identified without wondering whether the issue came from this change or another service. Platform-Level Functional Tests Testing a service alone isn’t enough. Changes can break upstream or downstream dependencies. Platform-level tests spin up the entire platform in CI and validate that services interact correctly. These tests ensure the platform continues to work as a system. If these three layers pass, you should have high confidence in the change. But not complete confidence. 🌙 Nightly Testing Some failures take time to appear. Memory leaks, performance degradation, and cross-platform integration issues may not show up immediately. That’s why I like to run a nightly build (or every few hours). This environment runs end-to-end customer journey tests, performance tests, and chaos tests. These are typically the same tests used during release validation, but running them continuously accelerates feedback. If something breaks, you learn about it early, before the pressure of a release. See the comments for the link to the full post.

A few releases ago, we were preparing a deployment that looked perfect on paper. All Apex classes had 90% test coverage. No failed test methods. CI/CD pipeline in Copado showed green across the board. But when we pushed to production — users started reporting broken automations. Why? Because our tests didn’t test behavior — they only tested execution. Here’s what I learned (and changed) after that release 👇 1️⃣ Write Meaningful Unit Tests Don’t just aim for 75% coverage. Validate expected outcomes. Example: Instead of only inserting an Account, assert that related Contacts and Opportunities were created correctly. 2️⃣ Use @testSetup Methods Wisely Create reusable data for all test cases. It saves time and ensures consistency. 3️⃣ Mock External Calls For REST and SOAP integrations, use HttpCalloutMock to simulate responses — never make real API calls in tests. 4️⃣ Test Negative Scenarios What happens when a required field is blank? Or when a Flow fails? True stability comes from testing for failure, not just success. 5️⃣ Automate Regression Tests in CI/CD Use tools like Copado, Gearset, or Salesforce DX to automate test runs before every deployment. “Code coverage tells you what executed. Assertions tell you what worked.” After that project, I stopped aiming for green bars — I started aiming for confidence. #Salesforce #Testing #Apex #Copado #SalesforceDeveloper #TrailblazerCommunity #CI/CD #BestPractices

💡Meta's research introduces 𝗔𝗖𝗛 (𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱 𝗖𝗵𝗲𝗰𝗸 𝗳𝗼𝗿 𝗛𝗮𝗿𝗱𝗲𝗻𝗶𝗻𝗴), a new 𝗺𝘂𝘁𝗮𝘁𝗶𝗼𝗻-𝗴𝘂𝗶𝗱𝗲𝗱 𝗮𝗽𝗽𝗿𝗼𝗮𝗰𝗵 using LLMs for generating more 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝘂𝗻𝗶𝘁 𝘁𝗲𝘀𝘁𝘀. ACH uses 𝗺𝘂𝘁𝗮𝘁𝗶𝗼𝗻 𝘁𝗲𝘀𝘁𝗶𝗻𝗴 to generate 𝘁𝗮𝗿𝗴𝗲𝘁𝗲𝗱 𝘁𝗲𝘀𝘁𝘀 that can detect specific issues, like privacy vulnerabilities, and ensures they are buildable, reliable, and meaningful. 𝗪𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝘁𝗵𝗶𝘀 𝗮𝗽𝗽𝗿𝗼𝗮𝗰𝗵 𝗶𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝗶𝗻𝗴? • 𝗠𝘂𝘁𝗮𝘁𝗶𝗼𝗻 𝘁𝗲𝘀𝘁𝗶𝗻𝗴 helps identify gaps in test coverage by introducing small changes (mutants) to the code, which are then checked by the test cases. • 𝗟𝗟𝗠𝘀 are used to 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰𝗮𝗹𝗹𝘆 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝘁𝗲𝘀𝘁𝘀, making the process faster and more efficient, with a focus on issues like privacy and security. • The method results in 𝗯𝗲𝘁𝘁𝗲𝗿 𝗰𝗼𝘃𝗲𝗿𝗮𝗴𝗲, ensuring that tests are actually catching bugs and improving code quality before release. As someone building in this space, this research is a great reminder of 𝗵𝗼𝘄 𝗔𝗜 𝗰𝗮𝗻 𝗺𝗮𝗸𝗲 𝘁𝗲𝘀𝘁𝗶𝗻𝗴 𝘀𝗺𝗮𝗿𝘁𝗲𝗿—𝗻𝗼𝘁 𝗷𝘂𝘀𝘁 𝗳𝗮𝘀𝘁𝗲𝗿. We're on it to make it generally available with Keploy 🐰 🔜.🔥 The idea of hardening code against potential vulnerabilities through automated, AI-driven tests sounds promising, let's take testing beyond traditional approaches. 🚀 Check out the full paper: “Mutation-Guided LLM-based Test Generation at Meta” https://lnkd.in/gUWgbvgB #AI #MutationTesting #LLM #SoftwareTesting #Security #Keploy

MCP Server Delivers AI-Generated Unit Tests and Advanced Fuzz Testing The server generates intelligent unit tests with proper edge cases, performs AI-powered fuzz testing to identify potential crashes, and conducts advanced coverage testing for maximum code path analysis. Each function receives 4-6 test cases while boundary testing uses 20 diverse inputs to probe system limits. The server combines BAML's structured generation with Gemini's language understanding, built on the FastMCP framework. It performs AST-based code analysis to detect branches, loops, and exception paths while integrating coverage.py for real-time reporting. The modular architecture allows teams to extend testing capabilities as needed. Software reliability becomes measurable and achievable at scale. Automated testing reduces manual QA overhead while catching edge cases that human testers might miss. Development cycles accelerate without sacrificing code quality, making robust software testing accessible to teams of any size. Vaibhav Gupta 👩💻https://lnkd.in/eHqRAJ38

The number of lines in your function has ZERO correlation to its capacity for catastrophic failure... I've seen a two-line function take down an entire communications network. I've watched a single line of code brick devices during firmware updates. Yet junior engineers often ask me, "Why does my 'simple' two-line function need dozens of unit tests?" Here's the truth: If you can't articulate the specific risk each unit test is mitigating, you're not doing engineering—you're just performing a ceremony. Unit testing isn't about achieving 100% coverage (that's a vanity metric). It's about systematically trying to destroy your function in a controlled environment before it gets a chance to destroy your product in the wild. For that "simple" two-line function, we're not just testing the code; we're stress-testing our assumptions: • What happens at INT_MAX? Does it overflow? • What if the input pointer is NULL? • What if it's called from two different threads without a mutex? • What if the underlying hardware register is in a weird state? • What about division by zero? Off-by-one errors? Each test case is a deliberate question we ask our code. The code may look simple. But the state space it operates on could be a minefield. Good mentorship isn't saying "Because I said so." It's explaining exactly why each test matters—making the invisible risks visible. What's the most deceptively simple function that caused the biggest disaster you've ever had to debug? Share below! 👇 #EmbeddedSystems #UnitTesting #TDD #Firmware #SoftwareEngineering #Cprogramming #Cpp #QualityAssurance #TechLead #StaffEngineer

Test data is often a source of information for more test ideas. Look for "safe" test data, specially crafted to conform to application expectations. You will notice patterns of symmetry, items that match each other in value or quantity. Sizes of lists, arrays. Items which seem to have a relationship between each other, preserved in the data. Make note of what each of those patterns might be and come up examples of data that defy the pattern. Sometimes you write these examples out by hand. Sometimes tools can help you do that. One example are pairwise combination tools. If two pieces of data seem to go along with each other, define each as a variable in the pairwise tool, and fill in the list of different interesting values for each. Tell the tool to generate all combinations. You will likely wind up with pairings that match the safe path, and many more that do not, a result of the tool pairing values of the variables not truly designed to go together. Unhappy for the code makes for bug hunting data. I found such an example one time examining unit tests. The cartoon today portrays a similar data relationship. The mocked data had two properties, each an array of objects. The objects had different named properties, but in the test data I noticed both arrays were the same size, and the property values echoed each other. I also noticed no instances in the unit test of either array being null. There was maybe a half dozen checks using this data type. I fed null, empty array, single item array, double item array, and then items of different values for the properties in each of the array properties into a pairwise testing tool, along with values for the other properties in the data structure, and the "Generate All Combinations" calculation produced 200 different versions of the test data. Probably didn't need all 200, but I have a feeling somewhere between a half dozen and 200 lies a test case that exposes a bug. Well, take that back, I KNOW it exposed a bug, because that is why I went looking. A bug had been fixed and there was no unit test update. Examining the fix, the condition covered in the fix was not in the original set of unit tests. Using my approach above, there several instances that hit that condition in combination with other permutations. #softwaretesting #softwaredevelopment You can find more of my articles and cartoons about testing in my book Drawn to Testing, in Kindle and paperback format. https://lnkd.in/gM6fc7Zi