Meta has successfully deployed its Just-in-Time Tests approach into production to catch software bugs before code lands in production. The system utilizes large language models to generate executable tests that demonstrate concrete behavioral differences introduced by code changes, addressing the limitations of traditional testing methods on massive codebases.
The deployment relies on two primary workflows: an intent-aware workflow that infers code risks and a dodgy diff workflow that treats code changes as potential mutants. To minimize false positives, the infrastructure combines an LLM-as-Judge ensemble with a traditional rule-based assessor called RubFake. This automated filtering is paired with a human-on-the-loop approach, reducing the review load on engineers while maintaining high true-positive rates for bug detection.



