As every developer knows, writing thorough tests are tedious yet indispensable to software quality. Large Language Models (LLMs) have been explored to automate test generation and maintenance. However, it is very hard to ensure that LLM-generated tests capture deep semantic properties, handle language-specific nuances like dynamic typing, and effectively reveal hidden faults beyond superficial coverage. This project aims to systematically integrate LLMs with classical software engineering techniques—genetic algorithms, type inference, and adversarial specification validation—to generate rigorous, context-aware, and semantically trustworthy tests that uncover real bugs and enforce what the code should truly do.

Property-Based Testing Tests should not just check if code runs, but verify whether it behaves correctly under all conditions. Yet LLMs often generate shallow tests that only exercise the happy path. We push LLMs to think in terms of properties—the rules, invariants, and contracts that define correct behavior. Our approach teaches the LLM to challenge its own generated tests by looking for loopholes: if a test passes the code but fails to catch a subtle violation of the intended rule, we refine the test until it does. By learning from existing test patterns and actively probing for semantic gaps, our method has uncovered 45 real-world bugs in widely-used libraries.

Test4Py Testing dynamic programming languages (like Python) is notoriously tricky because types are not explicitly declared—making it hard to know what inputs a function expects or what outputs it should produce. LLMs often generate tests that simply cannot run due to type mismatches. Test4Py makes LLMs "type-aware": it understands how functions are actually used in practice and generates test inputs that are both valid and meaningful.