Developers who work with AI‑generated Python snippets often face three core problems: the code may be incomplete or malformed, it can contain unsafe calls that jeopardize the host environment, and there is no reliable way to measure its quality before putting it into production. A practical workflow solves these issues in a few strict steps.
First, locate the function definition inside the raw text. Strip any surrounding markdown fences, normalize line endings, then search for a line that starts with “def” followed by the target name. From that point collect subsequent lines until you encounter another top‑level definition, a class statement, a module guard like “if __name__”, or an assignment that begins at column zero. This yields a candidate block that is likely to be the complete function.
Second, verify the block’s syntax with Python’s ast module. If parsing fails, iteratively trim lines from the end until the fragment parses or give up. A successful parse guarantees syntactic correctness.
Third, run a static safety check on the AST. Reject any node representing imports, exec‑like functions, attribute access to private members, or calls to blacklisted builtins such as eval, open, or __import__. Also disallow names that start with double underscores. Only if the check passes does the code proceed to execution.
Fourth, execute the function in a isolated subprocess that provides a restricted globals dictionary containing only a safe subset of builtins (abs, len, list, etc.). Supply your unit‑test cases as argument‑expected pairs, run them, and collect pass/fail counts. Enforce a timeout to halt runaway loops.
Finally, compute a simple quality score: award points for passing the syntax and safety checks, add a weighted fraction for the proportion of tests that succeed, and subtract a small penalty based on cyclomatic complexity (capped at 0.25). The resulting score lets you rank candidates objectively and move only the safest, most functional snippets forward.
#AI #Product #Developer #Python #Security #Testing