Sign in Subscribe

Patchsmith - AI-enhanced static code analysis

Patchsmith wraps CodeQL with an agent that writes better queries, triages findings, and drafts fixes — from CLI.

Patchsmith - AI-enhanced static code analysis

In this new project, I wanted to explore how coding agents can enhance existing software through automation and analysis. In particular, to use Claude Agent SDK to transform the user experience of an amazing, albeit quite complex, piece of software: CodeQL.

The challenges of traditional Static Code Analysis

Static Application Security Testing (SAST) is a key component of a secure Software Development Lifecycle. It consists of analysing the source code of a piece of software prior to build and executing it, with the goal of identifying potential security issues.

There are typically three challenges associated with using SAST:

  • Fine-tuning: Having good results requires serious fine-tuning to adapt the generic scanning engine to the particular domain of your application. In the context of CodeQL, this implies writing custom queries to guide the engine in identifying data sources and sinks. Identifying the relevant queries and crafting them is non-trivial. Updating them is time-consuming.
  • False positive: The analysis is done by applying rules, policies, and queries against the code base, often at a local level. This can lead to false positives due to the overall lack of context available to the engine. An engineer thus needs to carefully go through the alerts and triage them.
  • Fixing efforts: Finally, SAST generates work. Every issue needs eventually to be fixed with some degree of urgency, requiring an engineer to dive into the code, figure out a solution, and prepare a pull request. This is usually in a context of stretched resources and already high technical debt.

Coding Agents to enhance SAST

The three challenges described above are core features usually tackled by coding agents:

  • They are capable of reading through a codebase and preparing custom queries based on some business insight shared in a prompt by a user.
  • They can read security report (SARIF files) and review the code to decide if it is a false positive, better characterize the risk suggest remediation.
  • They can write a fix and prepare a pull request based on the security report.

This, in fact, isn't new. The IRIS study and dataset demonstrated substantial improvement on CodeQL detection capabilities by using LLM to write custom queries and review findings. There are also multiple commercial SAST products now advertising some form of AI component.

Results of the IRIS paper showing improvement from CodeQL baseline thanks to the use of LLM to write custom queries and review findings

CodeQL + Claude Code = Patchsmith

Patchsmith is a CLI tool that acts as a CodeQL wrapper with agentic capabilities. If you have Claude Code and CodeQL already setup, you can install in one line:

pipx install "git+https://github.com/eschnou/patchsmith.git"

From there, it is straightforward; you can easily set up and run your first analysis.

cd <a project you want to analyze>
psmith init
psmith analyze

The real power, though, comes from the possibility of letting the agent create custom queries:

Examples:
psmith finetune                         # Auto-detect and generate
psmith finetune --focus "SQL injection" # Focus on specific areas
psmith finetune --max-queries 10        # Generate more queries
psmith finetune --languages python      # Target specific language
psmith finetune -f "auth" -f "XSS" -n 3 # Multiple focus areas

You can also ask the agent to investigate a particular finding (patchsmith investigate F-3), or prepare a fix for a particular finding (patchsmith fix F-3).

Join the Community

If Patchsmith looks like something that can improve your secure software posture, I encourage you to try it out and share your experiences. You can:

Conclusion

What started just as an experiment in merging Agentic behavior with an existing CLI tool proved extremely useful. I was surprised by the quality of the output and the results I managed to obtain on existing software (both open source projects and commercial work from my company).

I'm curious about the feedback from the wider community on this topic and hope this leads to more integration of existing CLI tools with Agentic SDKs. There is a real opportunity to make these tools more useful, more autonomous and user-friendly, thanks to agentic integration.

Need help? Ping me on Discord/LinkedIn, and I'd be happy to assist, I'd love to see real use cases on real codebases and identify improvement opportunities.