ifixai-ai/iFixAi

English · 简体中文 · 日本語 · 한국어

Catch your agent's mistakes and blind spots before the shit hits the fan.

Quick start • Test your agent • What you get back • Docs • Contributing

How do you know it will do its job the way it's supposed to, respecting your business as a whole: its intentions, its goals and who in the organization may do what? And how do you know it won't do something nobody asked for or authorized?

That's why we built iFixAi, the independent auditor for agentic operational alignment and assurance. It puts your agent in a simulation of your business, sends it the requests a real day brings, including the ones that push it past its role, and a judge grades every reply. Every finding shows what was asked and what your agent did.

The risk goes beyond cybersecurity, governance or compliance on their own, so iFixAi's inspections combine AI red teaming, governance, assurance and philosophy.

See it work in a minute. No API key needed: it runs on a built-in demo agent with canned replies and canned grades, so you see the flow before you connect anything real. For a real audit, iFixAi picks a judge from a different provider than your agent. You need Python 3.10 or newer (a stock Mac's python3 is 3.9, so check python3 --version). In a fresh virtualenv (python3 -m venv .venv && source .venv/bin/activate):

pip install ifixai
ifixai author reference -o reference.yaml
ifixai validate reference.yaml
ifixai run --provider mock --eval-mode self --fixture reference.yaml

This builds the simulation environment for an example agent, a support agent at a fictional fintech (its profile), checks it, and runs the 10 default inspections on the demo agent. The scorecard prints in your terminal and the reports land in ./ifixai-results/.

On Windows, run py -m venv .venv, then .venv\Scripts\activate. If ifixai isn't found, use python -m ifixai.cli.main.

Any agent with a chat endpoint must be audited, and iFixAi can audit it. An OpenAI-compatible endpoint works as it is; anything else takes a few lines of Python. Any model can be the judge: OpenAI, Anthropic, Gemini, Azure, Bedrock, OpenRouter and more. iFixAi picks one from the keys you've set, never your agent's own provider.

There are two ways to run it.

Install the plugin or the skill, then ask your coding agent to "run iFixAi on my agent". It finds your agent, builds its simulation environment, shows the cost before anything is billed, runs the audit and explains what it found.

In Claude Code, run these, then ask, or type /ifixai:ifixai (/reload-plugins if it doesn't show up):

/plugin marketplace add ifixai-ai/iFixAi
/plugin install ifixai@ifixai-community

Already installed? Run /plugin marketplace update ifixai-community, then /reload-plugins, to get the latest.

Codex: run these in your terminal, then start Codex and ask. Moving from ifixai@ifixai-ai? See plugin/README.md.

codex plugin marketplace add ifixai-ai/iFixAi
codex plugin add ifixai@ifixai-community

uvx ifixai install --agents cursor

Then type /ifixai-skill in that agent. Swap cursor for claude, vscode, windsurf, cline, continue, gemini, zed or all. More in get started.

1. Describe your agent: copy the reference profile to my-agent.json and edit it (how). Or let the plugin or skill do it: your coding agent reads your agent's code and writes the profile for you.
2. Build its simulation environment and check it. Install the extra for the judge you'll use: openai here, or anthropic, gemini and so on.

pip install "ifixai[openai]"
ifixai author my-agent.json -o my-agent.yaml
ifixai validate my-agent.yaml

3. Set up the run with the wizard. It asks for your agent's endpoint, the judge and the suite, finds the judge's key and saves it all to ifixai.yaml. Then pass your agent's own token, see the plan before anything is billed, and run:

ifixai setup
export AGENT_API_KEY=...        # your agent's own token
ifixai run --fixture ./my-agent.yaml --api-key "$AGENT_API_KEY" --dry-run
ifixai run --fixture ./my-agent.yaml --api-key "$AGENT_API_KEY"

--dry-run shows how many tests and judge calls the run makes, and any checks that don't apply to your agent and why, then stops.

Prefer flags, for scripts or CI? The same run in one command. This example grades with an OpenAI key; for another judge, set that provider's key.

export OPENAI_API_KEY=sk-...
export AGENT_API_KEY=...        # your agent's own token
ifixai run --provider http --endpoint <agent-url> --api-key "$AGENT_API_KEY" --fixture ./my-agent.yaml --dry-run

<agent-url> is your agent's base URL, up to /v1. Use a staging copy, because the probes are real requests. No auth on it? Swap --api-key "$AGENT_API_KEY" for --auth-method none.

• A pass or fail for each inspection, with the evidence behind it.
• An A to F grade from five core pillars: fabrication, manipulation, deception, unpredictability and opacity.
• Three mandatory minimums (B01, B08 and P01). Failing any of them caps the grade at D.
• Reports in JSON and Markdown, so you can compare runs or gate CI with --min-score.

More on scoring and every inspection.

ifixai run runs these 10 inspections. They're quick, they all work over chat, and together they're enough for a grade. Add --full-run for all 61 inspections; any that can't apply to your agent are set aside first, and the run tells you why.

Other suites and options: CLI.

• New here: Get started
• Doing something: Test your agent · Build a simulation environment
• Looking it up: CLI · Python API · Scoring · Inspections
• How it works: Methodology · Case studies
• Beyond live runs: Audit recorded actions · Policy Lab
• Contributing: CONTRIBUTING.md · good first issues. Questions: a GitHub issue or info@ime.life. Security: SECURITY.md. License: Apache 2.0.