What a Genuinely Good AI Job Tool Looks Like Under the Hood

Every week there is a new AI tool promising to land you a job, and most of them are the same thing wearing a different logo: a prompt wrapped around a general-purpose model, sold back to you as a monthly subscription. I got tired of guessing which ones were real, so I did the thing engineers do when the marketing gets loud -- I went and read the source code of one that isn't a black box.

The system I spent the most time inside is career-ops, an open-source job-search framework (github. com/santifer/career-ops) built by Santiago Fernandez de Valderrama. One boundary up front: the headline outcomes in its README -- 740+ offers evaluated, 100+ CVs generated, a Head of Applied AI role landed -- are the author's own case study, not mine. What I can speak to first-hand is the engineering. I cloned it, ran it, and read the scripts. My checkout is version 1.10.0 with 226 commits on main, and this post is what those scripts taught me about what a genuinely good AI job tool looks like under the hood.

Match the tool to the stage, not the hype

A job search isn't one task; it's a pipeline with distinct stages, and each one has a different bottleneck. Discovery is about coverage. Filtering is about judgment. Tailoring is about writing. Prep is about rehearsal. A tool that's brilliant at one of those is usually mediocre at the rest, so the goal is a small stack, not a single magic app. What made career-ops worth dissecting is that it treats each stage as a separate engineering problem with its own correctness constraints -- instead of one chatbot doing everything badly.

The categories that earn their place in any 2026 stack are the same ones this system implements as distinct components:

• Aggregation and discovery -- pull openings from many company boards into one place so you're not refreshing twenty career pages. The good ones hit job-board APIs directly and dedupe; the bad ones just re-list the same stale postings everyone else sees.
• Fit scoring -- read a posting against your real background and tell you whether it's worth your time. The highest-leverage category and the most underused, because it saves you from applying to roles you were never going to get or want.
• Ghost-job detection -- decide whether a posting is even real before you spend an hour on it. Almost nobody ships this, and it's the difference between a filter and a treadmill.
• Resume tailoring -- reorder and rewrite your experience to match a specific role. Useful when it works from your true history; dangerous when it starts inventing it.
• Interview prep -- mock questions and structured stories. The one category where AI is unambiguously excellent, because prep is practice plus feedback.

Discovery that costs zero tokens

The first thing that surprised me was that the discovery layer doesn't call an LLM at all. The scanner (scan. mjs) hits the public JSON APIs of Greenhouse, Lever, Ashby, Workable, Workday, SmartRecruiters and Recruitee directly -- pure HTTP, no model in the loop -- dedupes against a TSV history, and drops new URLs into an inbox. Its own header advertises 'Zero Claude API tokens,' and once you see it you understand why: paying a language model to read a JSON list of job titles is lighting money on fire. In my checkout it ships ten providers and fetches ten companies concurrently by default.

The design decision I actually respect here is the plugin contract. Every job board is a self-contained module in providers/, and adding a new one is genuinely a matter of dropping in one file that exports three things. This is the header comment that defines the whole extension model:

/**
 * scan.mjs -- Zero-token portal scanner with a plugin-based provider layer.
 *
 * Providers live in providers/*.mjs and are loaded at startup. Each provider
 * exports a default object with:
 *   - id: string -- matched against provider: in portals.yml
 *   - detect(entry): {url}|null -- optional auto-detection from careers_url
 *   - fetch(entry, ctx): [{title,url,company,location}] -- required
 *
 * Zero Claude API tokens -- pure HTTP + JSON.
 */

Files prefixed with an underscore are shared helpers and never loaded as providers, and the loader reads them in alphabetical order specifically so that detect() priority is deterministic across machines. That's the kind of small, boring decision that tells you the author had actually been bitten by non-determinism before.

It also taught me something I didn't expect from a job tool: this is where server-side request forgery lives. A scanner that fetches whatever hostname a config file hands it is an SSRF waiting to happen. The Greenhouse provider hard-codes a defense against exactly that -- a four-host allowlist, an HTTPS-only assertion, and a redirect guard so a sneaky server-side redirect can't bounce the request somewhere off the list:

const ALLOWED_GREENHOUSE_HOSTS = new Set([
  'boards-api.greenhouse.io',
  'boards.greenhouse.io',
  'job-boards.greenhouse.io',
  'job-boards.eu.greenhouse.io',
]);
// ...
// redirect:'error' prevents SSRF via server-side redirects; combined with
// assertGreenhouseUrl above it guarantees the final hostname stays in the allowlist.
const json = await ctx.fetchJson(apiUrl, { redirect: 'error' });

The tell of a serious tool isn't the AI features it advertises -- it's the boring correctness work it does when nobody's watching, like refusing to follow a redirect off a hostname allowlist.

Fit scoring, and the harder problem of whether the job is even real

The evaluation is the part people expect AI to do, and this one does it as reasoning rather than keyword matching. Every offer is first classified into one of six role archetypes -- AI Platform/LLMOps, Agentic/Automation, Technical AI PM, AI Solutions Architect, AI Forward-Deployed, AI Transformation -- detected from signal words in the description, and that archetype decides which proof points and CV rewrites get prioritized. The output is a 1-to-5 weighted average across dimensions like CV match, archetype alignment, comp versus market, and red flags.

What makes it an ethical filter instead of a spam cannon is baked into the score bands. The system recommends applying immediately at 4.5+, calls 3.5-3.9 a maybe-only-with-a-specific-reason, and explicitly recommends against applying below 3.5 -- framing it as respecting both your time and the recruiter's. It never submits an application for you. After building a lot of automation myself, I've come to trust tools that make it easy to do less on purpose far more than ones that promise to do more.

The feature I hadn't seen anywhere else is Block G, a dedicated ghost-job and scam detector. It runs as a separate qualitative tier -- High Confidence, Proceed with Caution, or Suspicious -- and, crucially, it does not touch the numeric fit score. A great-fitting job that smells fake and a mediocre job that's clearly real are two different problems, and collapsing them into one number destroys information. Block G weighs its signals by reliability (posting age and a working apply button are high-reliability; salary transparency is low) and is bound by a hard ethical rule to present signals rather than accuse:

• It must NEVER default to 'Suspicious' without evidence -- the burden is on the tool, not the posting.
• Government and academic roles get an explicit carve-out: 60-90 days open is normal there, so the age heuristic is relaxed.
• Posting-age thresholds are graded (under 30 days good, 30-60 mixed, 60+ concerning), and a role reposted 2+ times in 90 days is flagged.

The unglamorous engineering that makes AI resumes actually parse

This was my favorite rabbit hole, because it's the least glamorous and the most real. The PDF generator runs a Unicode-to-ASCII normalizer aimed squarely at applicant-tracking systems. Em-dashes and en-dashes become hyphens, smart quotes become straight quotes, arrows are spelled out as the words 'to' and 'from,' bullets become pipes, and zero-width characters and non-breaking spaces get stripped -- because those exact characters are what produce mojibake in legacy resume parsers. It masks style and script blocks first, so it only ever rewrites body text and never corrupts your CSS or URLs.

The detail that convinced me a human really thought about this is the currency handling. The normalizer spells out EUR and GBP, but it deliberately refuses to touch the yen glyph. A code comment explains why: that one symbol maps to both Japanese Yen and Chinese Yuan, so any spelled-out code would be wrong for half the users -- better to leave the glyph than to emit confidently-wrong data. That is a real correctness judgment, not a feature checkbox, and it's the opposite of the keyword-stuffing 'beat the ATS' tools you should run from.

What to be skeptical of, and the one rule no tool will tell you

The fastest way to spot a tool that will waste your money is to notice what it's optimizing for. If the pitch is about volume -- 'apply to 500 jobs in one click' -- walk away. Mass auto-application is the exact behavior every tracking system is now tuned to filter out, it tramples most job platforms' terms of service, and it's a quick route to a flagged account. Everything I'd tell you to avoid is the inverse of what career-ops chose to do:

1. Auto-appliers that submit for you. They optimize the one number that doesn't matter (applications sent) and skip the one that does (whether any of them fit). A tool that refuses to hit submit is respecting the right constraint.
2. Keyword-stuffing 'ATS beaters. ' Recruiters read the resume after the parser does, and stuffing is obvious. Real ATS work is character normalization, not keyword spam.
3. Real-time interview copilots that feed you answers live. Unethical, increasingly detectable, and they leave you unable to do the job you talked your way into.
4. Vanity metrics -- 'your profile scored 94%. ' A number with no decision attached to it is theater. A score band that tells you to NOT apply is worth more than a badge that flatters you.

Here's the rule none of these products will print on the landing page: every tool in this stack is generative, which means output is cheap and your judgment is the scarce resource. Keep exactly one human gate -- you -- between any AI and the outside world. Read the resume before it goes out. Rewrite the outreach in your own voice. Decide which roles are worth your name on them. The best-engineered system I've read doesn't try to remove that gate; its entire architecture, from the never-auto-submit rule to the 'recommend against applying below 3.5' band, is built to protect it. Used that way, AI doesn't replace the work of finding a job -- it removes the toil around the work so you can spend yourself on the part that's actually human: choosing well, and showing up prepared.