AI Agent Job Scout

An autonomous AI career agent that searches LinkedIn, scores job fit, and tailors resumes and cover letters — with human-in-the-loop approval and a hybrid semantic matching engine.

Problem

Job searching is a full-time job — except most people already have one.

Every open role means the same manual grind: scan LinkedIn, read a job description, guess which keywords the applicant tracking system wants, tweak the resume (again), write a new cover letter (again), and hope for the best. Multiply that by 20 or 30 applications, and it’s easy to see why so many strong candidates give up early or apply half-heartedly — not because they’re unqualified, but because they’re exhausted.

Meanwhile, the roles that are the right fit often get missed entirely, buried under a flood of listings that don’t match the applicant’s skills, seniority, or goals.

Recruitment has already been transformed by AI on the employer’s side — smart filtering, automated screening, ranked shortlists. Job seekers are still doing this the way we did it in 2005: manually, repetitively, and without any real feedback loop. That imbalance is the opportunity. If AI can help companies find the right candidate faster, it can just as easily help candidates find — and win — the right job faster.

Approach

JobScout is an agentic AI job-search assistant that automates searching, scoring, and tailoring resumes/CVs for job postings with human-in-the-loop approval. It orchestrates multiple LLM-powered modules — resume parsing, tailored content generation, cover-letter writing, semantic matching, and persistent storage — in a repeatable scouting cycle.

What it does NOT do: submit applications on your behalf. It stops at producing ready-to-review files; you apply and track status manually. That boundary was a deliberate design choice, not a missing feature.

The pipeline

One run of the agent does, in order:

Search — pulls open roles from LinkedIn (via a job-board API).
Skip what’s already been checked — a local record of every job seen before, so the same posting is never re-scored or re-tailored twice.
Score — compares each new job against a base resume using a hybrid of semantic similarity and skill overlap, producing a 0–100 match score.
Filter — keeps only jobs above a configurable threshold, sorted best-first.
Tailor — rewrites the resume for each surviving job: reordering and re-emphasizing real experience to match the role, never inventing skills, employers, or claims that aren’t in the original.
Write a cover letter — grounded in the same real facts, specific to that job.
Package — writes a folder per job (job posting, tailored resume, cover letter as polished DOCX/PDF) and adds one row to a tracking spreadsheet you update by hand (status, follow-ups, notes).

JobScout matching and tailoring pipeline

Value

For applicants: less time searching, higher-quality applications, better match rates, less burnout
For the hiring process broadly: candidates show up better prepared and better aligned to the role, which benefits recruiters too
For the market: a small step toward leveling the playing field between employers’ AI-powered hiring tools and candidates’ still-manual job search

Architecture

Layer	Role
CLI / Harness	Entry point (`run_scouting_cycle`) that orchestrates the end-to-end pipeline: parse → search → score → tailor → write → store
Matching Engine	Scoring module (semantic + keyword overlap) against a local Chroma vector database
Tools	Modular LLM callables — `resume_parser`, `resume_tailor`, `cover_letter_writer`
Storage	ChromaDB for persistent job embeddings + seen-URL deduplication; CSV / Excel exports; SQLite metadata cache (via Langfuse telemetry)
External Services	OpenAI-compatible LLM API, Ollama (local `nomic-embed-text`), Langfuse (telemetry), custom Parse API (`PARSE_API_KEY`)

High-level architecture

graph TB
    subgraph USERS["Users"]
        CLI["CLI / python -m src.harness.run_cycle"]
        WEB["Static Dashboard<br/>dashboard.html + data.js"]
        AGENT["Agent Harness<br/>python -m src.harness (SDK tools)"]
    end

    subgraph HARNESS["Scouting Cycle Harness<br/>src/harness/run_cycle.py"]
        PIPELINE["run_scouting_cycle()<br/>(7-step pipeline)"]
        PKG["package_output()<br/>(render + export)"]
        CLI_MAIN["main() — argparse CLI<br/>--list-seen / --resume --query"]
    end

    subgraph RESUME_PARSING["Resume Parsing"]
        RP_PARSE["parse_resume() → ParsedResume"]
        RP_FORMATS["PDF (pypdf) / DOCX (python-docx) / .txt"]
        RP_MODEL["ParsedResume<br/>pydantic model"]
    end

    subgraph JOB_SEARCHING["Job Search"]
        JS_SEARCH["search_jobs.handler() → List[JobResult]"]
        LS_LINKEDIN["linkedin_search.py<br/>parse.bot API"]
        JD_RESULT["JobResult / JobDetail<br/>pydantic models"]
    end

    subgraph MATCHING_SCORING["Matching & Scoring"]
        SCORER["match_and_score() → MatchResult"]
        SCORE_FORMULA["0.6 × cosine_sim + 0.4 × skill_overlap"]
        FILTERER["filter_top_matches()"]
        MR_RESULT["MatchResult<br/>score, matched_skills, missing_skills, rationale"]
    end

    subgraph CONTENT_GENERATION["Content Generation"]
        RT_TAILOR["tailor_resume() → TailoredResume"]
        CLW_WRITE["write_cover_letter() → CoverLetter"]
        DR_RENDER["doc_renderer.py<br/>render_*_docx / render_*_pdf"]
    end

    subgraph PERSISTENCE["Persistence"]
        DB_MGR["DatabaseManager<br/>(SQLite seen_jobs / apps)"]
        EXCEL_TRACKER["tracker.xlsx<br/>openpyxl output"]
        FOLDERS["applications/<br/>per-job folders"]
    end

    subgraph LOCAL_SERVICES["Local Services"]
        OLLAMA["Ollama<br/>nomic-embed-text (768-dim)"]
        CHROMA_DB["ChromaDB<br/>data/chroma_db/"]
    end

    subgraph CLOUD_SERVICES["Cloud Services"]
        LLM_API["OpenAI-compatible LLM API<br/>Claude / GPT via MODEL_NAME + API_URL"]
        PARSE_BOT["parse.bot API<br/>LinkedIn scraper"]
    end

    subgraph TRACING["Observability"]
        LANGFUSE["Langfuse (optional)<br/>CycleTrace + spans"]
    end

    CLI --> PIPELINE
    AGENT --> PIPELINE
    WEB --> EXCEL_TRACKER

    PIPELINE --> RP_PARSE
    RP_PARSE --> RP_MODEL
    PIPELINE --> JS_SEARCH
    JS_SEARCH --> LS_LINKEDIN
    LS_LINKEDIN --> PARSE_BOT

    PIPELINE --> SCORER
    SCORER --> SCORE_FORMULA
    SCORER --> CHROMA_DB
    SCORER --> OLLAMA
    SCORE_FORMULA --> MR_RESULT

    PIPELINE --> RT_TAILOR
    PIPELINE --> CLW_WRITE
    RT_TAILOR --> LLM_API
    CLW_WRITE --> LLM_API

    PIPELINE --> PKG
    PKG --> DR_RENDER
    DR_RENDER --> FOLDERS
    PKG --> EXCEL_TRACKER

    PIPELINE --> DB_MGR

    PIPELINE -. trace .-> LANGFUSE
    SCORER -. trace .-> LANGFUSE
    RT_TAILOR -. trace .-> LANGFUSE
    CLW_WRITE -. trace .-> LANGFUSE

    CLI_MAIN --> PIPELINE
    CLI_MAIN --> DB_MGR

Runtime topology

graph TB
    subgraph LOCAL["Local Machine"]
        ENV["Python ≥ 3.11 / venv<br/>pip install -e .[dev]"]
        PY_APP["JobScout Application"]

        subgraph OLLAMA_SVC["Ollama (local, port 11434)"]
            MODEL_EMB["nomic-embed-text<br/>(~768-dim)"]
        end

        CHROMA["ChromaDB<br/>data/chroma_db/ on disk"]

        subgraph LANGFUSE_OPT["Langfuse (optional)"]
            LCLOUD["cloud.langfuse.com or self-hosted"]
        end
    end

    subgraph CLOUD["External APIs"]
        LLM_API["OpenAI-compatible LLM API<br/>(Claude, GPT-4, etc.)"]
        PARSE_BOT["parse.bot Scraper API"]
    end

    PY_APP --> OLLAMA_SVC
    PY_APP --> CHROMA
    PY_APP --> LANGFUSE_OPT
    PY_APP --> LLM_API
    PY_APP --> PARSE_BOT

Agent harness — SDK tool registration

flowchart LR
    subgraph AGENT_HARNESS["src/harness/__main__.py (Agent)"]
        SDK_DISC["Discover __sdk_tools__<br/>from each tool module"]
        MCP_SVR["MCP Server<br/>(tool endpoints)"]
        INTERACT["Interactive Agent Loop<br/>(auto-approved permissions)"]
    end

    subgraph TOOL_MODULES["Tool Modules (each exposes __sdk_tools__)"]
        RP_TOOL["resume_parse tool<br/>from resume_parser.py"]
        SCORER_TOOL["match_and_score tool<br/>from scorer.py"]
        JS_TOOL["search_jobs tool<br/>from job_search.py"]
    end

    AGENT_HARNESS --> SDK_DISC
    SDK_DISC --> MCP_SVR
    MCP_SVR --> INTERACT

    RP_TOOL -. exported via __sdk_tools__ .-> SDK_DISC
    SCORER_TOOL -. exported via __sdk_tools__ .-> SDK_DISC
    JS_TOOL -. exported via __sdk_tools__ .-> SDK_DISC

Pipeline orchestration

flowchart LR
    subgraph ARG["CLI Args"]
        RESUM["--resume path/to/file"]
        QUERY["--query senior designer"]
        LOC["--location NYC"]
        LIM["--limit 10"]
        THR["--threshold 70"]
        MAX["--max-tailored 5"]
        PROV["--provider ollama|anthropic"]
    end

    ARG --> PIPELINE

    subgraph PIP["Pipeline Stages (run_scouting_cycle)"]
        S1(["Step 1: parse_resume"])
        S2(["Step 2: search_jobs"])
        S3(["Step 3: dedup via DBManager"])
        S4(["Step 4: match_and_score × N"])
        S5(["Step 5: filter_top_matches"])
        S6A(["Step 6a: tailor_resume"])
        S6B(["Step 6b: write_cover_letter"])
        S6C(["Step 6c: cap check"])
        S7(["Step 7: package_output"])
    end

    PIPELINE --> S1
    PIPELINE --> S2
    S2 --> S3
    S3 --> S4
    S4 --> S5
    S5 --> S6C{"score ≥ threshold?"}
    S6C --> S6A
    S6C --> SKIP["skip job"]
    S6A --> S6B
    S6A --> S6C2{"tailor < max_tailored?"}
    S6C2 --> S6A
    S6C2 --> CAP_OUT["cap out — skip tailoring"]
    S6B --> S7

    S1 --> RP_MODEL["(ParsedResume)"]
    S4 --> SCORER_MODEL["(MatchResult)"]
    S6A --> RT_MODEL["(TailoredResume)"]
    S6B --> CLW_MODEL["(CoverLetter)"]
    S7 --> PKG_RESULT["(CycleResult × N)"]

Tech stack

Category	Package	Purpose
Language	Python ≥ 3.11	—
Async HTTP	aiohttp	Async job-search HTTP requests, API calls
LLM SDK	claude-agent-sdk	Claude / Anthropic model client
Vector Store	chromadb	Local persistent embeddings & deduplication
Embeddings	Ollama (`nomic-embed-text`)	Local 768-dim embeddings — sensitive resume content never leaves the machine
PDF Handling	pypdf	Parse `.pdf` resumes
Docx Handling	python-docx	Parse `.docx` resumes & generate documents
Spreadsheet Output	openpyxl	Export results to `.xlsx` / `.csv`
Data Validation	pydantic	Typed schemas for `CycleResult`, `TailoredResume`, `CoverLetter`, etc.
Telemetry & Tracing	Langfuse	Embedding traces, latency tracking, LLM call monitoring
Config	YAML + `python-dotenv`	`.env` for API keys, YAML for thresholds and scoring weights
Testing	pytest + pytest-asyncio	165+ tests (unit, integration, harness)

External services

Service	Role	Config
OpenAI-compatible LLM API	Claude / GPT-4 model calls for parsing, tailoring, cover-letter generation	`PARSE_API_KEY`, `OPENAI_API_KEY`
Ollama	Local embedding model for vector similarity scoring	Installed locally; no remote API key
Langfuse	Traces, metrics & cost monitoring for every LLM call	`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` (optional)
parse.bot API	LinkedIn job posting scraper	`PARSE_API_KEY`

Key design decisions

Human-in-the-loop boundary. The agent stops at producing ready-to-review files. It never submits applications — you apply and track status manually. This keeps the human accountable for what actually gets sent to an employer, while removing the repetitive grunt work of searching, matching, and formatting.
Hybrid matching, not semantic-only. The scoring engine combines 0.6 × cosine similarity + 0.4 × skill overlap. Pure semantic similarity misses exact keyword matches that ATS systems and recruiters look for; pure keyword matching misses paraphrased or related skills. The hybrid weights both.
Local-first embeddings. ChromaDB data lives on-disk and embeddings run through local Ollama (nomic-embed-text) — sensitive resume content never leaves the machine for the retrieval/scoring step. Only the tailoring and cover-letter generation steps call a cloud LLM.
Deduplication via SQLite. A seen_jobs table records every job URL ever scored. The same posting is never re-scored or re-tailored twice, even across multiple scouting cycles.
Agent harness with auto-discovered tools. Each tool module exports __sdk_tools__, and the harness discovers them at runtime via an MCP server — no hardcoded tool registry. Adding a new tool is a matter of creating a module and exporting its interface.
Never invents experience. The resume tailor reorders and re-emphasizes real experience from the parsed resume. It never fabricates skills, employers, or claims that aren’t in the original — a guardrail against the obvious failure mode of AI-assisted job applications.
Pydantic schemas throughout. Every data structure (ParsedResume, MatchResult, TailoredResume, CoverLetter, CycleResult) is a typed Pydantic model. The LLM’s output is validated against the schema before it’s used — no free-text parsing.

Observability

Metric	Instrumented via	Notes
LLM token usage & cost	Langfuse spans	Per-parse, per-tailor, per-cover-letter
Embedding latency	Langfuse + custom logging	Vector-store fetch time tracked separately
Match scores (semantic vs overlap)	`CycleResult` payload	Exported to Excel for review

Tracing is entirely optional — with no Langfuse keys set, every tracing call silently no-ops. The agent runs the same pipeline with or without observability.

What I am learning

This project explores how to build an agent system where the boundary between automation and human judgment is deliberate. The agent automates the repetitive work — searching, deduplicating, scoring, formatting — and stops exactly where human judgment should take over: deciding which jobs to apply to and submitting the application.

The deeper lesson is about tool orchestration patterns. Each tool module (resume_parser, resume_tailor, cover_letter_writer) is a standalone LLM callable that also exposes itself via __sdk_tools__ for the agent harness. The same tool works whether called directly in the pipeline or interactively through the agent loop — one implementation, two invocation patterns. The hybrid matching engine (semantic + keyword overlap) taught me that real-world matching is rarely one-dimensional: the 0.6/0.4 split between cosine similarity and skill overlap came from testing, not from a formula.