JobScoutAI

1. What is JobScoutAI?

JobScoutAI is an AI-powered job hunting agent that scrapes job boards, rates each posting against your skills using AI, generates tailored resumes, drafts freelance proposals, tracks applications, and sends daily email briefings. It is built with Python, Flask, and SQLite, and runs locally on your machine — no accounts, no cloud, no shared data.

Instead of manually refreshing LinkedIn, Indeed, and Freelancer every morning, you point JobScoutAI at the keywords that matter, tune your skill profile once, and let the agent do the filtering. Jobs that survive the AI rating pass appear in your tracker with a score, a summary, and an Apply button. The ones that don't are learned from — the system remembers why you skipped a listing and scores similar ones lower next time.

2. Features Overview

3. Requirements

Required

• Python 3.11 or higher — the app relies on modern type hints and stdlib features.
• A web browser — Chrome is recommended for scraping paths that use undetected-chromedriver; any modern browser works for the UI.
• Anthropic API key — powers AI features: job rating, resume generation, proposal drafting.

Optional but recommended

• ScraperAPI key — unlocks Indeed, ZipRecruiter, and Google Jobs. Without it, you're limited to LinkedIn and Freelancer.
• Gmail OAuth credentials — needed for the email digest scraper and for sending daily notification emails.
• Ollama — a free, local AI alternative for job rating. Lower quality than Claude but useful when call volume is high.
• Chrome browser — installed locally so undetected-chromedriver can bypass CAPTCHA walls on some sites.

4. Quick Start — First-Time Setup

Follow these seven steps once. After that, running the app is a single command.

1. Clone the repository

   git clone https://github.com/HrSYT420/JobScoutAI.git
   cd JobScoutAI

2. Create a virtual environment

   # Windows
   python -m venv .venv
   .venv\Scripts\activate
   
   # macOS / Linux
   python -m venv .venv
   source .venv/bin/activate

3. Install dependencies

   pip install -r requirements.txt

4. Set up your API keys

   Copy .env.example to .env and add at least your Anthropic key:

   cp .env.example .env
   # then edit .env and set:
   ANTHROPIC_API_KEY=sk-ant-...
   SCRAPER_API_KEY=your_key   # optional

   Or skip the file and enter the keys in the web UI under Settings → API Keys once the app is running.

5. Set up your profile

   Copy the example config files and fill them in:

   cp config/profile.example.json config/profile.json
   cp config/resume_content.example.txt config/resume_content.txt

   Edit config/profile.json with your skills, proof points, contact info, and job preferences. Edit config/resume_content.txt with your work history and bullet points. You can also edit both from Settings → Profile and Settings → Resume.

6. Start the app

   python -m tracker.app

   Open http://localhost:5001 in your browser.

7. Run your first scrape

   Go to the Control page and click Run Scraper. Or from the terminal:

   python -m job_hunter.scraper

5. Scraper Sources — What Works and What Doesn't

Job boards actively fight automated scraping. JobScoutAI supports many sources on paper, but in practice the reliability varies. This table is current and honest.

6. The Pipeline — How It Works

Every scraper run walks through eight phases:

1. Scrape — searches enabled platforms for each keyword in your active profiles.
2. Deduplicate — checks each URL against seen_urls.json with a 7-day rolling window.
3. Keyword Filter — keeps only jobs matching at least one of your filter terms.
4. AI Rating — Claude (or Ollama) scores each job 1–10 against your profile.
5. Two-Tier Sort — Standard jobs (score 7+, budget $200+) and Quick Wins (score 5+, low budget or hourly).
6. Save — results written to a dated JSON file: data/jobs_YYYY-MM-DD.json.
7. Notify — email sent with new matches and follow-up reminders (if Gmail is configured).
8. Display — jobs appear in the Available Jobs section of the tracker.

7. The Web UI — Page by Page

Tracker /

Main dashboard. Stat cards, Available Jobs with AI scores, In Progress applications, Skipped Jobs with learning data, Apply Later queue. Sorting, filtering, and pagination on Available Jobs.

Control /control

Run the scraper, watch live log output, stop a running scrape. Profile selector chooses which search profiles to run.

Dashboard /dashboard

Analytics charts: pipeline funnel, proposals over time, win rate trend, scraper daily activity, skills demand tracker, skip tag frequency, bid distribution, response times.

Rated Jobs /rated-jobs

Review AI-rated jobs by date with thumbs up / thumbs down feedback.

Resumes /resumes

Index of all generated resumes. View, download, or delete.

Settings /settings

Ten tabs: Scraper, AI Filter, Profile, Notifications, Statistics, Overrides, Proposal Rules, Profiles, Resume, API Keys.

Demo /demo

Read-only showcase page for presentations. Pipeline visualization, live stats, scraper control.

Application Detail /application/<id>

Individual application view with communication timeline, proposal text, and full job description.

8. Settings Reference

9. AI Providers — Claude vs Ollama

JobScoutAI ships with two interchangeable AI backends. Swap between them in Settings → AI Filter or Settings → Scraper.

Claude (Anthropic API)

• Models: claude-haiku-4-5 for job rating, claude-sonnet-4-6 for proposals and resumes.
• Requires a paid API key.
• Best quality for structured JSON output and creative writing.
• Recommended for proposal drafting and resume generation.

Ollama (Free, Local)

• Runs on your machine — no internet needed for rating.
• Install from ollama.com, pull a model (e.g. ollama pull llama3.2).
• Lower quality than Claude for structured JSON — expect more parse errors.
• Good for high-volume job rating (50+ calls per scraper run) to save on API costs.
• Not recommended for proposals or resumes where quality matters.

10. Resume Builder

The resume builder produces a tailored one-page HTML resume per job, using your profile and work history as the only sources of truth.

1. Click Create Resume on any Available Job or In Progress application.
2. The app looks up the job details from scraped data.
3. Claude generates a tailored HTML resume matching the job requirements.
4. The anti-fabrication checker highlights any skills Claude invented (red badges).
5. The resume opens in a new tab — review, print, or save as PDF.
6. All resumes are saved to data/resumes/ and accessible from the Resumes page.

11. Proposal Drafter

The drafter uses your past proposals and your writing rules to keep new drafts consistent with your voice.

1. Click Draft Proposal on any Available Job.
2. The app loads your past proposals from the tracker as style examples.
3. Proposal writing rules from config/proposal_rules.txt are applied.
4. Claude drafts a proposal matching your tone and structure.
5. The draft is saved as a new application with status draft — edit before submitting.

12. Skip Tag Learning

The system learns from every job you skip and uses that knowledge to score future jobs.

• When you skip a job, you select reason tags (e.g. Too complex, Not my skill set).
• Tags accumulate weights based on how often you use them.
• The AI rater receives these weighted tags and scores matching jobs lower automatically.
• Over time, the system filters out jobs you'd skip before you even see them.
• View and adjust tag weights in Settings → Statistics and Settings → Overrides.

13. Mobile Usage

• All pages are responsive — works on iPhone (390 px wide) and tablets.
• Bottom navigation bar on mobile (Jobs, Stats, Resumes, Settings, Demo).
• Card-based layout replaces tables on small screens.
• Run the scraper from your phone via the Demo or Control page.
• The background scraper keeps running even if you lock your phone or close the browser.
• Access remotely via ngrok: ngrok http 5001 gives you a public URL.

14. Known Limitations

• Scraping is fragile. Job boards actively block automated access. LinkedIn (via JobSpy) is currently the only reliable free source. Indeed, ZipRecruiter, Upwork, and Craigslist are all blocked without paid proxies.
• ScraperAPI dependency. Indeed and ZipRecruiter require ScraperAPI ($49/month) to work. Without it, you're limited to LinkedIn + Freelancer.
• AI costs money. Claude API calls cost roughly $0.01–0.05 per scraper run (50+ rating calls). Ollama is free but lower quality.
• Single user. The app runs locally for one user. No multi-user support, no server deployment, no authentication.
• No auto-apply. The app finds and rates jobs but cannot submit applications automatically. You still apply manually on each platform.
• Gmail OAuth setup is complex. Requires a Google Cloud project with Gmail API enabled and a client_secret.json file. Not beginner-friendly.

15. Roadmap — Room for Improvement

16. Project Structure

JobScoutAI/
├── config/
│   ├── settings.py              — keywords, budget, platforms, location
│   ├── profile.json             — your skills and contact info
│   ├── proposal_rules.txt       — AI writing guidelines
│   ├── resume_content.txt       — work history for resume builder
│   ├── filter_config.json       — AI score thresholds
│   ├── notification_config.json — email notification settings
│   ├── profiles/                — search profile configs
│   └── ...
├── job_hunter/
│   ├── scraper.py               — main pipeline orchestrator
│   ├── filter.py                — two-tier AI job rating
│   ├── ai_client.py             — unified Claude / Ollama interface
│   ├── notifier.py              — email notifications
│   ├── proposer.py              — proposal drafter
│   ├── resume_builder.py        — AI resume generator
│   ├── browser.py               — Selenium / undetected-chrome manager
│   ├── background.py            — background thread execution
│   └── *_scraper.py             — per-platform scrapers
├── tracker/
│   ├── app.py                   — Flask app entry point
│   ├── database.py              — SQLite queries
│   ├── helpers.py               — shared utilities
│   ├── routes/                  — Flask blueprint routes
│   └── templates/               — HTML templates
├── data/                        — job results, database, resumes (gitignored)
├── .env                         — API keys (gitignored)
├── requirements.txt
├── CLAUDE.md                    — developer guide
└── README.md

17. Troubleshooting

"No module named 'distutils'"

Run pip install setuptools. Required for undetected-chromedriver on Python 3.12+.

Indeed / ZipRecruiter return 0 jobs

These platforms block scraping without ScraperAPI. Check Settings → Scraper for your ScraperAPI key, or disable these platforms.

"Invalid grant" Gmail error

Delete gmail_token.json and restart the app to re-authenticate.

Dashboard shows Internal Server Error

Check for Jinja2 template errors in the terminal. Common cause: dict keys that clash with Python method names — use bracket notation d["values"], not d.values.

Resume builder fabricates skills

Review all generated resumes before submitting. Skills highlighted in red were not found in your profile.

Scraper runs for 40+ minutes

Normal with multiple profiles and polite delays. Use the background scraper (Control page) so the UI stays responsive.

App freezes during scraping

Use the background scraper (Control page's Run Scraper button) rather than running python -m job_hunter.scraper while the web UI is also running in the same shell.

18. Contributing

• Fork the repo and create a feature branch.
• Follow the coding standards in CLAUDE.md.
• All functions must have docstrings.
• Use f-strings for string formatting.
• Handle errors gracefully with clear print messages.
• Test before submitting a PR — run the scraper end-to-end and check the tracker UI.

19. License

JobScoutAI is released under the MIT License. You are free to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, provided the license notice is preserved.

The software is provided "as is", without warranty of any kind. See the LICENSE file in the repository for the full text.