Re-engineering a Self-Hosted client-Discovery Engine
This project represents my first large-scale implementation of a self-hosted Retrieval-Augmented Generation (RAG) pipeline deployed on a private VPS. The system automatically ingests portfolio projects, active codebases, and public freelance/contract job boards, processes them through a multi-stage semantic lookup, and outputs structured client leads and personalized cold outreach pitches.
While the initial monolith has been marked as dormant for a cleaner rewrite, the architecture demonstrates how to leverage local LLMs for production-level decision-making without recurring API fees.
---
System Architecture & Pipeline Flow
The workflow is divided into six distinct, decoupled phases designed to process unstructured documents and feeds into scored, high-quality business leads:
``` [ DATA SOURCES ] [ INGESTION ] [ VECTOR STORE ] ┌──────────────┐ ┌──────────┐ ┌─────────────┐ │ Portfolio │──────────▶ Chunk + │───────────▶ ChromaDB │ │ GitHub │ │ Embed │ │ Collections │ │ Job Boards │──────────▶ (nomic- │───────────▶ ─ portfolio │ │ LinkedIn RSS │ │ embed) │ │ ─ market │ │ Substack │──────────▶ │ │ ─ leads │ └──────────────┘ └──────────┘ └──────┬──────┘ │ [ QUERY / TRIGGER ] │ retrieve top-k ┌──────────────┐ ┌──────────┐ ┌──────▼──────┐ │ Scheduler │──────────▶ Query │◀──────────│ Ranker │ │ (daily cron) │ │ Engine │ scored │ (cross-enc) │ │ Manual chat │──────────▶ │ chunks └─────────────┘ └──────────────┘ └────┬─────┘ │ prompt + context [ LLM LAYER ] ┌────▼─────┐ ┌──────────────┐ │ Ollama │ │ LLaMA 3.1 8B │◀─────────│ LiteLLM │ │ (primary) │ │ Gateway │ │ Mistral 7B │──────────▶ │ │ (fallback) │ └────┬─────┘ └──────────────┘ │ structured output ┌────▼─────┐ [ OUTPUTS ] │ Post-proc │ ┌──────────────┐ │ + Router │ │ Lead Report │◀─────────│ │ │ Draft Email │◀─────────│ │ │ Match Score │◀─────────└───────────┘ └──────────────┘ ```
---
Core Pipeline Components
#### 1. Ingestion Layer
- Portfolio Crawler: Built with BeautifulSoup4, scraping structural project metadata (titles, descriptions, tech stack, dates, and locations).
- GitHub Extractor: Uses the PyGitHub API to scan active repositories, analyzing languages used, commit frequency, and README files to build an automated profile of available skills.
- Market Signal Scraper: Headless Playwright workers polling Upwork, LinkedIn, and local Dutch freelance platforms (like YER and Freelancer.nl) for relevant keywords.
- Substack Ingestor: Feedparser pulls personal Substack articles to analyze my written communication style, ensuring the LLM matches my natural voice.
#### 2. Vector Storage & Semantic Search
- Local Embedding Generation: Uses the `nomic-embed-text` model running locally via Ollama. It supports up to an 8192 token context window and is optimized for semantic retrieval of long code snippets and descriptions.
- Persistent Indexing: ChromaDB maintains collections for `portfolio` (developer background), `market` (job boards), and `leads` (processed history).
- Hybrid Retrieval: Combines semantic cosine similarity search ($70\%$) with lexical BM25 keyword matching ($30\%$) to capture conceptual matches while guaranteeing exact matches for specific tech terms (e.g., "WebRTC", "Odin", "Raylib").
#### 3. Reranking & Scoring
- Cross-Encoder Reranking: Applies a lightweight `ms-marco-MiniLM-L6` model to re-score the top 20 retrieved chunks.
- Fit Scoring Algorithm: A custom matching formula scoring prospects from 0 to 100 based on stack alignment. Job postings with scores below 55 are automatically weeded out.
#### 4. Local Generation Layer
- Model Orchestration: Primarily runs `LLaMA 3.1 8B Instruct` (Q4 quantization) locally via Ollama. Provides high token output rates while keeping context processing local.
- Unified Gateway: routes requests through LiteLLM to maintain an OpenAI-compatible API structure, allowing seamless fallback testing.
- Outreach Drafter: Combines Jinja2 prompt templates with matched project history chunks to output targeted cold outreach templates referencing similar work.
---
Infrastructure & Deploy Specifications
- Hosting: Private VPS with 6 vCPUs, 24 GB RAM, and 360 GB SSD.
- Deployment: Multi-container Docker Compose setup managing isolated networks for the FastAPI endpoint, Ollama server, ChromaDB instance, and n8n orchestration.
- Security: Closed port architecture tunneled securely via Caddy reverse proxy with SSL, ensuring raw lead data and private credentials are never exposed.
