Niches Project

german market niche finder → site builder → auto deploy

Scan 1.4 million .de domains to find profitable tool niches, verify with AI, deep-research the best ones, then build modern German replacement sites and deploy them on Vercel. Reference: arbeitszeitrechner.eu

1.4M
Domains in DB
6
Scripts Built
4
Stages
3/4
Stages Done
Stage 1 — Domain Discovery & Verification DONE
Step 1 — HTTP Check
check_domains.py
Async HTTP checker scans domains from the 1.4M database. Marks live domains, deletes parked and dead ones.
  • Async batch processing with --limit arg
  • Classifies: live / parked / dead
  • Removes dead domains from DB
  • Output: live domains in analysis table
Step 2 — AI Verification
codex_verify.py
Curls each live domain, sends HTML to Gemini CLI for structured analysis. Extracts JSON classification.
  • curl domain, get HTML (12s timeout)
  • gemini -p "prompt" with Google Search grounding
  • Extracts: fetch_status, is_tool, category, subcategory
  • Also: traffic_potential, monetization_potential, monetization_model
  • Stores in codex_analysis table
Verified tool domains flow down
Stage 2 — Domain Scoring & Selection DONE
Step 3 — Profit Scoring & Picking
pick_domain.py
Queries DB (analysis JOIN codex_analysis), excludes already-researched domains. Computes profit score and presents interactive table.
  • Formula: (traffic * mono * 10) + (confidence * 10) + (risk * 3)
  • Scaled to 0-100, color coded (green 70+, yellow 45+, red below)
  • Shows Rich table with top N domains
  • Interactive: pick by number to launch research
  • Also supports --auto (pick #1) and --top N
traffic_pts: high=3 med=2 low=1 mono_pts: high=3 med=2 low=1 risk_pts: low=3 med=2 high=1
Top-scored domain selected
Stage 3 — Deep Research Pipeline DONE
Full Research Pipeline
research_domain.py
End-to-end research for a single picked domain. Browses the site, analyzes with AI, finds German competitors via Serper.dev, browses competitors, does visual analysis with Codex, and stores everything in the database.
Step 1
Browse Target
agent-browser
Step 2a
AI Analysis
gemini -p
Step 2b
German SERP
serper.dev
Step 3
Browse Competitors
agent-browser x3
Step 4
Visual Analysis
codex exec -i
  • Step 1: browse.full(url) returns snapshot text + screenshot PNG
  • Step 2a: Gemini CLI generates keywords, analyzes the niche
  • Step 2b: Serper.dev POST with gl=de, hl=de for real German top 10
  • Step 3: Browse top 3 competitors, get their snapshots + screenshots
  • Step 4: codex exec -i img1.png -i img2.png - visual comparison
  • Step 5: Store in domain_research + domain_plans tables
Researched domain ready to build
Stage 4 — Site Generation & Deployment TODO
Step 5 — Generate PRD
generate_prd.py
Generate per-domain SITE_PRD.md from the research data. This becomes the blueprint for the AI site builder.
  • Pull research from domain_research
  • Pull plans from domain_plans
  • Output: detailed SITE_PRD.md per domain
Step 6 — AI Build
build_site.py
AI builds the actual site (HTML/CSS/JS) from the PRD. Creates a modern, fast, SEO-optimized German tool site.
  • Read SITE_PRD.md as blueprint
  • Generate all HTML, CSS, JS files
  • German language, SEO meta tags
  • Responsive, ad-ready layout
Step 7 — Deploy
deploy_site.py
Full deployment pipeline: init git, create GitHub repo under niches-de org, deploy to Vercel, attach custom domain.
  • git init + gh repo create niches-de/slug
  • vercel --yes --prod
  • vercel domains add domain.de
  • Push to main triggers auto-deploy
Step 8 — Mark Built
mark_built.py
Update the database to mark the domain as built and deployed. Closes the loop.
  • Update analysis.site_status = built
  • Record deploy URL, repo URL
  • Domain excluded from future picks
Database Flow 
        graph LR
          A["analysis
1.4M domains"] -->|"check_domains.py
HTTP filter"| B["analysis
live domains"]
          B -->|"codex_verify.py
Gemini CLI"| C["codex_analysis
verified tools"]
          C -->|"pick_domain.py
profit score"| D["Selected
domain"]
          D -->|"research_domain.py
full pipeline"| E["domain_research
+ domain_plans"]
          E -->|"generate_prd.py"| F["SITE_PRD.md"]
          F -->|"build_site.py"| G["HTML/CSS/JS"]
          G -->|"deploy_site.py"| H["Live on
Vercel"]
          H -->|"mark_built.py"| I["DB updated
site_status=built"]

          classDef done fill:#0d3d38,stroke:#34d399,color:#34d399
          classDef todo fill:#3d2a0d,stroke:#fbbf24,color:#fbbf24
          classDef db fill:#1a1040,stroke:#a78bfa,color:#a78bfa

          class A,B db
          class C,D,E done
          class F,G,H,I todo
Infrastructure & Tools
Database
MySQL on Remote Server
169.197.82.165
SSH tunnel + pymysql
AI - Analysis
Gemini CLI
gemini -p "prompt"
Google Search grounding
AI - Vision
Codex CLI
codex exec -i img.png -
Screenshot analysis
SERP Data
Serper.dev API
POST gl=de, hl=de
Real German Google top 10
Browser
Agent Browser
browse.full(url)
Returns text + PNG path
Hosting
Vercel
Static sites, free tier
Git-connected auto-deploy
Source Control
GitHub
niches-de org
1 repo per site
Rank Tracking
SpySERP
Post-launch only
Track own rankings