GitHub Repository Scraper

A GitHub data extraction tool that scrapes repository metadata, contributor activity, and trending projects for market research and developer tooling.

Repository Data:

• Repo name, description, and topics
• Star count, fork count, and watchers
• Primary language and full tech stack
• License type and README content
• Last commit date and activity level
• Issue and PR counts (open/closed)

Search & Discovery:

• Search by topic, language, and star count
• Trending repos by timeframe (daily/weekly/monthly)
• Organization repository mapping
• Dependency graph extraction
• Similar repo recommendations

Contributor Analysis:

• Top contributors with commit counts
• Contributor email extraction (public commits)
• Contribution frequency and patterns
• Bus factor calculation
• Sponsor/funding information

Market Intelligence:

• Technology adoption trends
• Framework popularity tracking
• Open-source competitor analysis
• New project launch detection
• Developer hiring signals

Tech Stack:

• Python + GitHub REST/GraphQL API
• Rate limit management and token rotation
• CSV/JSON export
• Docker deployment