3 ms·

- as a data science beginner i am always curious when i see these kinda posts - where did ya get this dataset from (kaggle, somewhere else?) - what type of anal

• Krishnaa_ 8mo ago
  Data source: I scraped it directly from YC's public API. If you go to the YC company directory (ycombinator.com/companies) and inspect the network tab, you'll see it hits an Algolia search endpoint. That gives you structured JSON for every company: name, batch, one-liner, tags, industry, team size, location, etc. I pulled all companies from the last 5 batches (W25 through W26), which gave me 793 companies. For founder bios, I scraped the individual company pages on YC's site, each one lists the founders with short bios. That gave me 1,625 founder profiles to work with. Analysis: A mix of things, all in Python: -> Basic aggregations (counts by industry, tag, batch, geography) -> Trend analysis across batches (what's rising/falling) -> NLP clustering (TF-IDF + KMeans on company descriptions to find hidden themes) Cosine similarity between company descriptions to find competitive overlap ("crowding") -> Cross-correlations between features (is_ai × is_b2b, founder count × hiring, etc.) -> Founder bio keyword extraction to map backgrounds (ex-FAANG, PhD, repeat YC, etc.) -> A simple heuristic classifier for the AI wrapper vs deep tech breakdown Nothing fancy ML-wise — mostly pandas, scikit-learn, and some regex. Built in less than 30 mins using Claude.