7 ms·
Really, all the performance intensive parts are in various c++ aggregator and recommendation type services. But the webserver is Django, yes.
by kleton 3y ago
Really, all the performance intensive parts are in various c++ aggregator and recommendation type services. But the webserver is Django, yes.
- redsky880 3y agoCorrect me if I'm wrong, but Django is not a web server. It's a framework used in conjunction with an app server e.g. gunicorn and a web server e.g. nginx
- taopai 3y agoThanks for your comment. I'm really interested in this topic. How do you know the web server is Django? I searched but couldn't find this. Why would they use Django? I did some small projects in it but I assumed it wasn't very fast and wouldn't be suitable for a big app like this. I would like to know the pros and cons. Why didn't they build up something in C++ or Rust? Won't python limit the speed of responses despite being build the hard stuff in compiled languages? Sorry for being so naive, I am an amateur.
- leodriesch 3y agoIt is fairly normal to build web servers in a way so they can be scaled horizontally (more instances rather than larger instances). So they can just have more containers run their Django servers and distribute the load between them.
- dporter 3y agoAt least a few years ago, most of Instagram’s server side code was in Python. Python may be slower than C++, but it’s not about raw speed it’s about being “fast enough.”
- tinco 3y agoThey deemed getting to the market fast was more important than hardware costs. If their architecture is good, then they were probably really quick and flexible using Python to glue their architecture together which would have been the development bottle neck to getting the service up and running. All the components of the service can be done by separate teams in whatever language they deem most effective.
- DarkNova6 3y agoMost of Facebook is built on PHP. I’m surprised they didn’t choose Laravel.
- yurishimo 3y agoLaravel wasn't created until 2010/11~
- cptcobalt 3y agoLaravel is not a requirement to use PHP effectively.
- DarkNova6 3y agoOh, I know. But as a joke I chose the most well known framework. Personally evren I prefer php over python.
- sangnoir 3y agoIt turns out calling the app "Threads by Instagram" is not just a branding gimmick. The Instagram backend was always Python/Django since before the acquisition.
- mrweasel 3y agoThat's pretty interesting, I wonder if that means that it's build by a team at Instagram, and not Facebook. I'd assume so. I get that Facebook is still a huge success, but I do find it telling that they opted to put Threads under Instagram, rather than Facebook.
- coding123 3y agoWhether it's in C++ or Rust or Python, almost all of any slowness would be from database waiting anyway.
- patrec 3y agoIt wouldn't. Databases are fast, python is slow.
- patrec 3y agohttps://techspot.zzzeek.org/files/2015/pymysql_runsnake.png https://techspot.zzzeek.org/files/2015/pymysql_runsnake.png
- jerrygenser 3y agoNetwork is slow. So you are waiting on the db for most of a request lifecycle.
- pjmlp 3y agoThat depends on how much work is done in stored procedures instead of wasting network traffic and client CPU cycles to process the results.
- patrec 3y agoNetwork latency within an AWS AZ is <1ms and throughput is in the GB/s range. What percentage of python webapps do you think are hitting this as their latency and throughput limit? (Assuming effective DB use of course, i.e. not doing dozens of DB roundtrips to server a single result or getting megabytes of data and filtering on the client etc.)
- chaoz_ 3y agoTrue... When I measured something similar in a large python app, the biggest chunk of time went into python object serialization/deserialization.
- 3y ago
- deleted 3y ago[deleted]
- reissbaker 3y agoI worked at IG on a previous iteration of Threads, with some of the engineers who wrote the current Threads app. It's Django! (Heavily modded, run on a custom Python JIT, and using an extremely custom FB-developed database, also used for IG and FB.) It's Django because IG was originally written in Django back in the day. FB's general approach to scaling is keep the interface generally similar, and slowly replace things behind the scenes as necessary — rather than doing big rewrites. It's seemed to work pretty well. Ultimately the language used for the web server isn't a huge deal compared to the performance of the database, for these kinds of massive social apps. Plus, well, they do have the custom JIT — but that's very new, and when I first joined IG in 2019, we were running vanilla Python in production.
- taopai 3y agoThank you, awesome answer! HN comunity is awesome. Thanks also to every other answer I received for my question, I appreciated all of them.
- kleton 3y agoTo answer the first, because I am no longer bound by nda. Why Django? Because that's what it originally was. Same with YouTube frontend btw. The apps just grew and grew.
- fleetfox 3y agoDjango is WSGI/ASGI framework not a webserver. What do they actually use to terminate HTTP?
- freeplay 3y agoMost likely a reverse proxy of some sort.
- manfre 3y agoNot sure about what Meta is using for Threads, but gunicorn and nginx are a common set up for Django in production. Some will use `python manage.py runserver` in production, and they are using the defacto wrong set up. Don't ever do that.
- Myrmornis 3y agoHow heavily are type annotations and type checking used in the codebase?
- phyrex 3y agoIt’s fully typed and nothing can be merged that doesn’t pass the type checker
- wing-_-nuts 3y agoWhat are the benefits of python with types vs a statically typed language like java, golang, etc?
- jajajajajajajaj 3y agoThe main “benefit” is that Instagram is written in Python and always has been. It’s millions of lines of code, you can’t just change it to be Java one day.
- sgt 3y agoInteresting and lovely to hear they decided to use Django. What is your source, though?
- appplication 3y agoHow is Django now? I’m a long time python dev (data eng space) but new-ish to web dev. I started a Flask project 2 years ago and found it to be pretty full of footguns, mixed messaging on best practices for scalable apps, and the ecosystem feels overbloated with vapor ware extensions. Is this just a Flask problem, or does Django have the same issues?
- coder_san 3y agoDjango is fantastic. It's much bigger than flask, and has one of the best ORMs I've ever used. Also comes with a built in admin panel which is always nice. You will want to use Django REST Framework to make REST APIs, but if getting a site up and running is all you want, it's perfect. If you want to purely make APIs though, give FastAPI a try.
- sgt 3y agoSame opinion. Django is marvelous for getting stuff done.
- SCUSKU 3y agoI use django professionally and for personal projects. I started with Flask and then did FastAPI for a while, but I like django the most since it's the most mature. The ORM and django admin are killer features out of the box that the other frameworks don't have. I will say though that FastAPI is really nice, especially if you need async support. However, I have found that using django ninja [1] adds a lot of nice to haves that FastAPI has to django that makes it much more fun to use again. [1] https://django-ninja.rest-framework.com/ https://django-ninja.rest-framework.com/
- VerminOctopus1 3y ago+1 for django ninja. I built a little side project with it a few months back and was super impressed by how easy it was to get up and running. I didn’t want to leave the Django ORM behind so was very pleased to find this project.
- echelon 3y agoThis would partially explain why it feels so slow. Some page refreshes are taking up to a couple of seconds. It's probably hurried along python code and fairly simple horizontal scaling. Some population of individual instances are probably pegged, and random requests are frequently landing on their backlogged request queues. This seems to have been rushed out the door to capture a unique market opportunity, and they'll clean up the engineering once they get engagement.
- deleted 3y ago[deleted]