3 ms·
I can only guess the format for archived tweets is very different from newer one, maybe just a compressed text file, which the engineers have to automate extrac
by mmahemoff 14y ago
I can only guess the format for archived tweets is very different from newer one, maybe just a compressed text file, which the engineers have to automate extracting and parsing at scale. Doesn't seem that hard, but the format was probably a hack job back in the day and there are probably a lot of considerations, e.g. metadata like RTs and favorites.
Plus, CEO's comments sound like it's been set up as a side project. The kind of management style books like Mythical Man Month and Peopleware warned us about.
- aes256 14y agoIndeed, this would also explain why the search function only allows one to view tweets going back seven days.
- dylanvee 14y agoI interviewed with a Twitter search engineer and he told me that that's because their in-memory search index is not big enough hold all tweets ever posted. On top of that, their efforts to scale up the index are counteracted by the ever-increasing volume of tweets posted per day.
- diego 14y agoAnd the reason it's not big enough is because they don't want to invest in it. They found out search is not as monetizable as they thought it would be, and at the same time they cannot take away the functionality. The compromise is to keep what they have.