14 ms·

Scaling Etsy

• paxys 7y ago
  Does anyone have a version that isn't 37 Tweets?
  • booi 7y ago
    I gave up.
  • conickal 7y ago
    https://twtext.com/article/1194713712516423681 https://twtext.com/article/1194713712516423681
  • merlincorey 7y ago
    The worst part is someone unrolled it on Thread Reader App, and now: > Sorry this content is not available on Thread Reader > It has been removed by the author.
  • redwood 7y ago
    Thanks honestly the way we use twitter has dumbed down our culture so much it's infuriating.
• neya 7y ago
  On an unrelated note, I hate how this has become the trend for long-form content- What should've been a normal blog post is now being chunked into smaller distracting paragraphs with each containing its own paths to deviate from the original topic (comments, retweets, etc.) I wonder what's the motivation though - is it the likes? The informal setting? I really miss those days when we just had content sitting with default font styles inside plain 'ol HTML tables.
  • __Joker 7y ago
    Echo that. I somehow feels this reflects the op's post. We went from blogs to twitter because you know for whatever reasons. Now we are tweeting out blogs as series of threads and using something like ThreaderApp[https://threader.app/ https://threader.app/] to recreate the blog.
  • johnpowell 7y ago
    I made it about three tweets deep and just hit the back button hoping someone would cover the salient points in the comments here.
  • dazc 7y ago
    'I wonder what's the motivation though - is it the likes?' It does look like some sort of attempt to garner likes rather than an innovative means of publishing content? Then again, that's twitter.
  • cellularmitosis 7y ago
    I really don't understand all of the format hate here. Am I the only one with a working scrollwheel? This just seems like such a non-issue.
    • fooblat 7y ago
      I really like to minimize distractions while reading (I love FF reader view). Reading a "long form" post on twitter is like trying to listen to someone tell you a story in a room full of people shouting at each other.
    • iudqnolq 7y ago
      Also it seems as though Twitter never checks their non-logged-in mobile UI. More than 50% of the time I get a generic error message. Sometimes when I scroll down I'm randomly jumped to the top of the page. Comments load non-deterministically.
  • sirn 7y ago
    Though not universally the case, I like to quote @foone's thread on why he publish on Twitter and not a blog post: https://twitter.com/foone/status/1066547670477488128 https://twitter.com/foone/status/1066547670477488128 (Threader: https://threader.app/thread/1066547670477488128 https://threader.app/thread/1066547670477488128)
    • DandyDev 7y ago
      Genuine tip then: write your story on Twitter, then copy paste the compiled story from Threader into a proper blog post and replace your tweets with a link to the blog post. Problem solved!
      • jeremyjh 7y ago
        People expect more from blog posts. Its ok for a Twitter thread to be rambling, to backtrack and clarify the premise halfway through, but this is not accepted in a blog post, which needs to be structured more like a traditional essay and it needs to be edited. Someone without the attention-span or patience to write a blog post in the first place is certainly not going to go back and revise/edit a perfectly good tweet-storm just to make someone on HN happy.
    • gist 7y ago
      I buy into that explanation. In short it implies a few things: 1) It's not about you the reader it's about me 2) I don't intend to have so many tweets but after writing 1 or 2 I am inspired in a way that keeps me going and I end up at 20. 3) I have not figured out a way to channel my creative energy if I am faced with something that feels like work rather than a reaction or to fill an immediate creative inspiration.
  • dannyw 7y ago
    It takes no effort to write out a story. It takes effort to make a Medium blog, and worse still, you have to get an audience for it.
  • jays 7y ago
    For many, it helps build followers and get lots of likes. I’m not sure that was the author’s intention.
  • z3t4 7y ago
    It's the path of least resistance. I used to write one blog post per day, but when I switched to a static site generator I only write one blog post per month. The difference was that before I went to my web site, then wrote the content in a plain textarea. Or sometimes I wrote it in a text editor and copy/pasted it into the text area. Now I have 1) open the CMS 2) Create a new page 3) Fill in some meta-data like title, date and description 4) Write the blog post. Before all blog post was on one long page. Now it's one blog per page. So before I was OK with writing a very short message.
  • Gasparila 7y ago
    The author has a long-form blog going into much of a similar topic: http://boringtechnology.club/ http://boringtechnology.club/. Very nice read
  • pm90 7y ago
    I like it. Every tweet is constrained. Little opportunity to insert stupid memes everywhere. The conversational style is engaging. Asking questions is easy: post a tweet.
• deleted 7y ago
  [deleted]
• maxpert 7y ago
  This reminds me of similar debate I had with one of my friends falling for NodeJS FOMO. The actual codebase was RoR and he wanted to go for Node for no apparent reason. I was able to talk him out of it but what I understood from discussion was peer pressure of “hey all of this could have been async”. While I admit blocking threads might not be permanent solution, they can still take you pretty far. Not to mention the folks wanted a MERN stack and I believe leaving Postgres for Mongo without a damn good reason is just crazy idea.
  • demosito666 7y ago
    Isn't SQL -> mongo a meme now? I mean, are people still seriously considering this outside of one or two very specific use cases? At my workplace the only outcome of this would be an awkward silence and weird looks.
    • aniforprez 7y ago
      Yeah having worked on a mongo stack there are N problems caused due to the unstructured nature of mongo and there's so many problems with existing ORM solutions chiefly the fact that they're trying to create structure where it doesn't exist. This was a decision taken aeons ago and there's not much we can do to change it now but hope the guy who made that decision is happy cause I sure am not
      • celim307 7y ago
        Only thing I’ve found mongo good for is the warm data path where you need slightly more permanence than a straight pub sub (like getting the last 2 minutes of events upon connection) but you really don’t care about throwing the data away after that
• malisper 7y ago
  As someone who scaled the database at a company that dealt with over a petabyte of data, here's a few thoughts I have: It seems like the biggest issue Etsy had was a lack of strong engineering leadership. They relied on an outside consultant to rewrite their application in a fundamentally flawed way. The lack of eng leadership resulted in both poor technical decisions being made and bad engineering processes for how the app was developed. Etsy's biggest technical mistake building an application using a framework that no one understood. This led to the predictable result that when they deployed the application to production, it didn't work. Even if the application had worked, Etsy still would have needed to maintain the Twisted application indefinitely. Maintaining an application written in a framework no one understands sounds like a recipe for a disaster. Sooner or later you are going to run into issues that no one will know how to fix. Process wise, Etsy made the mistake of not properly derisking the Twisted application. They only found out that it didn't work when they were deploying it to production. They made the same issue a second time when they tried to deploy the replacement. When I'm building a new service, the first bit of code I write is to test that the fundamental design of the new service would work. Usually this only takes a few days instead of the months it takes to implement the full service. It sounds like Etsy could have setup a simple Twisted app that did a fraction of what the final version did. If so, they would have found a number of fundamental flaws in the design before having spent a ton of time building out the service. To be honest, this story shows how a business can succeed even with a bad engineering team. It would be one thing if this sort of incident killed off Etsy. Instead Etsy has gone on to become the $5 Billion company it is today. I'm not saying engineering doesn't matter. All I'm saying is you can build a pretty successful business even with a bad engineering team.
  • _the_inflator 7y ago
    > All I'm saying is you can build a pretty successful business even with a bad engineering team. I liked your comment. I would say "bad engineering". Look at all the banking stuff (disclosure: finance/banking lead) and you will immediately recognize this as a reality. Why? Because there are no KPIs that relate to speed, sustainability, maintenance. There is a distinction between input, output and outcome. Also a lot of architecture is done using power point. Drawing a line in PP is easy, connecting the services in reality is often times anything else than trivial.
• excerionsforte 7y ago
  Ha, Spouter a middle layer for talking to DB instead of just talking to DB (proxy to do connection pooling, etc could be between). This kind of project came up at my previous job and while I didn't know about Spouter at the time, I knew I didn't like the idea one bit. Managing a DB + middle layer for a small and an already stressed DB team? Can't imagine that going well. The problem was that they spent too much time doing reviews for simple SQL patterns for CRUD + operational issues like debugging database performance issues or watching devs perform migrations. Of course they have other things to do in their job. My opinion was the root of the problem is database choice and practices. If all I want to do is simple CRUD, then give me good scaled Redis cluster (AOF enabled), dynamo or something that constrains the query model. DB team can worry about about the cluster management leaving me to worry about structure. I could consult DB team if I needed opinions for multiple cases. Give me a way to watch db performance as well, so DB team does not need to watch over a migration at 12am or some off period. Sometimes I may need higher performance or more dynamic queries, so just creating a table or elastic search with only indexable values to get the ids works. Use those IDs to fetch from original store.
  • AmericanChopper 7y ago
    > My opinion was the root of the problem is database choice and practices. If all I want to do is simple CRUD, then give me good scaled Redis cluster... Throughout my entire career I have come across very few simple crud applications that will actually work properly with denormalized data structures. It’s not even about scaling users, it’s about scaling your schema. The very first instance you encounter where a document has a nested array will start to cause problems, and the only way to solve it is to push more complexity and state management onto the client. Which is bad interface design, and quickly gets out of control. If you really do have a simple crud app, and don’t want to put too much effort into running your RDBMS, just use MySQL imo.
    • excerionsforte 7y ago
      I was working at a company of like 400 employees using MSSQL, so simple CRUD is not limited to simple crud app. Simple CRUD operations are insert, select, update, delete. Given that we auto generated the code and sql (stored procedures) and pasted it into the codebase made me think, why bother with this if I could develop new features without SQL at all. Why should my queries be a stored prod or prepared stmt if all I need is get, insert, update, delete an object. Yes, the auto generated code wrote to the database as an object as in all fields included in the query. Why? There's no transactions, foreign keys, etc. since some tables where stored in different databases. Using a relational db as object store included people inefficiently using it as an object store. In my own projects, if I need the speed, columns are indexed while unindexed data is binary. I can explode out (unserialize) the binary data into cache. Point of my post was centralizing database access tends to be a bottleneck for team efficiency. If not the devs then the db owners. Augment my abilities to execute don't try to completely change it where it brings a whole set of unwanted problems that were not there before.
• sethammons 7y ago
  I can sympathize. We have a Twisted database abstraction layer that was built in house. You call http endpoints that perform database operations. It has routing logic so applications only need to call someEndpoint.json and not worry if we've sharded the backend recently, scaled read replicas, migrated a write master, etc. It has caching, query piggy-backing, connection pooling, and other bells and whistles. While we have been able to scale it out for volume of requests, it did not scale well for multiple teams and multiple services. It became a single point of failure whereby the access patterns of one team to their own databases could affect other teams abilities to access their own databases. We've since moved to a new model whereby team's own their datastores and access methods, speeding up development and reducing negative impact teams can have on each other's data layer. Legacy access continues to be migrated off endpoint by endpoint, database by database. I look forward to the day of never having to look at Twisted code ever again. The framework is aptly named. If you are looking for a similar solution for MySQL, I currently recommend ProxySQL. It allows for a topography whereby teams can control their own proxy layer and still have most of the benefits outlined above.
• durnygbur 7y ago
  Get a blog, mate.
  • mcfunley 7y ago
    Got one brah
• PragmaticPulp 7y ago
  It’s amazing how relatable these anecdotes are. Change a few key words here and there, and this tweetstorm could describe most engineering leadership failures I’ve seen. Nearly every tweet describes scenarios that can only happen when engineering management is M.I.A. or too inexperienced to recognize when something is a bad idea. - Attempting to solve problems with a rewrite in a different programming language. This can be the correct long term decision in specific scenarios, but it’s rarely the correct answer for bailing out a sinking ship. You need to focus on fixing the current codebase as a separate initiative rather than going all-in on a rewrite to fix all of your problems. Rewrites take far too long to be band-aid solutions. - Rewriting open-source software before you can use it. If Django doesn’t fit your startup’s needs, the solution is never to rewrite Django. The solution is to use something else that does fit your needs. Your startup’s web services needs are almost never unique enough to merit rewriting popular open-source frameworks as step 1. Pick a different tool, hire different people if necessary, and get back to focusing on the core business problem. Don’t let the team turn into open-source contributors padding their resume while collecting paychecks from a sinking startup. Save the open-source work for later when you have the extra money to do it right without being a distraction. - Hiring consultants to confirm your decisions. Consultants can be valuable for adding outside perspective and experience, but the team must be prepared to cede some control to the consultant. If you get to the point where you’re hiring a “Twisted consultant” instead of a web store scaling consultant, you’re just further entrenching the team’s decisions. - “Nobody was in charge”. Common among startups who pick a “technical cofounder” based on their technical skills, rather than their engineering leadership skills. When you assemble a team of highly motivated, very smart engineers, it’s tempting to assume they can self manage. In my experience, the opposite is true. The more motivated and driven the engineers, the more you need explicit leadership to keep them moving in the same direction. Otherwise you get the next point: - Multiple competing initiatives to solve the problem in different ways. Letting engineers compete against each other can feel like a good idea to inexperienced managers because it gets people fired up and working long hours to prove their code is the best. That energy quickly turns into a liability as engineers go full political, hoarding information to help their solution succeed while casually sabotaging the competing solutions. In a startup, you need everyone moving in the same direction. It’s okay to have disagreement, but only if everyone can still commit to moving in the one chosen direction. If some people can’t commit, they need to be removed from the company. - The “drop-in replacement” team. This is just a variation of having engineers compete against each other. Doesn’t work. - Allowing anyone to discuss “business logic” as if it’s somehow different work. This leads to engineers architecting over-generalized “frameworks” for other people to build upon instead of directly solving the company’s problems. At a startup, never let people discuss “business logic” as something that someone else deals with. Everyone is working on the business logic, period. I have to admit that when I was younger and less experienced, I plowed right into many of these same mistakes. These days, I’d do everything in my power to shut down the internal competition and aimless wandering of engineering teams. Ironically, these situations tend to benefit from strategically shrinking headcount. It’s not a fun topic, but it’s crucial for regaining alignment. The key is to remove the dissenters, the saboteurs, the politicians, and the architects creating solutions in a vacuum. You need to keep a cohesive core team that can move fast, commit to one direction even when they disagree, and not let their egos get in the way of doing the right thing. The real challenge is that those employees tend to fly under the radar. The people quietly doing the important work and shipping things that Just Work can be overshadowed by bombastic, highly opinionated “rockstar” engineers. Founders need to be willing to let those rockstars go when they no longer benefit the company, no matter how good their coding skills might be in isolation. A coordinated team of mediocre but diligent engineers will run circles around a chaotic team of rockstars competing against each other.
• allard 7y ago
  "Interesting" "feature" of their platform that I see monthly, and more recently given the season. I get a message like "where's my table?" But I'm not a seller. My username is simple and a common first name. I've approached them a few times, and there's no interest in fixing it. Yesterday, I found where I could open/sever the connection between those messages and my email. It's been going on for years. Many confused users, although I have little idea how long they stay that way.
• etxm 7y ago
  > an infamous incident where one of the investors had to drive to Secaucus to physically remove the other engineering founder from the cage. I really want hear this story.
• vegetablepotpie 7y ago
  >I managed to not get fired because through this whole thing I was talking shit about it. I know it’s unprofessional to comment on other developers work negatively, but I do see it often in the work place and it does serve to distance an individual from the group that is failing. It’s sad but true, and this is an example, employers say they want professionalism but they incentivize against it.
  • pm90 7y ago
    There’s 0 downsides to criticizing everyone else’s projects and systems so a lot of people do it. This is kinda exacerbated by the fact that we engineers have a reputation for being incorrect about the stability of systems we build ( there are good reasons for this, and I personally don’t think it’s possible to avoid that impression even when it’s not intended). I wonder if this team would have done better with tools that offer better observability (metrics, logs, tracing etc.). An example of a rewrite that went well https://avc.com/2019/12/grinding/ https://avc.com/2019/12/grinding/
• celim307 7y ago
  I’d love to hear more stories like this
• mcfunley 7y ago
  OP here—if it’s not obvious from the tweets the timeframe of this story is 2007 through 2008.