10 ms·
Astro framework lowers competitors by ~40% on performance graph to looks better
- jjdeveloper 3y agoAstro is great … but agree should never mislead users.
- garblegarble 3y agoThe width values in their HTML make this pretty clear, too. This JS will show the width value for each bar on their homepage[1]: document.querySelectorAll('div.noise-underlay').forEach(e => e.innerText = e.style.width); [1] https://astro.build/ https://astro.build/
- djfdat 3y agoAstro 97.0149% Score:98 Gatsby 52.2388% Score:68 Next.js 44.7761% Score:63 WordPress 37.3134% Score:58 Nuxt 31.3433% Score:54
- jjkeddo199 3y agoMy humble NextJS app is 100/100 on Lighthouse, I am really hesitant to believe that it scores so low
- Rauchg 3y agoSo is my own site: https://twitter.com/rauchg/status/1652093980958683137 https://twitter.com/rauchg/status/1652093980958683137 Scores need to be adjusted by and looked at through the lens of complexity. Next.js powers some of the most sophisticated web applications in the world, not just your blog and mine. e.g.: the instant you bring "necessary evils" like display ads, your scores WILL suffer.
- FireInsight 3y agoThe statistics are not by Astro or an affiliated entity. The graph shows average "real world speeds" on sites developed with different frameworks. The bias we're seeing here is that NextJS is more popular and slow sites are being made with it, while people who choose Astro are more inclined to value the speed of their site.
- TZVdosOWs3kZHus 3y agoSince there are no tics on x-axis, it is not wrong. But I see your point.
- bunga-bunga 3y agoMathematically not wrong, but most definitely misleading/unethical. I don't think this was done unintentionally either, because the graph would still be perfectly readable without this change (as opposed to "a few points variation" where such zoom would be required) It's perfectly fine to display data this way, but they should make it clearer that the start isn't 0 by using wiggly lines or by labelling the origin.
- OJFord 3y agoSeems an odd choice of deception, since the real graph really would just compress the competitors closer together, wouldn't make any of them look that much better really?
- notahacker 3y agoIt'd also make all them look closer to Astro in level of performance (not that the benchmark is particularly scientific anyway)
- OJFord 3y agoA bit, but even in the actual numbers, 98 compared to 68 is pretty decent? But yes, of course I'm not commenting on how fair the test was anyway, just the presentation.
- fxtentacle 3y agoIt's the same trick that Apple and Dell use. The left side of the graph is not 0% but out of habit we think it is.
- pacifika 3y agoWhat’s the left side offset? It makes no sense to me at any number
- jerf 3y ago"The left side of the graph is not 0% but out of habit we think it is." This is a learned behavior and it can be fixed. I'm mostly broken of it; I assume it's not until I see that it is. Only mostly, not claiming to be a bias-free superhuman here, but as a for instance I had to check the comments to see what was offending people about this chart because I didn't assume it was anchored at zero in the first place. I recommend making the effort if you can; most charts out of school aren't anchored at zero, many wouldn't even be useful if they were. What jumped out at me actually was the lack of units. At least it links to the original report, and I think this is the Lighthouse Performance Score, which... is itself in units I don't understand? It's some sort of percentage, not speed measure, I think... looks like percentage conformance to an SEO/performance test suite? I'm not sure this is really something that can be bar graphed in the first place sensibly. Percentage conformance to test suites is at least a marginal case at best for a bar graph... how "numeric" such a quantity is is very debatable. e.g., a 40% compliant product may in fact be "less" compliant than a 30% compliant product if the former is compliant on a whole bunch of incidental check points but misses the most important ones, where the latter is compliant on the important ones and misses the incidentals, so such "percentages" can break simple numeric properties like comparability or the transitivity of such comparisons and really aren't numbers. Compliance to such a test suite will also be highly dependent on how you use the system; odds are its competitors can be tuned to go higher without much work and that very easy-to-make mistakes will send a Astro-based site plummeting because of what a user might naturally do with it, through no fault of the framework.
- pizzapill 3y agoI looked at this chart just yesterday for a quick evaluation. The statistical axis deception was totally lost on me. However my first thought looking at this graph was that they don't compare themselves to the fast competition.
- thealchemistdev 3y agoYeah. Where's Zola or Hugo?
- FireInsight 3y agoWhile Astro technically competes directly with Zola or Hugo, a comparison with them would not make sense. The chart is not about build times or anything like that, it's about the average performance on real world sites built with the framework. Astro with first-class support for interactivity through JS frameworks, and triest to compete with other JS frameworks in speed. While Hugo and Zola can generate light and heavy websites easily depending on the developer's choices, Astro aims to express that it's easier to create fast websites, even ones that are interactive, with Astro than with a fuller framework like NextJS.
- thealchemistdev 3y agoYeah. I thought about that right after I commented, but their website says "content-focused" not interactivity. Every frontend developer who uses a framework (myself included) will eventually learn the hard lesson of "use the effing platform". Any barrier to serving and delivering HTML, CSS, and JS (for interactivity only where you need it) is a waste of time and energy. Maybe not at the start, but most certainly in the long term through package depreciation, repo weaponizations, or political activism. The core web standards are designed for future and possibly non-human archaeologists to be able to view just about every web document created since 1991. World's first website. http://info.cern.ch/hypertext/WWW/TheProject.html http://info.cern.ch/hypertext/WWW/TheProject.html So if the job is to deliver content focused websites, then the best tools are those that are designed to get out of your way. Astro and the every other tool that is not a single executable is completely outclassed due to their reliance on package management ecosystems. People want fast consumption and production? Send fewer bytes over the wire, use less bytes to produce them, and let the platform do its job. But this is just like my opinion, man. None of this matters if the power goes out. Fun game, though.
- I_am_tiberius 3y agoDoes twitter work again without account? Edit: It seems specific Tweet URLs and the status page work but the timeline does not.
- adewinter 3y ago[flagged]
- mynameisvlad 3y agoHow is their comment a complaint? If anything, they seem to be praising tweets being available for unregistered users now.
- add-sub-mul-div 3y agoThere deserves to be a serious discussion of whether Twitter content should even be allowed here anymore if it can't be seen. We already hated it before the acquisition, and it's gotten worse since then.
- onlypositive 3y ago[flagged]
- add-sub-mul-div 3y agoThere's at least substantive content behind an actual publisher's paywall.
- onlypositive 3y agoSure there is. Where's the NYT article on the grusch hearings? Journalism? Try establishment mouthpiece.
- orra 3y agoIn the UK this is almost a meme. Search for “Only the Lib Dems can win here”, for mathematical charting bemusement.
- bigpeopleareold 3y agoI don't know anything about Astro, but that graph reminded me of the Monty Python animation of Crelm toothpaste ... seems like Astro only provides 98% protection!
- reaperducer 3y agoI don't know anything about Astro, but that graph reminded me of the Monty Python animation of Crelm toothpaste ... seems like Astro only provides 98% protection! "Have you ever seen a hospital that doesn't have string?"
- KingOfCoders 3y agoI was enlightened about this decades ago by Edward Tufte [0] - sadly since then I get annoyed by most graphs I see, where they exaggerate the difference in values with a non-zero axis. [0] https://www.edwardtufte.com/tufte/books_vdqi https://www.edwardtufte.com/tufte/books_vdqi
- sokoloff 3y agoI came across this example in the last week and it cracked me up: https://www.boredpanda.com/average-women-height-data-chart-latvian-indian-women/ https://www.boredpanda.com/average-women-height-data-chart-l... (non-zero origin, but the graphics are zero'd to the axis)
- zero_shift 3y agoI've seen a lot of shenanigans in JavaScript framework benchmarks. For example when Pino (logging framework) compared itself using buffered mode (where the logs don't immediately pipe to stdout and a crash wipes x kb of logs). I don't know if the JS community is specifically vulnerable to this, or it's universal. It seems to happen a lot though.
- klysm 3y agoJS is in a state where it is so flooded with frameworks that they have to compete heavily in marketing and hype generation. Unfortunately that incentivizes shady and misleading practices like this.
- helf 3y ago[dead]
- toastercat 3y agoI just want to say that open-source projects like these that have these fancy marketing pages and rely on a huge amount of marketing tricks & twitter reposts makes me less likely to use them. Unfortunately, this is all too common in the JS world these days.
- draw_down 3y ago[dead]
- sureglymop 3y agoI highly agree. The main web frameworks I use are Svelte and Sveltekit. No comparison charts to other projects, easy to use, simple, created by a seemingly normal guy with sane opinions out of need for something like it for himself. It's not the end all be all, but works well for me. The JS world is pretty weird though. It's not only JS frameworks popping up in rapid succession but also browser features and proposals developing quickly. And what's really weird is how some new features rely on JavaScript, such as the newer <dialog> HTML element.
- bongobingo1 3y ago> No comparison charts to other projects At least, none they publish first. https://youtu.be/AdNJ3fydeao?t=1083 https://youtu.be/AdNJ3fydeao?t=1083
- revskill 3y agoYou seems misunderstand the role of web framework. People want better abstraction than HTML, and JS is the only simple way. Why i should use HTML input while i have better abstraction for it ? TLDR is, HTML is the issue, not JS frameworks.
- mattgreenrocks 3y agoIt’s amusing they have to resort to this when the quality and DX of a framework are very real things that prevent them from being fully commoditized.
- hoofhearted 3y agoCould you elaborate more on what you mean? I’d like to hear more of your thoughts on this.
- mattgreenrocks 3y agoThe experience of using a framework or a library is a very real and tangible thing. You might have 5 different contenders for a particular task and they all exist at different spots in the design space. Those differences manifest as real pains/joys when used by devs. Market those! I can see perf being a marketing point on some things. But for static site generators it feels like a symptom of the bar of quality being too low, or everyone just cargo cults Apple’s marketing schtick.
- hoofhearted 3y agoI believe I understand what you are saying! :)
- vlugorilla 3y agocan't see that link, since I don't have (and don't want to) a twitter account
- lpapez 3y agoI don't have it either but the tweet loads fine?
- cde-v 3y agoYeah same, I clicked then half a second later realized it probably wasn't going to load... then it did. Seems like Enron might be backtracking already.
- mentalpiracy 3y agoas of a few hours ago[0] twitter posts can be viewed without login again [0]https://www.engadget.com/twitter-quietly-backtracks-on-requiring-users-to-log-in-to-see-tweets-135230558.html https://www.engadget.com/twitter-quietly-backtracks-on-requi...
- SigmundurM 3y agoThis is roughly what the graph should look like: https://imgur.com/a/e09wCw3 https://imgur.com/a/e09wCw3 (Did this by just changing the width percentages of the bars to the numbers shown on the graph itself.)
- bogwog 3y ago> This post may contain erotic or adult imagery. By continuing, you acknowledge that you are 18+ years of age. What's the deal with that?
- SigmundurM 3y agoNo clue. Haven't used Imgur in ages tbh. It did ask me if the image contained matured content, to which I said no. But for some reason it still marked it as mature? Very weird...
- itzworm 3y agoImgur is used for both regular content and adult content. They can't guarantee a random picture uploaded by a random user isn't porn. (though they give a best effort). Additionally the side bars show what's most popular. In theory something NSFW could end up with a thumbnail on an unrelated picture. So they give you the warning ahead of time. (I've used imgur tons and never seen this.) This picture was a graph.
- rizky05 3y ago[dead]
- jeroenhd 3y agoRemember, any graph without properly labeled axes and units is trying to mislead you! This goes for Javascript frameworks, startups trying to get their first customers, Apple performance graphs, and any other type of commercial! I've started adding companies I've caught doing this to my Pihole blocklist so I can stay away from them myself, but the sad truth is that most people won't care about the lies and misdirection because advertising has normalized this type of behaviour.
- dmix 3y ago> but the sad truth is that most people won't care about the lies and misdirection because advertising has normalized this type of behaviour. This is probably the greatest time in modern capitalist history where this is wrong. If anything the youth have grown up being assaulted with marketing and become savvy dechiperers and careful. My parents generation were more easy suckers. Being honest matters a lot on social media. People love nothing more to expose this sort of thing (drama!) and point at a company to laugh at them. This thread on HN and Twitter is good proof. Although one could argue the no press is bad press angle.
- hoofhearted 3y agoI really didn’t hear about Astro until more recently after I started playing with blogging using Next.js. I gave Astro a look over and promised myself to be open minded; but I wasn’t really digging how opinionated their code was to themselves. Thousands and thousands of devs built on top of each others JavaScript improvements for years, and then Astro comes along and decides that us developers need to learn a new “.astro” syntax as well. Imagine if Tailwind or Laravel forced you into funky .tailwind/.laravel named files because they felt a certain way about their name. I took this as a sign that Astro might become a very money hunger shark at the first drop of blood like other popular CMS providers have done in recent memory. I’m working on a free open source side project now for fun with the goal of helping more people create and own their own content online, without paying an arm and a leg for it. I’d love more support and feedback if anyone has any to give :) If you feel that developers need an open and extensible product for creating content, please give me a star, and open some PR’s and issues. https://github.com/elegantframework/elegant-cli https://github.com/elegantframework/elegant-cli
- dmix 3y agoI develop my company front end in Vue 3 and as much as I love Vue I'm constantly looking over at React with a bit of jealousy. There is massive value in being the most popular framework. Even if it's not as nice as the other ones just the pure scale and quality of the 3rd party packages, plug and play integrations, component libraries, support posts, etc... that stuff really matters when it's your day job. One big one that's burned me is a lack of (well supported) Vue SSR integration in Ruby on Rails. I'd personally just stick to Next.
- hoofhearted 3y agoYeah I hear your complaints about Vue 3. I had the same experiences when I was building a few apps with it pre 2020. Next/Nuxt really take Vue and React to a whole new level when building a wet application. I’m very well versed in all 3 of big reactive JavaScript frameworks, and React is my go to for now. I switched from Vue to React as my default after struggling with complex data tables in Vue; when React had a bunch of libraries on GitHub that would work for my project at the time. The final nail in the coffin was a contract requirement for .Net core framework, and it came bundled with React in it already. From there, I found Next, Typescript, and Tailwind, and haven’t looked back :)
- bbbbbenji 3y agoHere is how they are calculating the actual axis refinement [0] const scoreWidth = (Astro.props.score - 33) * (100 / (100 - 33)) [0] https://github.com/withastro/astro.build/blob/main/src/pages/_components/CoreWebVitalsGraphItem.astro#L7C20-L7C65 https://github.com/withastro/astro.build/blob/main/src/pages... > (Astro.props.score - 33) * (100 / (100 - 33))
- dmix 3y agoThe benefits of OSS marketing sites. You can make a PR/issue and point directly at their sneaky code.
- derkoe 3y agoSeems like someone has already done that: https://github.com/withastro/astro.build/pull/730 https://github.com/withastro/astro.build/pull/730
- wallmountedtv 3y agoAnd its been merged now haha. You can see the change too on https://astro.build/ https://astro.build/ It's awesome just how quickly this got addressed.
- endigma 3y agoHappy to help
- ricardobeat 3y agoHave you checked it on mobile? Now it’s skewed the other way due to the space reserved for the label.
- endigma 3y agoI'm aware, this was actually an issue prior to my PR as well, but less obvious. I think this would need some more defensive CSS to maybe shift the label around to the inside of the bar? Or perhaps the labels are unnecessary (it is a p90 of an average of a ton of lighthouse scores after all, not like the number quantifies to much) Edit: https://github.com/withastro/astro.build/pull/731 https://github.com/withastro/astro.build/pull/731
- tryfinally 3y agoThis is inexcusable, and provides no benefit other than intentionally confusing the reader. I wish companies were more afraid of using blatant dark patterns in whatever they publish. I’m sure there’s a solution somewhere out there - perhaps a crowdsourced website akin to the Google Graveyard[0], or certain fact checking sites, where bad faith decisions could be archived and accumulated until it becomes obvious which companies are simply not worth the trouble of working with. [0]: https://killedbygoogle.com/ https://killedbygoogle.com/
- planede 3y agoBar graphs not starting at 0 are misleading, even when the axis are labeled. The point of a bar graph is that you can intuitively compare the visual weights or areas of the bars, which doesn't work if they are cut off. Not labeling the axis is of course even worse.
- azangru 3y agoHey, Twitter lets anonymous users view tweets again!
- fckthisguy 3y agoOnly if the URL links to a specific tweet. The feed, and users' post history requires a login.
- ThatMedicIsASpy 3y agoI would prefer a ban. I can't view them no matter what since I block twitters cookies.
- shadowgovt 3y agoThis would be in the "damn lies" category of the "lies, damn lies, and statistics" aphorism.
- rcme 3y agoI feel like this is an intentional error designed to illicit exactly the sentiment expressed by many people commenting here. That is to say, this is manufactured outrage. And look at the results: Astro made it to the front page of HN!
- eis 3y agoThe report [0] this graph was based on also has major issues as it does not consider versions of the frameworks and includes sites using old versions of Next or Nuxt compared to only new sites for Astro as it's a new framework. There have been major changes for example between Nuxt2 (Vue2) and Nuxt3 (Vue3). They at least disclose that at the end of the report but it still leaves a bad taste because it should have been possible to differentiate these without too much work and they were aware of the problem. Another issue is that Astro seems to be used a lot for static sites whereas the others mostly for dynamic ones. I feel like it's really not a fair comparison. [0]: https://astro.build/blog/2023-web-framework-performance-report/ https://astro.build/blog/2023-web-framework-performance-repo...
- totalhack 3y agoAren't they comparing to an old version of Nuxt too? From what I've seen on Twitter Nuxt 3 should be pretty close, no firsthand experience though.
- __MatrixMan__ 3y agoThe web needs an annotation feature so that we can apply the "misleading" tag to that chart. It would have a link to details about why. That way (if you trust the annotator) it jumps out right there on the marketing page instead of requiring you to browse HN or twitter to find out. It's like we have a fire alarm going off in a building other than the one that's on fire.
- nvln 3y agoHope they correct it and don't repeat the mistake. They really don't have to. I really like astro. They got a lot of things right. It is my go to choice for websites these days. I say this as a creator of one of the popular Static Website Generators.
- woodpanel 3y agoNever used Astro but, wow, the comments here are meaner than the „crime“. AFAICT this wasn’t done with the intention of outright deceit. all this while the stats themselves seem to be correct. It‘s probably done to provide a more interesting graph. Something quite common in infographics. Playing devil’s advocate here, if you aren’t able to see that a number higher than 50% isn’t supposed to be left of center, you’re probably in the wrong business anyway. Arguably it would have been more innocent if they indicated the shift by cracked bars [1], but then this is probably not provided by the charting library they use (or any that I know of). [1] https://peltiertech.com/images/2011-11/Ybroken.png https://peltiertech.com/images/2011-11/Ybroken.png
- Havoc 3y agoIs that not just the classic marketing gimmick of not using 0 as starting point on graph? Cheeky indeed but not sure I’d phrase that as “lowering competitors”
- lcnPylGDnU4H9OF 3y agoAt a glance, it looks like this is not the case but a ruler could reveal more. The distance for the 35% difference between 33 (first red line) and 68 (where the first gray bar ends) seems to be roughly half of the distance for the 30% difference between 68 and 98.
- cozzyd 3y agoplotting frameworks that default to suppressed zeros are terrible. (ok, they pretty much all do this...)
- wg0 3y agoThese frameworks have fancy websites. Framework themselves aren't something you can jott down on a weekend. I mean so many. What's in it for them? Bunch of free souls just showcasing their excellence or what's at play? Or there's hidden underground cash flow rivers that a casual observer misses totally? Genuinely oblivious and curious.
- dahwolf 3y agoBaseline framework performance is pretty much bullshit in any case. Real world performance degradation happens due to poor coding, endless dependencies, large amount of API calls that block rendering, and a huge amount of 3rd party (tracking) tags. The core framework is the least of your problems.