7 ms·
Wild claims about K performance
- deleted 5y ago[deleted]
- tosh 5y agoThe notes by mlochbaum were also discussed briefly in the most recent ArrayCast episode on k https://www.arraycast.com/episodes/episode-08-attila-vrabecz-and-the-k-group-of-programming-languages https://www.arraycast.com/episodes/episode-08-attila-vrabecz...
- anonu 5y agoSo the author is incredulous but provides no proof or stats, just a long rant. Another overlooked "performance" feature of terse languages is that of the developer. Vector based, concise languages give you more in one line of code. And I don't find that readability suffers. Yes you need to stare at that one line longer, but there's more going on.
- eatonphil 5y ago> So the author is incredulous but provides no proof or stats, just a long rant. I thought the same thing but on the other hand, what can you do? If you're not allowed to publish benchmarks you only can rant like this even if it sounds petty. By getting attention here on HN it helps put pressure on commercial vendors to stop sharing unconfirmable benchmarks otherwise they will lose credibility since they've been called out. The author does make some good points about the fundamental nature of interpreters too. Although it's hard to confirm there too what these languages do without seeing their source. I mean that if K or some language is actually interpreting without a virtual machine at some stage, that is indeed going to be slower than any language with code running on a virtual machine. And also that any code that runs on a virtual machine must be slower than an equivalently intelligent compiled code since it's running on another layer of abstraction that compiled code isn't. ("equivalently intelligent" because yes a virtual machine could be faster than terrible compiled code.)
- sz4kerto 5y ago"And also that any language with a virtual machine must be slower than an equivalently intelligent compiled code since it's running on another layer of abstraction that compiled code isn't. " I think this is not correct. A VM has access to the data that the code is operating on, while an AOT compiler doesn't. It's possible that a VM (eg. JVM) produces faster code than a normal compiler.
- icen 5y agoSadly it also has to consider the time it takes to compile code like that - we grumble, but not too loudly, at the time LLVM takes to compile things. If the JVM took that same time whilst running live that wouldn't be acceptable.
- WJW 5y agoSurely this depends entirely on the total runtime of the program. A server-style program which will run for a couple of weeks can accommodate quite a bit of JIT compilation overhead.
- eatonphil 5y agoI'm talking about where the code is run. If it's run on a VM it's run on a VM. If it's run on the cpu by JIT-ed code it's not running on a VM. Edit: But fair I did say "language with a VM" which would include Java JIT-ed or not. Edit, edit: Modified the original comment to say "code on a VM" not "language with a VM".
- theelous3 5y agoI'm not familiar with k, and I'm not understanding this whole "no benchmarks" thing. So it's a proprietary lang? What stops benchmarks being run? Just how ridiculous are the licenses? Why has seemingly nobody run them anyway?
- eatonphil 5y agoI think the article answers all your questions.
- tosh 5y agoConcise languages are also more fun to use in a REPL (another aspect of "performance")
- icen 5y agoYou might enjoy the language created by the author of the article, then. BQN is a language that favours terseness, even above and beyond k. It is in many ways an improvement on APL and J, importantly through a more convenient set of operators than found in either.
- rak1507 5y agoPersonally from what I've seen I wouldn't say BQN values terseness more than k, probably less than most other array langs. (Edit: if you downvoted this: why?)
- icen 5y agoI agree; I don't think BQN values terseness more than k. I think k values terseness a lot. I think it may be more convenient to write terse code in BQN over k, because there is greater facility to control function calls in a terse way. Things like under and over are quite common patterns that BQN finds easier (and shorter) than k. I've not spent a great deal of time yet with BQN, so this could be wrong.
- rak1507 5y agoYeah, under and over are great (wish k had more 'combinators' in general), but k does manage to cram a lot of stuff in with overloads. I feel like k is terser in general but I don't really have much evidence to back it up, just from playing a bit with both of them.
- mlochbaum 5y agoThe different character sets make it impossible to say what this would mean. My limited code golf experience is that most often BQN solutions use fewer characters than K ones, but BQN has 63 primitive glyphs to K's 25 or so (depending on version). I do think K design values using its limited resources to enable terse code more than BQN, but if screen space is what you care about then BQN will tend to use less of it.
- tomp 5y agoThe author explicitly states that K licence prevents him from providing stats.
- gd1 5y agoThere seem to be benchmarks available in other places: https://tech.marksblogg.com/benchmarks.html https://tech.marksblogg.com/benchmarks.html
- linspace 5y agoI'm surprised nevertheless that this is enforceable. Is there a precedent of someone being sued and losing?
- eatonphil 5y agoIf you're reliant on a vendor for a service why risk getting on their bad side?
- linspace 5y agoWhat makes you think everyone doing benchmarks is reliant? it could be research, it could be curiosity, you actually could be competing... Edit: and nevertheless is quite depressing that as a paying customer you should be afraid
- 1vuio0pswjnm7 5y agoNo.
- mlochbaum 5y agoI wrote this article to link in response to evidence-free claims about K, and have in fact used it in this way twice in the two months since I wrote it. Didn't expect HN in general to take an interest. So I need to stress again that K is a really cool language and I like it a lot! If the article can provoke clear benchmarks against current Dyalog or J then that would be an ideal outcome, even if it proves me wrong. I already see some benchmarks linked here and will try to analyze these.
- coldtea 5y ago>So the author is incredulous but provides no proof or stats, just a long rant. Isn't the burden on proof on the ones making the claim for the speed of their product? Incredulity should be the default scientific stance here...
- apricot 5y ago"This car is the fastest one on the planet, but we legally forbid you from talking about how fast it can go." Sure.
- 1vuio0pswjnm7 5y agoNot quite. More like "but we contractually forbid you from publishing the results of any speed tests without contacting us first." It's possible, even probable, they look at your results and then give you permission to publish. As long as they do not refuse permission or delay giving you an answer indefinitely, asking you to contact them first before publishing is arguably a reasonable requirement.
- ajuc 5y ago> It's possible, even probable, they look at your results and then give you permission to publish. Which makes observable results inherently biased and any comparison useless.
- 1vuio0pswjnm7 5y agoHow so. I dont understand.
- ajuc 5y agoBecause what constitutes "bad" benchmark is subjective and db providers are incentivized to stop publishing benchmarks that put them in bad light. And even if all they do is filter out actually crappy benchmarks - other db providers (like postgres or mysql) don't do that - so the comparison still isn't fair. It's like you had 2 candidates for a job and phoned their previous bosses, but candidate A forced you to only phone guys he approves of.
- 1vuio0pswjnm7 5y agoUsually there would be some language that states they cannot refuse permission to publish unless they have reasonable grounds. Your hypothetical, i.e., a "bad" benchmark, would not qualify as reasonable grounds. Blocking the publication of benchmarks because they do not suit one's business interests is not going to be an enforceable provision in a license agreement. I challenge you to name a single case that went to trial over such a provision where a court upheld the provision's enforcement.
- IshKebab 5y agoYeah the fact that they forbid benchmarks pretty much tells you all you need to know. Anyway once you get to something that autovectorises array operations there's not really anywhere to go is there? I seriously doubt it is really any faster than Matlab, Fortran, Eigen, etc. etc.
- eatonphil 5y ago> Yeah the fact that they forbid benchmarks pretty much tells you all you need to know. I don't think this is the case. Oracle and MS SQL have the same clause and it's pretty hard to believe they aren't competitive with postgres and mysql given the resources behind the former. More so it's just annoying of them to have this restriction.
- nullc 5y agoHow is that hard to believe? Clearly Oracle itself doesn't believe it. :P
- athrowaway3z 5y agoHa. This is extremely easy to believe.
- cameronh90 5y agoI was on a team that migrated an app from SQL Server to Postgres a some years ago, and the performance dropped by around 30%. Of course, due to the benchmark rules, I'm not allowed to post about it... ;) That being said, in our application it came down to three main issues: - Postgres doesn't allow optimizer hints, and the optimizer generally produced worse plans than Microsoft's (e.g. CTE were an optimization fence in Postgres). - Postgres at the time did not support parallel queries. - Postgres connection handling is terrible. Pgbouncer helps but introduces its own problems. - Database bloat / vacuum issues Generally I found SQL Server would perform very well with no tuning and minimal maintenance. And generally, if it doesn't do the right thing, a query hint will fix it. Postgres required a lot more configuration and testing to obtain acceptable performance. We still switched to Postgres. It has lots of other benefits - and none of the licensing.