mirker

24 ms·

211.
by mirker 5y ago
Paper is stretching by picking favorable benchmarks. It’s not ResNet on ImageNet, folks. It’s AlexNet on Fashion-MNIST. AlexNet (2012) is out of date. The performance each parameter brings is less than newer models. Newer models, apart from
212.
by mirker 5y ago
They use AlexNet, which is way out of date at this point and I believe understood to be parameter inefficient. Most of those model parameters are likely useless, and at least less useful than those of other newer models. The model they chos
213.
by mirker 5y ago
I don’t know about nodejs, but serving a twitter-like feed with an in-memory database of everyone’s tweets is one use-case. It’s a database query workload and would benefit from caching raw data or views of the twitter feed and has a latenc
214.
by mirker 5y ago
It’s a Catch 22. New algorithms stink because conventional hardware stinks at running them and vice versa. Before AlexNet and GPU-optimized kernels, deep learning took too long to train large models, so large models were not used, and other
215.
by mirker 5y ago
Most of the effort is in fine tuning the existing methods, which is important, but also leads to reinforcing the status quo in terms of algorithms. In other words, transistor scaling gets a sizable fraction of optimization, and the viable a
216.
by mirker 5y ago
I don’t think it’s irrelevant to look at changing timescale. If the law broke down to be 3 years, there isn’t any reason it won’t be 4, 5, or some other N years in the future.
217.
by mirker 5y ago
On the same podcast you can find David Patterson (known for writing some widely used computer architecture books), who disputes this claim. https://www.youtube.com/watch?v=naed4C4hfAg At 1:20:00
218.
by mirker 5y ago
I’d just like to add that “Dynamic Programming” is one example of word fluff. “Dynamic” is somewhat of an ambiguous marketing buzzword, unfortunately.
219.
by mirker 5y ago
The question is how much those features were worth in terms of cost (is a feature bloat?). If you don’t have a cost constraint, then yes, you have feature diversity but the cost for any feature can be arbitrarily high.
220.
by mirker 5y ago
Windows 95 needed 4MB memory. Windows 7 needed 1GB. Try running Windows 10 with less than 4GB. They all run processes, do they not?
221.
by mirker 5y ago
> Which software exactly is functionally the same yet consumes more resources? Can you give some specific examples? Operating systems, particularly Windows. Given fixed hardware, you cannot expect an operating system to continue working
222.
by mirker 5y ago
Same is true in academic research (though old-timers catch it often). The common pattern is: * approach “A” was invented in 1970 or so and didn’t work * “B” extends “A” in multiple ways and now works * noobs assume “B” invented “A” and trea
223.
by mirker 5y ago
Depends. You can make the same argument about an OS not giving bare-metal performance. For example, disk bandwidth may be different from filesystem performance. You may have some daemon doing e.g., disk-defragmentation in the background. If
224.
by mirker 5y ago
It’s risk vs. efficiency. Duplicating work lowers risk at the cost of efficiency. Without redundancy, the market is unstable. Look at the pipeline shortages of the past year. A chip shortage screws up many industries because all eggs are in
225.
by mirker 5y ago
You’d have to also control external “laundering” of money through the insiders. The hedge funds could pay the insiders for trades or information.
226.
by mirker 5y ago
The problem is low-end laptops ship with Windows barely working, then Windows inevitably bloats itself and more-or-less bricks the device. I agree that many won’t like Linux due to it being different, but it’s at least functional in this ca
227.
by mirker 5y ago
A family member had a slow windows install. I checked the system utilization, suspecting paging. Guess what: 3/4 GB of the laptop were used by windows, leaving 1GB for all other applications. Apparently, deprecation via automatic updat
228.
by mirker 5y ago
Perfection is the enemy of the good.
229.
by mirker 5y ago
What’s the difference between a senior faculty and a politician? In both cases, you’re more of an icon than and individual contributor. The difference gets even smaller if faculty become dean.
230.
by mirker 5y ago
Thanks, I see what you mean at the interface level. In terms of performance analogy though, hard drives do not get random access to blocks, but RAM does. The practical block size of hard drives is sequential reads of 100kiB+ due to seeks.
231.
by mirker 5y ago
I’m confused. What’s the difference between a cache line and a row in RAM? They’re both multiples of bytes. You have data sharing per chunk in either case. The distinction seems to be how big the chunk is not uniformity of access time (is a
232.
by mirker 5y ago
For what it’s worth, gameplay AI in real time strategy games beats humans by performing more actions per minute than humanly possible. You can find Starcraft 2 AI that perfectly controls every unit while a human can only dedicate time to co
233.
by mirker 5y ago
Is there a non-vacuous example of AI being used in this context? Is it matrix multiplication, linear regression, same old control theory, or deep learning?
234.
by mirker 5y ago
Sufficiently large boolean satisfiability also enough to reach general AI. I kid, but these sorts of papers are theoretical position papers and do not account practical considerations e.g., for how many suns of energy must be expended.
235.
by mirker 5y ago
Not sure if there is any case where this thought is useful. The only thing this says is the primitives are correlated and we don’t understand them. It’s similarly not useful to think about atoms “talking” when they exchange heat.
236.
by mirker 5y ago
Deep mind’s protein folding algorithm is probably worth a chunk of change. As far as I know, they’ve been holding onto the secret sauce rather than publishing it.
237.
by mirker 5y ago
For the chip planning, isn’t Moore’s law a transistor prediction (rather than layout)?
238.
by mirker 5y ago
Oof… trading food for shelter seems like a problem I wouldn’t want to think about
239.
by mirker 5y ago
Location matters too. I don’t think the problem is making a log cabin in the woods.
240.
by mirker 5y ago
Show interns this story: https://devblogs.microsoft.com/oldnewthing/20201110-00/?p=10...

More ›