16 ms·
David Patterson Says It’s Time for New Computer Architectures and Languages
- Razengan 8y agoI fear, the more we advance, the more we are going to become permanently entrenched in The Way Things Are. To really shake things up at a fundamental level will probably require running into an alien species that does basic things differently than us. Even creative people with lots and lots of spare time and resources at their disposal will still be too "colored" by existing human knowledge and established practices to really try anything new (and without falling into the trap of needlessly reinventing the wheel.)
- Animats 8y agoIt's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks. Much of machine learning is a simple repetitive computation running at low precision. Special purpose hardware can do that very well. So what's the next useful thing in that area?
- adamnemecek 8y agoI think that the sitch is different these days. Ideally, you’d have an end-to-end solution (hw, os, compiler). These days you can build an architecture, fork llvm and add a new backend and get Linux running. At that point you can start selling.
- marcosdumay 8y agoWell, past experiences are not that useful right now. I don't think the Itanium or the Cell were great architectures, but it's clear that their most obvious failure mode (mainstream architectures improving faster than them) isn't a showstopper anymore. For some guess on the next big thing, I would imagine that stuff that avoid the need of memory coherence have nice odds.
- scott_s 8y agoThe most obvious failure for the Cell architecture was that it was horrendous to program. That was partly caused by the lack of memory coherence between the PPE (main core that ran the OS) and the SPEs (the faster cores meant for offloading computations to), but also caused by a lack of developer tools. The main lesson I would take away from Cell is that you want to design an architecture that allows gradual performance refinement. With the Cell, it was more of a step function: good performance came from using the SPEs well, but using the SPEs well was hard. There wasn't much of an in-between.
- marcosdumay 8y agoDeveloper tools come with time, and time is the one thing non-standard architectures completely lacked until recently. I still don't think the Cell had much future, but we will never be sure. (Also, the Cell had complete memory coherence.)
- scott_s 8y agoYes, I mispoke by saying the SPEs lacked memory coherence, since you used the same physical addresses to load memory into the SPEs local storage. What I meant is that the SPEs are divorced from the normal memory hierarchy. You needed to explicitly retrieve and send all memory from each SPEs, which was a significant burden on programmers.
- saltcured 8y agoDoes this mean it was a mistake or just ahead of its time? I think software-managed memory tiers have been a dream for advanced architectures for a very long time. The problem, perhaps is assuming software-managed means programmer-managed. In a single system there is precedent in virtual memory systems using software-managed page mappings rather than static page table data structures. In HPC there is of course the precedent of distributed parallel systems with message-passing rather than shared memory abstractions. And the separation of GPU memory from system memory is certainly common today. Similar things are happening in software-defined storage (i.e. RAM/SSD/disk/tape tiering). Computation libraries and frameworks help straddle the gap between application programmer needs and current language/runtime/architecture semantics. I think one problem of current markets is that people tend to want to evaluate hardware independently of software, or in terms of yesterday's software. There is also a strange history of wanting to discard the software/firmware offered by the hardware vendor (for being insecure/inept/whatever), but blindly accepting the complex hardware design that enables our naive/platform-independent software to run well. The whole Spectre debacle shows how that may have been wishful thinking that we can divorce the software and hardware designs...
- snaky 8y ago> Remember the Itanium Well, why not. > Q: I feel like deja vu - at Hot Chips, Intel introduced VLIW-concept Itanium that pushed complexity onto the compiler. I see traces of that here. What are you doing to avoid the Itanium traps? How will you avoid IP from Intel? > A: Itanium was in-order VLIW, hope people will build compiler to get perf. We came from opposite direction - we use dynamic scheduling. We are not VLIW, every node defines sub-graphs and dependent instructions. We designed the compiler first. We build hardware around the compiler, Intel approach the opposite. https://www.anandtech.com/show/13255/hot-chips-2018-tachyum-prodigy-cpu-live-blog https://www.anandtech.com/show/13255/hot-chips-2018-tachyum-...
- saidajigumi 8y agoThat captures a lot of my take on Itanium at the time: despite being "VLIW" it took on what at the time was a vast amount of CPU controller complexity without an obvious design win or clear high-performance compilation model. As an outsider, it felt like an extremum of Intel's flavor of design hubris. It's interesting to compare the Itanium's approach to a VLIW contemporary from TI: the C6x VLIW DSP ISA. The C6x ISA took on very little controller complexity. Instead, it exposed a very clear operational and optimization model for the VLIW instruction blocks. An optimizing compiler did a decent first-step job at generating code. That was complemented by clear documentation of the optimization workflow, from compiler-stage iteration down to good guidance on the nuts and bolts of hand-optimizing. By comparison, IA64 was just ... a big head-scratching moment all around. I never got to work with C6x code professionally, but dang it looked like a blast to code and optimize for.
- gsnedders 8y agoWhat was the controller complexity of the Itanium you're referring to?
- stcredzero 8y agoIt's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks. Not all CPU architectures are equally successful. Likewise, not all highly parallel architectures will be equally successful. If progress in hardware is going to be characterized by more parallelization, things like Erlang Actors are going to become more important.
- jimmy1 8y agoI vaguely remember my computer arch course in college describing how everything we have today essentially suffers from the von neumann bottleneck, that even if we implemented a lambda calculus machine, it would still suffer from this bottleneck. Has this been revisited in academia lately? I am a bit out of touch, but this always interested me.
- stcredzero 8y agoYou can interpret my comment thus: I propose having lots and lots of Von Neumann bottlenecks!
- slater 8y agoYou're very clever, young man, very clever. But it's bottlenecks all the way down!
- mamcx 8y agoJust "clean" what we already have is more than enough, IMHO. Things can already target X64/Arm but with a minimal kernel and a foundation without C at the base. --- Even with a complete new CPU/Arch (that weirdly, I dream about that!) you can solve compatibility with VM/Containers. And provide a better user-land.
- planteen 8y agoEven SIMD, a feature of processors for over 20 years, is not handled well by compilers for automatic optimization. It seems like all code taking advantage of SIMD either drops into ASM, or uses a library from ARM/Intel to give descriptive C functions names for the underlying ASM.
- jagger27 8y agoI wonder why high level languages themselves don't add syntax support for wide-math instructions? I understand why loop unpacking is a little tricky, so why not let the programmer take care of it?
- deleted 8y ago[deleted]
- pjmlp 8y ago.NET and Julia do offer such support. Java is in the process of doing it as part of Arrays 2.0 roadmap.
- zvrba 8y agoThere is ispc: https://ispc.github.io/ https://ispc.github.io/
- pjmlp 8y agoItanium only failed because Intel had agreements in place that allowed AMD to build x86 compatible CPUs, and they were clever to come up with AMD64. If it wasn't for that AMD stunt we would all be using Itaniums by now.
- distances 8y agoIn other words, Itanium only failed because there was competition.
- pjmlp 8y agoNo, Itanium failed because Intel made the error to allow other companies to compete on the x86 space with their patents cross licensing deal. Competition from other architectures was always there.
- hawski 8y agoIn other words, Itanium only failed because there was competition in the x86 space.
- pjmlp 8y agoOf course having the option to run existing binaries made it a no brainer for companies to avoid spending money. Hence Intel's mistake.
- nickik 8y agoDynamic x86 imterpretation could have made competition as well. There were companies that did that and had Intel moved to EPIC others might have done what AMD did.
- deleted 8y ago[deleted]
- dschuetz 8y agoThe Cell was way ahead of its time. Its security architecture capabilities was almost perfect, but they've screwed up the implementation and they failed to provide good programming tools. And, it was very very expensive.
- coldtea 8y ago>It's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. I think the problem with those is that they failed to build anything compelling -- not that people didn't come.
- quickben 8y agoWhat's happening with that Mill Arch? Is it moving forward outside of them publishing papers? It seemed promising.
- utopcell 8y agoCurious as well. That'd be a refreshing change of pace, a completely new architecture.
- zaarn 8y agoI believe end of next month there will be another Talk where the Mill designers will talk about Spectre and Meltdown attacks in relation to Mill (though the spoiler would be that the talk mentions the point "and why Mill is immune" so I guess it'll be a very fun talk). I do hope a lot that Mill will succeed, it's an incredibly promising architecture.
- daemin 8y agoI do hope they just put paper to silicon and implement the damn thing, even if they just distribute simple RaspberryPi type boards with a slow version on it.
- ericand 8y agoI'm a VC and I've seen lots of pitches lately for various tech related to new architectures, be it GPU, FPGA or RISC. I'd add cryptocurrency to list workloads with insatiable computing demands.
- crb002 8y agoMassive parallel prefix and reduction operations in RAM. Start with just the MPI standard reduction ops. 1000x faster than Von Neuman bottleneck.
- crb002 8y agoThen an ASIC for sparse matrix multiply. Valiant's reduction of CFG parsing. Massive speedup to parsing workflows, and boon to security by getting rid of hand rolled parsers.
- snaky 8y ago> Massive parallel prefix and reduction operations in RAM Sounds similar to Micron Automata Processor.
- crb002 8y agoMicron's processor in memory doesn't have near the same bandwidth as a processor in memory array geared to doing parallel prefix.
- so_tired 8y agoDo u see more pitches for source-code level solutions ? i.e. analyzing and modifying the huge legacy of existing code base ? This is a very difficult area of course (both financially and technologically). But the pay off can be very large. Imagine a vmware-like transformation at the source code level.
- austincummings 8y ago> Imagine a vmware-like transformation at the source code level. I'm not sure what you mean by this, but I'm interested. Could you explain?
- deleted 8y ago[deleted]
- philipkglass 8y agoAre there languages that have first-class support for representing/optimizing memory hierarchy characteristics? Optimizing C compilers, for example, may have extensions to force specific alignments: https://software.intel.com/en-us/articles/coding-for-performance-data-alignment-and-structures https://software.intel.com/en-us/articles/coding-for-perform... But I'm not aware of languages where e.g. declaring alignments is part of the base language. Awareness of L1, L2, L3 cache characteristics, plus NUMA nodes, is increasingly important to writing high performance code. Every language I've ever used exposes nothing of that hierarchy; it's all just "memory". People who write performance-critical code are already aware of these issues and can measure/hint/optimize to work with the memory hierarchy in existing languages. But I feel like this could be better if we had languages that modeled these issues up front instead of only exposing the abstracted view. Maybe such languages already exist and I just haven't encountered them yet.
- niftich 8y agoRepresenting memory hierarchy in a somewhat-HLL was the goal of the programming language Sequoia, presented in the paper 'Sequoia: Programming the Memory Hierarchy' [1][2] from Stanford University. Abstract: "We present Sequoia, a programming language designed to facilitate the development of memory hierarchy aware parallel programs that remain portable across modern machines featuring different memory hierarchy configurations. Sequoia abstractly exposes hierarchical memory in the programming model and provides language mechanisms to describe communication vertically through the machine and to localize computation to particular memory locations within it. We have implemented a complete programming system, including a compiler and runtime systems for Cell processor-based blade systems and distributed memory clusters, and demonstrate efficient performance running Sequoia programs on both of these platforms." [1] DOI: 10.1109/SC.2006.55 [2] http://graphics.stanford.edu/papers/sequoia/sequoia_sc06.pdf http://graphics.stanford.edu/papers/sequoia/sequoia_sc06.pdf
- eslaught 8y ago(Disclaimer: I work on Legion) Sequoia has been largely superseded by its spiritual successor Legion [1], another programming system by Alex Aiken. Legion doesn't focus so much on low-level memory hierarchies, but it does a very, very good job of scaling to very large machines, and taking advantage of heterogeneous processors such as GPUs. It is also incomparably better at working with dynamic programs, whereas Sequoia needed to know basically everything about the program up front (including what machine it will run on). If you're looking for a modern version of Sequoia that actually runs on modern hardware, I would strongly recommend looking at Legion. (Or the language component of Legion, Regent [2].) [1]: http://legion.stanford.edu/ http://legion.stanford.edu/ [2]: http://regent-lang.org/ http://regent-lang.org/
- stellalo 8y agoWell, there are some interesting new languages out there already: https://julialang.org/ https://julialang.org/ In fact Moore’s law must have been the reason why Python is what it is today: “Who cares if it’s slow, tomorrow’s computers will be faster!”
- elihu 8y agoI think the current attitude could be summed up better as "Who care's if it's inefficient, yesterday's computers are fast enough!" I think that's true of most software, but not all -- there will always be people who care about the best performance, but we shouldn't expect them to be in the majority.
- fsloth 8y ago"there will always be people who care about the best performance, but we shouldn't expect them to be in the majority." Given how machine learning is going to kick in in lots of fields quite soon there will be a lot of real world applications that are not ambivalent about performance (if not else then from energy use point of view). We're going to get Clippy the Clipper 2.0 who can actually do something usefull, and how long he spends figuring out his suggestions and how good they are are likely to be dependent on the available CPU resources and how fast Clippy can think.
- pyrale 8y agoIn the case of Python, who cares if it's slow, it's a thin wrapper around a C lib!
- dnautics 8y agoI would say erlang is an interesting new language, except that it's not new. The idea of a functional language that is just very careful about its impurities and has a clever way of managing state without spinlocks or mutexes is a great thing. Beam languages have almost no footguns and programming them feels like programming with inflated gutterball balloons. A chip architecture that was highly optimized for the beam would be great.
- 8y ago
- snaky 8y ago> And Intel, he said “is trying to make all the bets,” marketing traditional CPUs for machine learning, purchasing Altera (the company that provides FPGAs to Microsoft), and buying Nervana with it specialized neural network processor (similar in approach to Google’s TPU). Actually Intel has even wider view of the things and interest to non-(yet?)-mainstream tech, like e.g. async circuit design. They has bought Achronix, Fulcrum, Timeless Design Automation. Fulcrum in 2002 made things like "a fully asynchronous crossbar switch chip that achieves 260-Gbit/second cross-section bandwidth in a standard 180-nanometer CMOS process" - https://www.eetimes.com/document.asp?doc_id=1145012 https://www.eetimes.com/document.asp?doc_id=1145012
- watmough 8y agoThe lack of obsolescence is terrible news for Dell and HP. Ran into a client this afternoon still running his seismic workflows on an old HP EliteBook 8570w with a couple big monitors, big SSD and 16 Gigs of memory. That family was launched mid-2012!
- agumonkey 8y agoProbably old news, but forth cpu arrays are so radically fun .. I wonder why they're not interesting. Every forth core can pass forth to its neighbors, forth is very expressive yet tiny since it's a stack language. Anyway my 2cents
- snaky 8y ago> the so-called Adaptive Compute Acceleration Platform (ACAP) will deliver 20x and 4x performance increases on deep learning and 5G radio processing, respectively, Xilinx claimed. The first chip, called Everest, will tape out this year in a 7nm process. > The centerpiece of the new architecture is word-based array of tiles made up of VLIW vector processors, each with local memory and interconnect. > tiles communicate with each other to create data paths that best suit their application. https://www.eetimes.com/document.asp?doc_id=1333632 https://www.eetimes.com/document.asp?doc_id=1333632 So GA144 is going to mainstream finally.
- agumonkey 8y agogreat, but do they have actual lineage to GA144 chips ? or just another matrix of cores ? did xilinx buy GA ? ps: funny reading 'post-moore' in the linked article :)
- Avshalom 8y agoI think it's more obscure than uninteresting. Fundamentally though it's a transputer-on-a-chip so there is an existing base of research on writing/optimizing that sort of thing.
- tabtab 8y agoI have to say that most bottlenecks are from lazy designs, not wimpy hardware. As a thought experiment, suppose parallel processing scaled easily without worrying about Amdahl's Law and similar bottlenecks. So then we put 128 cores into our computers. Software companies would eat up every single core eventually if there's no penalty for inefficient coding. They don't pay our electric bill, we do. It's kind of like freeways: the more you build, the more the traffic increases to fill them up again. It is "nice" to be able to slap components together like magic legos and/or a Dagwood Sandwich rather than smartly tune and coordinate everything, but doing such is often computationally inefficient. The speed of light will probably put a hard upper limit on this practice that no clever language or architecture can work around to a notable degree.
- chubot 8y agoYes, a pithier way of saying this is: Software is a gas; it expands to fill its container - Nathan Myhrvold Bad performance isn't really a software design problem; it's an economics problem (ditto for security). You could speed up the browser by 2x now, or reduce it's memory usage by 2x, and websites would be exactly as slow after a few months. I'm not trying to be negative here, just stating an unfortunate systems dynamic.
- tabtab 8y agoI generally agree: it's about weighing trade-offs, and calling software companies "lazy" is perhaps an oversimplification. However, I do believe the speed of light and size of atoms puts an upper limit on the degree you can keep throwing hardware at the problem. We may continue to find tricks such as quantum computing, but I suspect it will get ever tougher. There's a theoretical limit to how small you squeeze data and to how fast it can transfer.
- the8472 8y ago> We may continue to find tricks such as quantum computing Quantum computing does not magically speed up general purpose computing tasks. We only know a handful of algorithms where quantum computers provide algorithmic complexity reductions and even that that does not mean they necessarily calculate fast in terms of IPC, they just calculate previously intractable problems in reasonable timescales. TL;DR: you won't pop quantum GPUs/CPUs into your computer and suddenly get 50 times the framerates in DOOM.
- gone35 8y agoAs part of one of those "45 hardware startups" (in stealth mode) trying to tackle the problem, I think nevertheless there is plenty of fat to trim just on the software side still... I mean, just look at that very page: 200 MB [edit: in memory consumption] for a single static article! Information-theorically at least, I bet that is another 10-100x speedup opportunity right there (and next multi-billion dollar industry, perhaps)...
- gammatrigono 8y agoYou mean 200 KB.
- eveningcoffee 8y agoI think that we well see more and more specialized silicon like has been used for mining and training. This hardware is orders of magnitude faster than conventional CPU. Another point is that processors are already now incredibly fast and the performance is caped by the peripherals. This is were the huge potential is hiding for some time.
- astrodust 8y agoBetter FPGA could bridge the gap between ASIC and software. FPGA chips are still almost exclusively niche products, but if they became mainstream like GPUs the economy of scale kicks in and it's a whole different game.
- ArtWomb 8y agoIn gamedev, Nvidia's Turing enabled real time rendering at 4k 60fps could be "future-proof" well into the next decade. I'm just not sure end users are clamoring for more photo-realism. Instead, future of computing turns toward optimizing for the experience. With 3D printed form factors and smart haptics. European startup Canatu provides a glimpse of what carbon nano tube (CNT) sheets make possible: https://canatu.com/ https://canatu.com/
- the8472 8y agoNow it's time for display technology to catch up. Once things like light fields roll around those 4k will seem laughable. > I'm just not sure end users are clamoring for more photo-realism. People are definitely making fun of characters looking waxy, fire effects looking cheap, shadows being pixelated etc. Plus this is just the first generation of realtime raytracing. It's not like the entire scenes will be raytraced, just some parts will be raytracing-assisted, e.g. lighting. The next generations will most likely enable a higher percentage of the scenes being based on tracing.
- ghthor 8y agoThat music with their video[1] is like a drug for positive energy. [1] https://www.youtube.com/watch?v=Tq3w0XvzMKw https://www.youtube.com/watch?v=Tq3w0XvzMKw
- mahgnous 8y agoTell him to get on it then
- luma 8y agoOnly tangentially related to the article but... I'm left to wonder about the photograph selected. His left hand appears to be on a stack of 20-year old HP 9000s (maybe rp3440, pre-grey box?), and standing across from a rack full of ancient 4U systems mounted into 2-post relay racks. The white blanking panels are early 2000's-era HP kit, and in the background are racks full of tower systems stacked vertically. Is this a recent photo? Do they have a computer history museum @ UCB or is that datacenter actually running production workloads on that kit?
- Varcht 8y agoPer the exif data embeded in the image "Date Time Digitized: May 1, 2006, 9:10:59 PM, Creator: Peg Skorpinski photo".
- luma 8y agoThat would certainly explain it! After posting my comment it occurred to me that having Dr Patterson pose next to a PA-RISC system was probably intentional, given his history.
- mjfl 8y agoIt would be nice if low volume foundry costs went down an order of magnitude. I'm in a bio lab right now, but I did CS in college and really enjoyed VLSI. If I want to turn a design into a prototype in my bio lab, it's going to cost hundreds of dollars, but if I want to turn a chip I designed into reality, it's going to cost thousands to tens of thousands of dollars even with specialized low volume foundries like MOSIS. I feel like it should be cheaper.
- tonysdg 8y agoGenerally speaking, that's where FPGAs excel. You get a good chunk of the speedup that comes with designing your own circuit at a lower cost than actual ASICs (at least for protoyping and low volume batches).
- nuguy 8y agoThis is a question of computer science but more a question of economics. Computer systems will drop their own compassing solutions for more specific solutions as time goes forward.
- StillBored 8y agoHow about instead of inventing new languages we just pick a couple and tell people that if care at all about performance not to use the rest. I would start by throwing away all the interpreted/GC'ed languages because even after decades of massive effort they are frequently lucky if they even manage to keep up with unoptimized C.. But that really isn't the problem, the problem is that they cannot be debugged for performance without directly hacking the GC/interpreter. At least in C when you discover your data structures are being trashed by insufficient cache associativity (for example), you can actually fix the code. Put another way, up front people need to know that if they write in python/Java/etc to save themselves a bit of engineering effort and its anything more than throwaway low volume code, the effort to optimize or rewrite it will dwarf any savings they might have gotten by choosing python.
- mtinkerhess 8y agoFor a lot of problems performance really isn't the problem. In many, many cases the engineering cost to optimize C++ to get a benefit over Java / C# will be way more expensive than just paying for more servers. Even in game dev where milliseconds matter C# is very common. Personally I agree about dynamically typed languages not being great for large projects but that's more about maintainability than perf.
- gerritsg 8y agoSo would Rust be one of the languages you'd advocate including in the chosen few? Does Rust's approach to memory management put it more in the C group in terms of debugging for performance?
- StillBored 8y agoI wasn't really trying to pick/choose a language. Although I did pick on python a bit, because I just spent a few days wading through a gigantic "python shell script" that would have literally been 1/100th as many lines written in bash because every third line was calling some shell utility or doing the equivalent of 1 line sed calls in 4-8 line python functions. The point is that we literally have dozens and dozens of languages active on any given machine with absolutely massive frameworks duplicating functionality. Here is a few hundred meg ruby runtime, with a few hundred meg node.js runtime, with a few hundred meg python2 runtime (and another for python3), with a few hundred meg perl runtime, etc, etc, etc. Its insane because in most cases you can't even statically bind the whole thing into a single .jar like file. Not only is the cognitive load incredible, but the code quality in most of them is horrendous. Then there is the issue, that most of these language runtimes have for most purposes have zero documentation outside of the actual code. Then when someone says "what language should I choose" they are presented with a whole bunch of square pegs to pound into their round hole. If your going to work with a square peg, it should probably be one well understood and polished enough to actually have documentation for more than the 100 most common functions being called. What I'm trying to say is that we don't need yet another over-hyped language, and we should probably try to kill a few more of them off. Object Pascal is a better language than probably 1/2 the ones on common use, but its pretty much dead. Inventing another language that is _worse_ than object pascal is just stupid.
- mark_l_watson 8y agoA little off topic, but I greatly admire generalists: Dr. Patterson (along with Dr. Fox) taught a good Coursera course on agile web development with Ruby and Rails. Quite a departure for an originator of RISC architectures! I think Dr. Patterson makes a great point on there being plenty of headroom in the increase in software efficiency.
- api 8y agoHere's something that seems shockingly under-explored to me: languages that incorporate relational data structures natively. We have SQL of course, but SQL is not a general purpose language and is (intentionally) often not Turing-complete. I'm imagining something like Go, Ruby, JavaScript, or Rust with native tables, queries, and other SQL-ish relational data structures. The long term goal would be to kill the age-old impedance mismatch problem and eliminate CRUD code. Toward this long term end the language runtime or standard library could actually contain a full database engine with replication/clustering support.
- CMCDragonkai 8y agoThere's a bunch of relational algebra systems in Haskell, and one in Ruby that gives better first class control over these structures. However the fact that SQL engines all use SQL as text to communicate results in marshalling cost. However there are also languages that have embedded database such as K+.
- Karrot_Kream 8y agoTake a look at Datalog. It's a subset of Prolog optimized for use on top of structured data.
- chubot 8y agoData frames in R give you the relational model with a Turing complete language. I think of it as a better SQL, at least for analytics (as opposed to transaction processing). Besides its data structures, R is surprisingly similar to JavaScript -- it's has Scheme-like semantics but C-like syntax. For certain problems, it's a pleasure to program in. You don't have to go through "objects" to get to your tables; you just have tables! The examples here might give you a flavor: https://cran.r-project.org/web/packages/dplyr/vignettes/dplyr.html https://cran.r-project.org/web/packages/dplyr/vignettes/dply... R itself has a lot of flaws, but the tidyverse [1] is collection of packages that provide a more consistent interface. FWIW I agree that more mainstream languages should have tables natively. It is an odd omission. Somehow they got coupled to storage, but they're also useful for computation. [1] https://www.tidyverse.org/ https://www.tidyverse.org/
- amckinlay 8y agoI want a language where you can express time and constraints on time within the language and the type system. I want to be able to guarantee program correctness, pre-calculate fine-grained energy usage, optimize for energy/power-saving mode usage, interface with asynchronous signals, and whatnot -- all with respect to program execution at the ISA-level within some hard real-time constraints. Compilers make optimizations using detailed instruction timing information, but as far as I know, these details do not bubble up to the surface in any programming language. It may be wise to keep these details under the surface at the compiler level, but for 8-bit architectures, it would be awesome to have a language where you have explicit control of time.
- Waterluvian 8y agoAre you suggesting a language where you could hypothetically look at your IDE and it'll tell you via static analysis, "this code block takes x cpu units. On a Y core this is 0.95ns". Never thought about that. That would be so cool.
- baddox 8y agoThere are lots of attributes that are provably impossible to statically determine about a computer program. Whether the program halts is the obvious example, but Rice's theorem generalizes this to effectively any attribute of a computer program you would be likely to care about. https://en.wikipedia.org/wiki/Rice%27s_theorem https://en.wikipedia.org/wiki/Rice%27s_theorem
- dwohnitmok 8y agoRice's Theorem generalizes this to any implementation-independent attribute (i.e. properties of the idealized function on natural natural numbers a program represents). However, certain implementation-specific details such as run time (i.e. performance) are fair game. In particular it is quite simple to at least calculate lower bounds on run time. Simply choose a lower bound and then run the program and see if it halts before then.
- person_of_color 8y agoAny suggestions for MS in CompArch?
- ilaksh 8y agoDoes David Patterson know that Cython already exists?
- bcheung 8y agoIt seems like hardware is moving away from RISC and even CISC architectures in favor of lots of different types of silicon for specialized applications (GPU, machine learning, vision, DSP, hashing, encryption, SIMD). It makes sense from a hardware perspective but it seems software development is not keeping pace. General purpose languages have trouble targeting this kind of hardware.
- calebh 8y agoThis sounds good, as a person who specializes in domain specific languages. Maybe we'll see higher demand and corresponding salary increases...
- nickpeterson 8y agoIt's way easier to push the limits on simple things. We need languages with fewer features and clear design, running on hardware with less exotic features.
- AnIdiotOnTheNet 8y agoI agree in general, and while I think that is totally completely possible in languages and operating environments, I'm not so sure it is in hardware, at least not without sacrificing a lot of performance. People respected by people I trust, who have a lot more domain knowledge than I do, don't seem to think it is.
- justin66 8y agoOn the other hand, we can afford to sacrifice a lot of performance.
- deleted 8y ago[deleted]
- breckuh 8y ago> We are now a factor of 15 behind where we should be if Moore’s Law were still operative. We are in the post-Moore’s Law era. Is this true? According to https://ourworldindata.org/technological-progress https://ourworldindata.org/technological-progress it looks like we are maybe a factor of 2 or 4 off.
- Lerc 8y agoI have had this fun idea for an odd architecture floating in my head for ages. Lots of small processors with local work ram, cached ram, shared ram. Each processor has a numerical id, communicates with each processor with one bit different in the id by optical link plus an additional link with to the processor complimenting all bits. They send messages and fill their caches from the pool of shared ram. Place even parity processors on one board and odd parity processors on another board facing towards the first. Thus all processors have line of sight on their communication partners. Messages go back and forth with processors relaying messages by fixing one bit in the message address. All neighbours with a one bit better address are candidates for relaying. If any are busy, they have options to use another path. This means the most number of hops a 2^19 core system would have to do is 9. If more than half of the bits are wrong then jump to the compliment. So the example with half a million processors, each talking to 20 neigbours by optical links. Messages can be sent anywhere with a latency of up to HopTime*9. Filling a cache from anywhere in the shared memory pool would have a latency of twice that. If speed of light is the latency factor a 150ms latency would get you 9 hops across a 5 meter gap. Smaller is of course always going the make it better. This is the sort of thing that would also probably need a new language. I'm not entirely sure you could come up with an appropriate language without at least a simulation of the architecture.
- tralarpa 8y ago> This is the sort of thing that would also probably need a new language. I have a good idea for a name of that new language: occam ;)
- smilekzs 8y agoHop time is not constant. There will be longer and shorter optical links, and if you want the system to be synchronous, everyone would have to wait on the longer links.
- Lerc 8y agoTrue, angles would add a little extra depending on how large the proccessor panels were and their separation distance. I had envisioned every link comminicating independantly. what would be the advantage of communicating syncronously?
- tokyodude 8y agoNot a new language, maybe not even a new idea but Unity's new Entity Component System.is designed around making cache friendly parallelizable coding easy https://github.com/Unity-Technologies/EntityComponentSystemSamples/blob/master/Documentation/index.md https://github.com/Unity-Technologies/EntityComponentSystemS...
- graycat 8y agoIMHO, stochastic optimal (SOC) control is a framework that is much more powerful than anything like current artificial intelligence, machine learning, deep learning, etc. and, in particular, a much better version of something like actual learning. The difference is like going from gnats to elephants, like a bicycle to the star ship Enterprise. A good application of SOC looks not just "intelligent" but wise, prudent, brilliant, and prescient. There have been claims that necessarily SOC is the most powerful version of learning there can be -- a bit much but ... maybe. For computing architectures, the computing that SOC can soak up makes the computing needs of current deep learning look like counting on fingers in kindergarten. The subject is no joke and goes back to E. Dynkin (student of Kolmogorov and Gel'fand), D. Bertsekas (used neural nets for approximations for some of the huge tables of data), R. Rockafellar (e.g., scenario aggregation), and, sure, R. Bellman. Some of the pure math is tricky, e.g., measurable selection. I did my Ph.D. dissertation in SOC and there actually made the computing reasonable, e.g., wrote and ran the software -- ran in a few minutes, with my progress in algorithms down from an estimate of 64 years. For larger problems, can be back to taking over all of Amazon for 64 years. More algorithmic progress is possible, and, then, some specialized hardware should also help, sure, by factors of 10; right, we want lots of factors of 10. If want to think ambitious computing, past Moore's law, past anything like current AI, with special purpose hardware, go for SOC. SOC Applications 101. For your company, do financial planning for the next 50 years, 60 months. So, set up a spreadsheet with one column for each month, one column for the current state of the company and then 60 more columns. Goal is, say, to maximize the expected value of something, maybe the value of the company, in the last column, right, with control over the probability of going broke, if a bank always be able to meet reserve requirements and pass stress tests; if a property-casuality insurance company, stay in business after hurricane Florence, etc. For each variable of interest, have a row. In the cells, put in the usual expressions in terms of the values of cells in earlier columns. Also have some cells with random numbers -- the stochastic part. Also have some cells empty for the business decisions, the control part. This is the 101 version; the 201 version has more detail! Can't get the solution with just the usual spreadsheet recalc because the best solution is optimal and varies through the 60 months as more information is gathered -- and that is the core of the need for more in algorithms and taking over all of Amazon for the computing. Really, the work comes too close to looking at all possible business state scenarios over the 60 months yet still is astronomically faster than direct or naive ways to do this. Or, a little like football, don't call the play on 2nd down until see the results of the play on 1st down, BUT the play called on 1st down was optimal considering the options for the later downs and plays. The optimality achieved is strict, the best possible, not merely heuristic: No means of making the decisions using only information available when the decisions are made can do better (proved in my dissertation). Intel, AMD, Qualcomm, DARPA, NSF, Microsoft, etc., think SOC!!! You just heard it here -- I might not bother to tell you again.
- bogomipz 8y agoThe article states: >"Google has its Tensor Processing Unit (TPU), with one core per chip and software-controlled memory instead of caches" Is have heard of software-controlled caches before but I am imagining this is not the same thing? Could someone say? Might anyone have any decent resources on software-controlled memory architectures?
- zvrba 8y ago> "When performance doubled every 18 months, people would throw out their desktop computers that were working fine because a friend’s new computer was so much faster." So we have finally reached some sustainability. Good.
- mehrdadn 8y agoPeople still have to throw their computers out because other hardware still advances, software also advances, and companies decide not to support older hardware in their new software. The wastefulness really doesn't end.
- earenndil 8y agoThey don't have to throw them out as often, though.
- mehrdadn 8y agoionno... did people actually throw out their computers more often before? I'm honestly skeptical that people threw out their computers every 18 months just because their friend had a faster computer as was claimed here. I feel like people are very quick to buy new computers now. And not just that, but now we're having this problem with phones too.
- reacweb 8y agoIMHO, machine learning is a tiny aspect of the issue. The main issue is that, strangely, C is still the desert island language (first hit: http://www-cs-students.stanford.edu/~blynn/c/intro.html http://www-cs-students.stanford.edu/~blynn/c/intro.html). The is a huge intimacy between the C language and CPU, they have evolved together during the many years of Moore law. I think there should exist a far better language to access hardware. linux kernel should migrate to this language. And the intimacy between OS and hardware should drive CPU (and GPU) toward better directions. Maybe I am dreaming.
- dschuetz 8y agoWhile I agree with Patterson, I'd be careful with the expectations though. Computational engineering is highest tech. Whatever purpose, whatever specialization, processors are the central domain in modern technology and engineering. It would still take decades to find a feasible approach to make new architectures work (in silicon). But I agree that it is an exciting time, I want to see new clever architectures emerging, competing and be used for different purposes.
- davidhyde 8y agoThe author makes it sound as if there have been no material changes to programming languages recently but unless you've been hiding under a rock you will know that this is not true. He compares Python to C. Come on, please. It would have been more relevant to speak about the LLVM project and how that enables architects to build more expressive languages which can still make use of 30 years of optimization wisdom (most of which was built when CPU's were much slower) What about mentioning languages like Rust which are designed for high concurrency and zero overhead which let developers write software with the safety of python but the performance of C. What about mentioning that some old languages, like c++, have evolved though new language features AND guidelines into much more effective languages to guide us into the future of concurrent programming (the easiest way to enjoy the speedups of yesteryear)
- deleted 8y ago[deleted]
- j45 8y agoMost languages in use today have had to extend from their core to the web, or mobile through a framework. On the other hand, web or mobile first frameworks/platforms, are often high level, and have been a little ornery when digging into the weeds. Our paradigm has shifted from desktop-first, to web-first, to mobile-first a while ago, but our frameworks are still often anchored from a desktop-first world perspective. If there's examples that do, or do not highlight this, would love to see and discuss :)