bertr4nd

9 ms·

31.
by bertr4nd 5y ago
Oh sure, I mean, there are tons of contributing factors to human happiness and one’s specific job function in a specific company isn’t a complete determinant. A lot of my anxiety is directly attributable to a tendency towards intense self-c
32.
by bertr4nd 5y ago
Recently I’ve found myself wondering about the average stress levels of managers versus senior engineers. I’m at a big tech company that has roughly parallel tracks up to a quite high level, so I’m not hurting for income, but I see a lot of
33.
by bertr4nd 5y ago
5/100k deaths among unvaccinated can’t possibly be right, can it? There have been 790k Covid deaths so far in the US out of a population of 330 million, which is over 200/100k.
34.
by bertr4nd 5y ago
It feels like there’s a bunch of ML technology that exists that could plausibly be pieces of this solution: there are algorithms for computing embeddings, various audio processing models, recommender system networks, etc. I wonder what’s b
35.
by bertr4nd 5y ago
Interesting, I thought pytorch was a bit more competitive on those other benchmarks (but admittedly it’s been a while since I looked). Slicing shouldn’t be a fundamental problem, but perhaps there are some important details that have been
36.
by bertr4nd 5y ago
In a shameless plug, I want to note that running these sorts of workloads on CPU using Pytorch got much faster (some results on a benchmark from this post’s author’s suite in [0]) in the most recent torch release thanks to the addition of a
37.
by bertr4nd 5y ago
I don’t know. I’ve been in the position of being an “X-builder” (I’m a dyed-in-the-wool compiler engineer), and I’ve been through something like what she describes, and nothing about the decisions involved seemed as obvious (or as maliciou
38.
by bertr4nd 5y ago
I’m reading this list, and it’s a great list but it also makes me sort of depressed. I think I have the combination of time and intellect to achieve maybe half of what’s proposed here. What do you do when you can’t possibly achieve writin
39.
by bertr4nd 5y ago
Truly an amazing output! Did you really do all this in 2.5 months? Seems I need to dramatically up my coding game!
40.
by bertr4nd 5y ago
Where I tend to get stuck is somewhere between the “make something similar to tic-tac-toe, like bingo” and “make a level editor for beat saber”. It’s a difference both in scale, and in divergence from the known. While I’ve built large syst
41.
by bertr4nd 5y ago
The pytorch programming model is just really hard to adapt to an XLA-like compiler. Imperative python code doesn't translate to an ML graph compiler particularly well; Jax's API is functional, so it's easier to translate to
42.
by bertr4nd 5y ago
There’s a “profiling graph executor” that records shapes and then hands them off to a fusion compiler. The profiling executor re-specializes on every new shape it sees, but stops at 20 re-specialization. We’re working on eliminating the de
43.
by bertr4nd 5y ago
Oh also to answer your “as fast as possible” question: usually you’ll get the best performance by exporting your model to a perf-tuned runtime. We’ve seen really good results with TensorRT and (for transformers) FasterTransformer. I’ve al
44.
by bertr4nd 5y ago
It’s both. torch.jit started life as an optimizer. I think fusion of pointwise kernels on GPU - which we finally extended to CPU in this release - was one of the early wins via jit. But at some point it became a model export format for pro
45.
by bertr4nd 5y ago
This product is competing against NVidia for the deep learning space, not AMD. NV’s A100 features enormous memory bandwidth (1.4 TB/s) thanks to HBM and peak flops (300 TFlops/s) thanks to tensor cores, so using HBM and Advanced
46.
by bertr4nd 5y ago
I’m not the OP but if they’re L7+ the current salary (total comp, really) could be 800k+
47.
by bertr4nd 5y ago
While it may be true that many PhDs are not a financially good decision, I think the answer is not necessarily so clear cut for a computer science PhD. I didn’t have any sort of foot in the door at FAANG level companies before my PhD (ask m
48.
by bertr4nd 5y ago
The link to our GitHub repo in a sibling comment probably does more justice than I could do in an HN comment, but it's essentially an ML-graph-to-machine-code compiler that focuses on accelerators. The rationale for open-sourcing here,
49.
by bertr4nd 5y ago
Oh hey, their slides mention Glow, which is the open source ML compiler I worked on at Facebook. Neat to see it getting used here :-)
50.
by bertr4nd 5y ago
This. This actually hits the nail of my imposter syndrome precisely on the head. I've advanced to a pretty good position, and I'm definitely good at a fair number of things; but there are also things I am very much not good at
51.
by bertr4nd 5y ago
I find this post to be more instructive, with more detail and no quasi-religious goofiness: https://www.johndcook.com/blog/2018/04/11/anatomy-of-a-posit...
52.
by bertr4nd 5y ago
If your box has avx512, this paper has a neat vectorized double precision reciprocal: http://www.ecs.umass.edu/arith-2018/pdf/arith25_18.pdf I’ve not tried it yet but I like all the new math instructions.
53.
by bertr4nd 5y ago
I find that I procrastinate the most when I have to write something “required.” I basically never procrastinate on writing code. I also don’t procrastinate when I’m writing things for intrinsic reasons, like I want to describe a neat thing
54.
by bertr4nd 5y ago
I agree so much about “medium heat” or really any subjective measure of heat. When I was learning to cook I smoked up my apartment so many times before I learned that what recipe authors called “high heat” was really about 50% up my rang
55.
by bertr4nd 5y ago
I didn’t downvote (I can’t, and I wouldn’t have anyways) but I’ve often see the “Wait? Why?” (Or “wait, what?”) construct carries a connotation of “Why are you doing this insane bad thing?” which people sometimes react negatively to.
56.
by bertr4nd 5y ago
I’ll add another reason why these scenarios are ridiculous: they assume that the time that goes into the side hustle is “free” and that it’s impossible to advance any faster in one’s “primary” career. I don’t know how things work in medici
57.
by bertr4nd 5y ago
ghstack is amazing. When I started working on PyTorch in GitHub I desperately missed the stacked diff workflow of Phabricator, and ghstack basically made me whole again :-).
58.
by bertr4nd 6y ago
While it’s absolutely true that fixed width instructions make parallel decoding vastly easier, there’s a cost in terms of binary footprint size. x86 generally has an advantage in instruction cache and TLB performance for this reason, which
59.
by bertr4nd 6y ago
Cool project! As I read through I noticed that there doesn’t appear to be any explicit garbage collection, just stack-based deallocation. Is there something about the subset of lisp you implemented that removes the need for GC?
60.
by bertr4nd 6y ago
Just curious, what source are you using for the citation graph? I seem to remember looking for an API to something like ACM digital library at one point and not really finding what I wanted, but maybe I just didn’t know how to look. I love

More ›