t-vi

8 ms·

31.
by t-vi 4y ago
In fact, counting federal, state and local governments, Germany has some 6 million government employees (~4.8m directly and ~1.2m in public agencies). Also, there might be things that are government provided in one country and private in an
32.
by t-vi 4y ago
Do they need to learn rules though, or could they memorize enough to learn the probabilities of the most likely continuation? To my mind the blurry-jpeg-metaphor[1] is very much spot-on. While it is speculation, it would seem to me personal
33.
by t-vi 4y ago
In my understanding, at a very high level and omitting many crucial details, the key is that when you have mainly largish matrix multiplications (as in transformers) well-behaved (mean zero uncorrelated random or so) quantization errors can
34.
by t-vi 4y ago
> Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects While I'm not going to disagree that it matters, to my mind the main thing about early (0.x, 1.x) PyTorch&#x
35.
by t-vi 4y ago
Also Mark Shannon who drew up the original Faster Python plan that spurred this work, joined Microsoft to work on this.
36.
by t-vi 4y ago
Indeed, the "learning". To my mind, the most simple (but still speculative) explanation of the "learning" phenomena - working examples and limitations / failures - we see is that the large models implicitly memorize
37.
Ask HN: Implementing Licensing for a Python Application
2 points by t-vi 4y ago|0 comments
38.
by t-vi 5y ago
I try to have test coverage for the (small) independent libraries I do. It's a bit tricky to go really deep with tests when you don't have a reference, but I like to see that at least basic logic errors and typos in attribute name
39.
by t-vi 5y ago
As someone keenly interested in such topics, would you permit using these pictures, and if so, how would you want to be credited? (the background is https://TorchDrift.org/ , and the image reminded me of the right hand side
40.
by t-vi 5y ago
Julia is a great language and has a lot going for it in areas where Python (and the ecosystem) has weaknesses - my favourite are composability (experienced with math libraries) and the ability to reach high speed. If, on the other hand, you
41.
by t-vi 5y ago
import torch x = torch.randn(4000, 4000, device='cuda:0') y = torch.randn(4000, 4000, device='cuda:0') import time torch.cuda.synchronize() t0 = time.perf_counter() z = torch.zeros(4000, 4000, device=&#
42.
by t-vi 6y ago
Not directly. It signals that the compiler does well when optimizing the [[likely]] path at the expense of making the other paths less optimized.
43.
by t-vi 6y ago
Part of the problem is that Vega GPUs are too good for crypto currency mining. Example: I bought a 16GB Radeon VII for €550 (including 19% VAT). That would be a decent value proposition for Deep Learning even today. However, the card appear
44.
by t-vi 6y ago
What you describe sounds a lot like the PyTorch support before this announcement: You could download PyTorch from AMD's ROCm site or build it yourself for >= 2 years now and this worked very reliably. (Edit: The two years (Nov 2018
45.
by t-vi 6y ago
The maths seem right, but I don't think the 6.5 btc/10 minutes is a given for the future, so you're extrapolation contains the rather large assumption that it stays the same.
46.
by t-vi 6y ago
That is sad, I would use SpaCy more if it had Debian packages (in particular in Debian). Python stuff packaged by Debian seems to work very well for me and has been for years.
47.
by t-vi 6y ago
Given that SpaCy uses PyTorch, that is being worked on.
48.
by t-vi 6y ago
Well, the game in machine learning is averages conditional on the input. So if the input essentially identifies and individual you'll get to average over the inputs for the individual.
49.
Optimizing models using the PyTorch JIT(lernapparat.de)
3 points by t-vi 6y ago|0 comments
50.
by t-vi 6y ago
I'm not neutral, but so: Just as there are many different applications of deep learning, "Production" is quite heterogeneous (you put something behind a server, or integrate it in a large codebase written in $foo, or put it o
51.
by t-vi 6y ago
The slots for the functions implementing Python functions in C. The canonical thing is here: https://docs.python.org/3/c-api/objimpl.html
52.
by t-vi 6y ago
The NumFocus board agrees with you: > Jeremy’s talk offers the kind of exchange of ideas that makes an intellectual community vibrant and healthy.
53.
by t-vi 6y ago
The harm that has been done has been done. Code of conduct enforcement never is nice or fun on any side of it. Here it has been terrible, and for all Jeremy has suffered, there is little reason to doubt that the people on the other side of
54.
by t-vi 6y ago
I think PyTorch support is decent, I use it on my Radeon VII a lot. - You can compile on https://lernapparat.de/pytorch-rocm/ (disclaimer my own link) - Arch has support out of the box: https://aur.archlinux
55.
by t-vi 6y ago
I run Debian (mostly) on it. Seems to work well for me. Compiling PyTorch is a bit of a nuisance at 4GB, but it worked. One issue I had was that TRTorch wanted bazel to build and I couldn't bring myself to install that (and Java), so I
56.
by t-vi 6y ago
Quite often, people seem to overestimate the performance overhead Python brings (one can take the PyTorch C++ extension example (LLTM) and create a 1-1 LibTorch implementation to see a ~10% speedup or so). But Paul's situation is multi
57.
by t-vi 6y ago
The underappreciated (in my view/experience) part is that it also gets rid of a lot of GIL when used from Python because the part inside the JITed doesn't use Python anymore. When you have multithreaded setups, this typically is m
58.
by t-vi 6y ago
> The solution to Python’s GIL bottleneck is not some trick, it is to stop using Python for data-path code. At least for the PyTorch bits of it, using the PyTorch JIT works well. When you run PyTorch code through Python, the intermediate
59.
by t-vi 6y ago
Well, they are incentivizing them. Just to get a notion of the scale: https://github.com/search?q=amazing+project&type=issues
60.
by t-vi 6y ago
They have been asked to, but say they can't?

More ›