jhj

12 ms·

61.
by jhj 6y ago
I think the latency overhead can be low enough these days (kernel launch / copy data from CPU to GPU, read from global memory, do some compute, write to global memory, copy data back to CPU and wait for completion from the CPU side (wh
62.
by jhj 6y ago
Sensors such as CCDs are largely analog devices (up to the A-to-D converter), where thermal noise / shot noise etc. are things that matter. Cooling a CCD via active cooling or a heat sink will help lower the noise floor and improve the
63.
by jhj 6y ago
IEEE 754 also provides for decimal radix floating point, just that there is no real hardware support for it beyond some IBM and Fujitsu chips, and what you can put together on FPGAs.
64.
by jhj 6y ago
No, those national lab supercomputers typically quote double-precision flops, whereas this appears to be about float16 tensor ops for inner product (they call it "AI flops"). The A100 is 9.7 general purpose Tflops of float64, vers
65.
by jhj 6y ago
The biggest problem with HLS is that the HLS vendors still want to pretend it's "C++ / OpenCL / whatever to gates". What you get is pretending that there is no such concept of a clock even though you know it is alwa
66.
by jhj 6y ago
it's in the paper https://github.com/facebookresearch/tabert
67.
by jhj 6y ago
The backend is LLVM. Sean wrote his own C++17 compatible frontend to provide this functionality (thus, Circle is effectively a replacement for clang).
68.
by jhj 6y ago
I work for Facebook.
69.
by jhj 6y ago
The optimal memory layout usually depends upon which dimensions are the reduction dimensions (for 2d convnet convolution, the spatial dimension on which there is also reduction is likely less important than the batch or channel dimension).
70.
by jhj 6y ago
Much of my time is spent writing Mentor Catapult HLS for ASIC designs these days. Every HLS vendor or language has their own, incompatible arbitrary bitwidth integer type at present. SystemC sc_int is different from Xilinx Vivado ap_int is
71.
by jhj 7y ago
The first would be SN/Lush from circa 1987, of which Torch (from NEC Research), and then PyTorch are direct descendants in a lot of ways. https://leon.bottou.org/projects/lush
72.
by jhj 7y ago
there's a C++14 frontend https://pytorch.org/cppdocs/frontend.html#end-to-end-example though the typing doesn't dive into tensor dimensions etc.
73.
by jhj 7y ago
It is more or less the same, it's just that in NUMA you have a limited number of localities, except here it is in the thousands. The issue is one of scheduling that locality. Some process still needs to determine what data is actually
74.
by jhj 7y ago
> CNNs only make sense when working with image data anyways Not true, N-dimensional convnets, 1-d convnets (for NLP and time series analysis), spatially sparse convnets, graph and non-Euclidean space convnets, ... exist and are used. CNN
75.
by jhj 7y ago
I've used One Medical in NYC for the past two years but I'll probably drop them the next year. The primary care checkups were nowhere near as thorough as ones I've had with one-off doctors. The only benefit that I value that
76.
by jhj 7y ago
It's not like there is one SRAM, there are many SRAMs, so you get the same problem as NUMA but a thousand fold. Some computations you can map to a regular grid/hypercube/whatever quite easily, but it is unclear what the inter
77.
by jhj 7y ago
Nice read! I'm curious as to what settings you used for product quantization, and the general setup that you tried. I find that there can generally be a smooth quality degradation from raw float32, to float16, through 8/6/4 b
78.
by jhj 7y ago
Tbey mean radix-10 floating point, as compared to the radix-2 floating point you are thinking of. The packing of the decimal fractional digits in the significand of a radix-10 FP number need not be in BCD, it can use other encodings (e.g.,
79.
by jhj 7y ago
pretty much every EDA tool uses tcl
80.
by jhj 7y ago
tru dat
81.
by jhj 7y ago
The AC1/2 code was a pile of poo to be honest, but it still managed to do many things light-years ahead of other piles of game code poo at the time. Great concepts, but questionable execution (many people, many of whom were brilliant b
82.
by jhj 7y ago
MMOs are happily two orders of magnitude easier to write today. Before game engines you can license or get for free really existed, it was just you, the compiler, and a text editor, which rapidly turned into many people, the compiler, your
83.
by jhj 7y ago
The NPS doesn't have the data as you can camp (fairly) freely without permits. I've done it a few times before (all in the fall through spring). The major issue is in finding flat, suitable ground for a tent; the most interesting
84.
by jhj 7y ago
That claim can only be made with respect to using a Kulisch accumulator (what he calls the quire) for accumulation. This can represent the exact sum of any number of floating point values (or products of floating point values) with the inte
85.
by jhj 8y ago
No, it's the arithmetic intensity of the operation in a roofline model sense [1] that indicates whether or not the GPU is worth it, and whether you are in a memory bandwidth or compute bound regime. Asymptotic algorithmic complexity fo
86.
by jhj 8y ago
The data is sampled in the Fourier domain. A complete scan (frequency up to a desired Nyquist limit) takes a long time for the MRI machine to acquire all of these samples. If you can get by with sampling only a subset of this space and appr
87.
by jhj 8y ago
The rounding errors for linear posit are exact numbers (excepting division), assuming higher precision. The rounding errors for LNS add/sub are not exact numbers in the representation in the general case. 2 and sqrt(2) are represented
88.
by jhj 8y ago
For (linear) posit, what the "last place" is varies. Versus a fixed-size significand, there is no 0.5 ulp guarantee. If you are in the regime of full precision, then there is a 0.5 ulp guarantee. The rounding also becomes logarith
89.
by jhj 8y ago
The Kulisch accumulator and entropy coding of the floating point words (tapering) address this particular issue. They allow you to get away with much smaller word sizes while preserving dynamic range (and precision!) than would otherwise be
90.
by jhj 8y ago
None of the representations considered in the paper (log or linear posit or log posit) respect the IEEE standard, deliberately so :)

More ›