lopuhin

10 ms·

31.
by lopuhin 3y ago
Yes it might. "Reduction of number of neurons" is not static here, unlike traditional pruning approaches, here they still keep all weights, but the network dynamically selects which sub-portion of them to use. There is a related d
32.
by lopuhin 3y ago
There are two issues here -- for one, in big transformers, more compute is in the attention layers, while this work improves only feed-forward layers, which are more important for smaller models and smaller sequence lengths. Second, in many
33.
by lopuhin 3y ago
I think it's even from 2021-08-25, see https://patents.google.com/patent/KR20230030188A/en?oq=WO202...
34.
by lopuhin 3y ago
Impressively concise implementation, really interesting paper! Benchmark looks quite questionable though -- e.g. they use fp64 (while any sane person would use at least f32, if not f16), batch 1 (while normally one would try to get the max
35.
by lopuhin 4y ago
They released different sizes from "small" to "xxl", and at least "base" should be small enough to fine-tune virtually anywhere.
36.
by lopuhin 4y ago
In terms of models which are reasonably fast to run and easy to install, I think Flan-T5 is one of the best: https://huggingface.co/google/flan-t5-xxl - although out of the box it's more focused on giving short an
37.
by lopuhin 4y ago
A nitpick on the image Andromeda image: when viewed with telescope and in most images you can find online, the core of the galaxy and the region around the core is much brighter than the outer branches, while on the composite image it looks
38.
by lopuhin 4y ago
> they require a physical device instead of a TOTP app. Yeah, that's the line between reasonable and wtf. No, a TOTP app is also allowed.
39.
by lopuhin 4y ago
Not 'char' - because it's using BPE (byte pair encoding), so after tokenization you might get ["Transform", "ers"] instead of ["T", "r", "a", ...]. This is relevant to how it
40.
by lopuhin 4y ago
Technically it's not the largest, not char, not rnn... but it's close :)
41.
by lopuhin 5y ago
There is a very simple voxel terrian renderer: https://github.com/s-macke/VoxelSpace
42.
by lopuhin 5y ago
No, they don't need a pre-trained model, they start from scratch. Also it's only the approach which is common between modalities, they train them separately.
43.
by lopuhin 5y ago
That's not quite how I understood it > Do the cutout in feature space, not the original input space. I think they do the cutout in the original input space, based on examples they show, e.g. they mask parts of text and grey out part
44.
by lopuhin 5y ago
Zyte | Senior Data Scientist | REMOTE | Full-time | https://www.zyte.com/ At Zyte (formerly Scrapinghub), our goal is to help you get the data from the web, so we develop services such as smart rotating proxies, browser ren
45.
by lopuhin 5y ago
> Carrington Event Not a great time to predict this, at this point in the solar cycle we're close to a minimum of solar activity, the max should be between 2023 and 2026, but it's not expected to be too big either - cycle 25 is
46.
by lopuhin 5y ago
A small roll of the place at the time the photo was taken could also explain this difference.
47.
by lopuhin 5y ago
I prefer a tab for each machine, including local, and then each tab has a tmux or ssh + tmux.
48.
by lopuhin 5y ago
Probably they would be close - M1 still needs to use memory for the OS and other stuff, while 3090 can use fp16/mixed precision, which in many cases almost doubles effective memory. Also if we talk about training, then a more mature CU
49.
by lopuhin 5y ago
Yes, but then they brought it back here https://github.com/numpy/numpy/pull/18874 - apparently Apple developers fixed these issues.
50.
by lopuhin 5y ago
Kaggle has two parts - more popular "featured" competitions which have most of those features, but cost money to host. And then also self-hosted competitions which indeed lack all those features and are more relevant for this comp
51.
by lopuhin 5y ago
I don't think Saturn comes that close, should be 8 AU at best, given that perihelion is at 9 AU.
52.
by lopuhin 5y ago
I think the second paragraph in the wiki page about Bell's theorem explains that: https://en.wikipedia.org/wiki/Bell%27s_theorem
53.
by lopuhin 6y ago
How large is the CO2 market relative to the amount of CO2 currently injected into the atmosphere?
54.
by lopuhin 6y ago
The first option was by far the most popular and was used for years (including pip), only recently packages started dropping python 2 support. I'm not even aware of any packages which went with the second.
55.
by lopuhin 6y ago
> I chose MobileNetV2 to make iteration faster. When I tried ResNet50 or other larger models the gap between the M1 and Nvidia grew wider. (and that's on CIFAR-10). But why not report these results and also test on a more realistic
56.
by lopuhin 6y ago
The service itself is already released, you can play with the demo without registration at https://www.scrapinghub.com/data-api-news (scroll to "Try it out here for yourself"), and we also have a free API trial. W
57.
by lopuhin 6y ago
Yeah I don't think it's trivial. If an algorithm made predictions on the level of html elements - whether an element should be a part of article body or not, it would be possible to combine probabilities or at least vote. But (a)
58.
by lopuhin 6y ago
Great question! I'm an active Pocket user myself, would love to know what they use on the backend. From seeing their failures - when they think something is not an article or excluding some relevant stuff, I would guess they use someth
59.
by lopuhin 6y ago
Author here, ready to answer any questions. This is an evaluation of article body extraction for AutoExtract (ours), Diffbot, newspaper3k, readability-lxml, dragnet, boilerpipe and html-text. Since we're evaluating the quality of our s
60.
Article extraction benchmark: open-source libraries and commercial services(github.com)
19 points by lopuhin 6y ago|10 comments

More ›