5 ms·
You can navigate a website without visually decoding the image of a website.
by odie5533 1y ago
You can navigate a website without visually decoding the image of a website.
- bonoboTP 1y agoExcept if its a messy div soup with various shitty absolute and relative pixel offsets where the only way to know what refers to what is by rendering it and using gestalt principles.
- measurablefunc 1y agoNone of that matters to neural networks.
- bonoboTP 1y agoIt does, because it's hard to infer where each element will end up in the render. So a checkbox may be set up in a shitty way such that the corresponding text label is not properly placed in the DOM, so it's hard to tell what the checkbox controls just based on the DOM tree. You have to take into account the styling and placement pixel stuff, ie render it properly and look at it. That's just one obvious example, but the principle holds more generally.
- measurablefunc 1y agoSpatial continuity has nothing to do w/ how neural networks interpret an array of numbers. In fact, there is nothing about the topology of the input that is any way relevant to what calculations are done by the network. You are imposing an anthropomorphic structure that does not exist anywhere in the algorithm & how it processes information. Here is an example to demonstrate my point: https://x.com/s_scardapane/status/1975500989299105981 https://x.com/s_scardapane/status/1975500989299105981
- bonoboTP 1y agoIt would have to implicitly render the HTML+CSS to know which two elements visually end up next to each other, if the markup is spaghetti and badly done.
- measurablefunc 1y agoThe linked post demonstrates arbitrary re-ordering of image patches. Spatial continuity is not relevant to neural networks.
- bonoboTP 1y agoThat's ridiculous, sorry. If that were so, we wouldn't have positional encodings in vision transformers.
- measurablefunc 1y agoIt's not ridiculous if you understand how neural networks actually work. Your perception of the numbers has nothing to do w/ the logic of the arithmetic in the network.
- bonoboTP 1y agoDo you know what "positional encoding" means?
- measurablefunc 1y agoCompletely irrelevant to the point being made.
- ionwake 1y agoWhy are you talking about image processing ? The guy you’re talking to isn’t
- measurablefunc 1y agoWhat do you suppose "render" means?
- bonoboTP 1y agoThe original comment I replied to said "You can navigate a website without visually decoding the image of a website." I replied that decoding is necessary to know where the elements will end up in a visual arrangement, because often that carries semantics. A label that is rendered next to another element can be crucial for understanding the functioning of the program. It's nontrivial just from the HTML or whatever tree structure where each element will appear in 2D after rendering.
- measurablefunc 1y ago2D rendering is not necessary for processing information by neural networks. In fact, the image is flattened into 1D array & loses the topological structure almost entirely b/c the topology is not relevant to the arithmetic performed by the network.
- bonoboTP 1y agoI'm talking about HTML (or other markup, in the form of text) vs image. That simply getting the markup as text tokens will be much harder to interpret since it's not clear where the elements will end up. I guess I can't make this any more clear.
- ionwake 1y agoThe guy you are talking to is either an utter moron, severely autistic, or for some weird reason he is trolling ( it is a fresh account. I applaud you for trying to be kind and explain things to him, I personally would not have the patience.
- deleted 1y ago[deleted]