soraki_soladead

20 ms·

61.
by soraki_soladead 5y ago
The talk on assembly theory was really interesting. Thanks for sharing. I'd be surprised if the idea of causal constraints or quantifying complexity in chemistry was novel. Is it the context of life search that makes novel? Does you kn
62.
by soraki_soladead 5y ago
Here's the referenced tutorial: https://milayb.notion.site/GFlowNet-Tutorial-919dcf0a0f0c4e9... (Some of the links appear to require signing in to Notion but this one doesn't.)
63.
by soraki_soladead 5y ago
I think it depends on what’s meant by “deep learning”. If you mean the latest multi-billion parameter transformer architecture trained on narrow domain data then yeah, you’re probably right. If you mean large networks of simple computationa
64.
by soraki_soladead 5y ago
> Similar to our work, both BYOL (Grill et al., 2020) and DINO (Caron et al., 2021) regress neural network representations of a momentum encoder, but our work differs in that it uses a masked prediction task and we regress multiple neura
65.
by soraki_soladead 5y ago
> They don’t exist. They’re the imagined scapegoat for what is actually just a much harder to comprehend creeping bureaucracy cost. > The whole insurance system we have is an impractical mess designed to fail . You shifted the boogey
66.
by soraki_soladead 5y ago
Indeed. Neurons don't stop processing during events like saccades. They are hyperpolarized by inhibitory neurons. This suppresses further spikes but it's not a separate mode of operation.
67.
by soraki_soladead 5y ago
Perceptrons were modeled after neurons in the sense of simple distributed computational units connected by weights. It doesn’t go any further than that. Convnets were not inspired by biology directly and the only property they share in comm
68.
by soraki_soladead 5y ago
This is a poor hot take. The title isn’t sensationalized. It’s pretty accurate. Neural networks are only in the loosest sense inspired by biology. Backprop itself was not inspired by biology. The implementation of backprop has been known fo
69.
by soraki_soladead 5y ago
I have provided recent and relevant citations with data and detailed comments. You have provided? What? Attacks? I'm honestly not even sure. Given your comment history I think I'm done here.
70.
by soraki_soladead 5y ago
The papers I provided go a bit beyond "electrical usage". One of them is cited by the paper in question. Yes, they are approximations in lieu of more accurate data but that doesn't invalidate them as a tool or a motivation fo
71.
by soraki_soladead 5y ago
This is somewhat well covered in prior research, in so far as the information required is available. Here are some recent evaluations of the carbon footprint of modern models: https://arxiv.org/abs/1906.02243 (cited in
72.
by soraki_soladead 5y ago
This problem was recently addressed in detail here[1] which serves as a good overview. 1. https://arxiv.org/abs/2106.12423 (project page: https://nvlabs.github.io/alias-free-gan/ )
73.
by soraki_soladead 6y ago
The actual title is “How Complex Systems Fail” (full url: how.complexsystems.fail) which provides a slightly different meaning. The current title “Complex Systems Fail” implies that we should not have complex systems but that’s does not app
74.
by soraki_soladead 6y ago
This seems obvious but the reason has nothing to do with GPT-3. It’s just a bad idea to build a business around someone else’s API, especially when there’s no real competition to that API (for now) and it’s nontrivial to roll your own repla
75.
by soraki_soladead 7y ago
It might be too subtle and difficult to organize but it would be pretty funny and effective if _everyone_ showed up as Mei. It would be really hard for Blizzard to penalize anyone for it but the message would be pretty clear if enough peopl
76.
by soraki_soladead 8y ago
I'm not going to say that ML is a panacea or always perfect but its also not one "thing". It's a complex field with a lot of required knowledge and ways to set things up. IBM is largely to blame for that misunderstanding
77.
by soraki_soladead 8y ago
Evolutionary algorithms are a type of machine learning, so yes. Machine learned vision, speech, NLP, etc. outperform heuristic methods and those are just the recent advances. A random forest can easily outperform a hand-made decision tree—a
78.
by soraki_soladead 8y ago
Two reasons come to mind: DeepMind and OpenAI are indirectly in competition with each other. The Gym interface is okay but the implementation is poor and does not readily support multiple agents.
79.
by soraki_soladead 8y ago
SHAP is more recent and does work with (just about) any ML model: https://github.com/slundberg/shap There's still more work to do in interpretability but models are rarely opaque black boxes with no way to see ins
80.
by soraki_soladead 8y ago
Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with
81.
by soraki_soladead 8y ago
I believe it's both. The produced images do not encode pixel-perfect details necessary for reconstruction. These details are reconstructed from the imperceptible differences in the outputs which act as markers _and_ the model's me
82.
by soraki_soladead 8y ago
"Overfitting" would be a little more specific. Basically the task was easier to accomplish by memorizing the data than by generalizing. This was pretty clear from the original CycleGAN paper[0] (figure 4; bottom row) since it was
83.
by soraki_soladead 8y ago
My other comment addresses some of this but you're overstating things a bit. Throwing more data at the model is one solution. Its just not the only, or even best, approach. Properly measured performance on good holdouts and the applica
84.
by soraki_soladead 8y ago
That outcome doesn't seem terribly likely. It's true that, like big data, deep learning is often misused. This is largely because it works well enough in the off-the-shelf case and it's "easier" due to tooling, tran
85.
by soraki_soladead 8y ago
That's actually closer to how deep learning started. Initially, deep learning mostly consisted of unsupervised (task independent) features with a linear classifier on top. We had to fit an unsupervised model (e.g. autoencoder) layer by
86.
by soraki_soladead 8y ago
I agree good models like xgboost get buried and inexperienced practitioners jump to deep models too quickly. Often without understanding how to properly architect and tune them. Always start with a simple baseline (EDIT: and good process).
87.
by soraki_soladead 8y ago
Electronic medical records. Anything in healthcare/medical, financial services, legal. There's a long way from the "worst text classification" to "sufficient text classification" in most real world use cases. W
88.
by soraki_soladead 8y ago
This is just a bug in their code. The paper they cite uses 1D convolutions. Though, I suppose having an unused dimension only really hurts efficiency.
89.
by soraki_soladead 8y ago
For most businesses a GPU is not really expensive, particularly if the results are important to the business. Chasing a single metric is certainly too narrow but getting the best results often does matter in a professional context too. Whil
90.
by soraki_soladead 8y ago
Your understanding of what brute force means is incorrect. It is an exhaustive algorithm. Any non-exhaustive algorithm is no longer brute force. An algorithm (like AlphaZero) that can locally generalize, even to small degrees, is also not b

More ›