KhoomeiK

8 ms·

31.
by KhoomeiK 2y ago
Totally agreed—this is a design choice that basically comes from our agent architecture, and the codegen-based architecture that we think will likely proliferate for web agent tasks in the future. We provide Tarsier's text/screens
32.
by KhoomeiK 2y ago
Yup, evals can definitely be tough. We basically have a suite of several hundred web data extraction evals in a tool we built called Bananalyzer [1]. It's made it pretty straightforward for us to benchmark how accurately our agent gene
33.
by KhoomeiK 2y ago
Thanks! Yeah, it seems like a lot can be done with just text while we wait for multimodal models to catch up. The recent Platonic Representation Hypothesis [1] also suggests that different models, regardless of modality, build the same inte
34.
Show HN: Tarsier – Vision utilities for web interaction agents(github.com)
192 points by KhoomeiK 2y ago|61 comments
35.
by KhoomeiK 2y ago
1 month away ;)
36.
by KhoomeiK 2y ago
For those who don't know, the senior author on this paper (Sepp Hochreiter) was the first author on the original paper with Schmidhuber introducing LSTMs in 1997.
37.
by KhoomeiK 2y ago
Yep, until you generate code—it's harder from a technical POV but you can get way higher performance & reliability.
38.
by KhoomeiK 2y ago
This is essentially what we're building at https://reworkd.ai (YC S23). We had thousands of users try using AgentGPT (our previous product) for scraping and we learned that using LLMs for web data extraction fundamentally d
39.
by KhoomeiK 3y ago
A group of PhD students at Stanford recently wanted to take AI/ML research ideas generated by LLMs like this and have teams of engineers execute on them at a hackathon. We were getting things prepared at AGI House SF to host the hackat
40.
by KhoomeiK 3y ago
Karpathy’s micrograd [1] is literally 154 lines. Guess programming ended 4 years ago. [1] https://github.com/karpathy/micrograd
41.
by KhoomeiK 3y ago
Gymnasium is now maintained by the Farama Fpundation, an open-source consortium, not OpenAI. But most RL environment work for the past 5+ years has been Gym-compliant. The TextWord example in the repo, for example, instantiates a Gym-style
42.
by KhoomeiK 3y ago
Thanks! Yeah RL for LLMs is pretty underexplored I think beyond the RLHF stuff. Pretty tough to get working tho.
43.
by KhoomeiK 3y ago
Twitter thread: https://x.com/khoomeik/status/1766805213644800011?s=46
44.
by KhoomeiK 3y ago
Yup, it does simplify LLM agent inference on Gym environments but the main technical contribution is reducing your would-be code overhead for online RL
45.
Show HN: LlamaGym – fine-tune LLM agents with online reinforcement learning(github.com)
239 points by KhoomeiK 3y ago|28 comments
46.
De-Redacting Elon's Email with Character-Count Constrained Llama2 Decoding(twitter.com)
2 points by KhoomeiK 3y ago|0 comments
47.
by KhoomeiK 3y ago
Thanks!
48.
WikiLLM: LLMs as Collaboratively Edited Knowledge Bases(twitter.com)
4 points by KhoomeiK 3y ago|2 comments
49.
Llama2D: 2D Positional Embeddings for Webpage Structural Understanding(twitter.com)
2 points by KhoomeiK 3y ago|0 comments
50.
HoleFill: Automated Data Curation to Fill User-Centric LLM Knowledge Gaps(twitter.com)
6 points by KhoomeiK 3y ago|0 comments
51.
Ashish Vaswani announces new startup Essential AI (after having left Adept)(twitter.com)
2 points by KhoomeiK 3y ago|0 comments
52.
by KhoomeiK 3y ago
You might be interested in this workshop: https://www.neurreps.org/ And a possibly relevant paper from it: https://openreview.net/forum?id=Ag8HcNFfDsg
53.
by KhoomeiK 3y ago
We use Google OCR to convert the screenshot into whitespace-structured text for unimodal LLMs. For multimodal LLMs like GPT-4V, you don't need to use the OCR utilities. Thanks for the feedback though, we'll clarify!
54.
by KhoomeiK 3y ago
The title made me think this was an approach that used memory editing techniques (e.g. ROME [1]) to allow an LLM's neural memory (not just its context) to change over the course of conversation. Pretty happy to realize that this is jus
55.
by KhoomeiK 3y ago
Not gonna lie, I'm not looking forward to seeing the Cisco logo every time I go to Santana Row.
56.
by KhoomeiK 3y ago
Fyi, Meng et al 2022 [1] is pretty much required reading in order to understand this paper [1] https://arxiv.org/abs/2202.05262
57.
by KhoomeiK 3y ago
Please do elaborate! This [1] is pretty much the only succesful use case I've seen so far. The use case in NLP [2] I've seen was a negative result. [1] https://arxiv.org/abs/1802.04443 [2] https://
58.
by KhoomeiK 3y ago
I feel like everyone in data science or ML that has some higher math background goes through their TDA phase, trying to apply persistent homology to everything. My view now is that the applications are quite limited, but the math is still c
59.
by KhoomeiK 3y ago
> If all physical processes are subservient to a thermodynamical "telos", how could any manifestation of free will emerge? Not even going to get into how problematic it is to cite conciousess as a fundamental goal of when we ca
60.
by KhoomeiK 3y ago
For those who are unaware, this is the same worldview that underlies e/acc thought: https://effectiveaccelerationism.substack.com/p/repost-notes... TLDR: the universe's telos is entropy increase, and thus its

More ›