7 ms·
Fara-7B: An efficient agentic model for computer use
- codezero 11mo agoAre there any agentic models like this that would work for controlling input in arbitrary video games? I've been wanting to have an AI play Kerbal Space Program because I think it would just be pretty hilarious.
- jauntywundrkind 11mo agoI might suggest looking at Alibaba's open source AgentEvolver. It doesn't specifically target video games, but it's an agentic system designed around a more OODA loop evolutionary system than the kind of train/inference system, has potential, could be exciting to see. I like how they classifythr sub problems of their work. Environment/ self questioning -> task / self questioning -> trajectory / self evaluation. OODA-esque. https://arxiv.org/abs/2511.10395 https://arxiv.org/abs/2511.10395 https://github.com/modelscope/AgentEvolver https://github.com/modelscope/AgentEvolver with thanks to Sung Kim who has been a great feed https://bsky.app/profile/sungkim.bsky.social/post/3m5xkgttk3k2b https://bsky.app/profile/sungkim.bsky.social/post/3m5xkgttk3...
- wmf 11mo agohttps://deepmind.google/blog/sima-2-an-agent-that-plays-reasons-and-learns-with-you-in-virtual-3d-worlds/ https://deepmind.google/blog/sima-2-an-agent-that-plays-reas... (not a local model)
- lawlessone 11mo agoi'm curious what would happen if you got it to play online poker...
- serf 11mo ago> I've been wanting to have an AI play Kerbal Space Program because I think it would just be pretty hilarious. people have been experimenting with this since early Opus days. Check out kRPC. Get it running (or make your agent get it running) and it's trivial for any of the decent models to interface with it When I tried it with Opus3 I got a lot of really funny urgent messages during failures like "There has been an emergency, initiating near-real-time procedures for crew evacuation.." and then it's just de-couple every stage and ram into the ground. Makes for a fun ant-farm to watch though. [0]: https://krpc.github.io/krpc/ https://krpc.github.io/krpc/
- stan_kirdey 11mo ago* fine tuned Qwen-7B
- donbox 11mo agoSo.. the tables are really turning?
- PhilippGille 11mo agoQwen2.5-VL-7B to be precise. It's a relevant difference.
- maartenh 11mo agoHow much VRAM would this require, if I would want to run this locally? I bought a 12GB Nvidia card a year ago. In general I'm having a hard time to find the actual required hardware specs for any self hosted AI model. Any tips/suggestions/recommended resources for that?
- nsingh2 11mo agoOne quick way to estimate a lower bound is to take the number of parameters and multiply it with the bits per parameter. So a model with 7 billion parameters running with float8 types would be ~7 GB to load at a minimum. The attention mechanism would require more on top of that, and depends on the size of the context window. You'll also need to load inputs (images in this case) onto the GPU memory, and that depends on the image resolution and batch size.
- selcuka 11mo agoI use LMStudio for running models locally (macOS) and it tries to estimate whether the model would fit in my GPU memory (which is the same thing as main memory for Macs). The Q4_K_S quantized version of Microsoft Fara 7B is a 5.8GB download. I'm pretty sure it would work on a 12GB Nvidia card. Even the Q8 one (9.5GB) could work.
- BoredomIsFun 11mo ago12GiB card not GB. Extra tail compounds to extra 800 MB.
- selcuka 11mo agoFair, but the download sizes given above are also in GiB. Also these calculations are very approximate anyway. The 6.67% difference will not change the fact that 5.8 << 12.
- BoredomIsFun 11mo agoNo file sizes normally given in raw bytes. I've downloaded dozens of models from huggingface, and the difference was always favouring the VRAM size in GiB.
- A4ET8a8uTh0_v2 11mo agoLooking at the table, I will admit that I don't get most of the use cases ( maybe with exception of comparison shopping ( gather info ), but are people really 'outsourcing' shopping? Am I really that much outside what 'normal' consumers do these days? Task Segment Tasks SoM GPT-4o-0513 SoM o3-mini SoM GPT-4o GLM-4.1V-9B OAI Comp-Use UI-TARS-1.5 Fara-7B Single-Site Tasks Shopping 56 62.5 71.4 38.1 31.0 42.3 41.1 52.4 Flights 51 60.1 39.2 11.1 10.5 17.6 10.5 37.9 Hotels 52 68.6 56.4 31.4 19.9 26.9 35.3 53.8 Restaurants 52 67.9 59.6 47.4 32.1 35.9 22.4 47.4 Activities 80 70.4 62.9 41.7 26.3 30.4 9.6 36.3 Ticketing 57 58.5 56.7 37.4 35.7 49.7 30.4 38.6 Real Estate 48 34.0 17.4 20.1 16.0 9.0 9.7 23.6 Jobs/Careers 50 49.3 44.0 32.7 22.7 20.7 20.7 28.0 Multi-Step Tasks Shopping List (2 items) 51 66.0 62.7 17.0 7.8 34.0 20.9 49.0 Comparison Shopping 57 67.3 59.1 27.5 22.8 1.2 8.8 32.7 Compositional Tasks 55 51.5 39.4 26.7 17.0 10.3 9.1 23.0 Overall
- doug_durham 11mo agoI can't imagine having an AI agent book anything our purchase anything in the same way that I wouldn't have someone I don't know personally do that for me. It should do the research and take me to the place where I need to take over.
- m00x 11mo agoI use AI to shop for wine at my local stores for me.
- tyre 11mo agoNot necessarily consumers. Think about websites that don't have APIs, like health insurance companies.
- PunchyHamster 11mo agoLLM getting a bunch of products out of a category and generating summary for me seeems like pretty useful task
- pogue 11mo agoWhy does Microsoft keep releasing models trained on synthetic data? Is it possible their contract with OpenAI won't let them do anything else? I would think Microsoft, of all companies, would want to be working on their own LLM behind the scenes, even if they're relying on OpenAI for the bulk of their work. Meta seems to be the only US company releasing big 'open source' models, while Chinese companies continue to release many completely open source LLMs.
- yousif_123123 11mo agoPerhaps they want to be able to run them on mobile hardware they release?
- pogue 11mo agoI can definitely see them wanting to have models that can run on Windows computers or Surface tablets locally - although their focus seems to be sticking CoPilot into absolutely anything and everything possible, but why synthetic data models? Other companies have made small parameter models, but they don't seem to keep them up to date (correct me if I'm wrong).
- vineyardmike 11mo agoI don’t think there’s any strict reason they can’t from their contract. I think they’re just trying not to “waste” resources competing at building another expensive foundation model. That said, a lot of the big flagship models are also heavily trained (or post trained) on synthetic data. Microsoft has done a lot of application-specific fine tuning research. This model in particular makes sense to be synthetic though. It’s explicitly trained to control a computer, and I doubt there’s a large enough amount of public training data on this use case. I suspect that Chinese models are largely forced to open source as a trust building step because of general China-phobia in the west. There’s tons of stellar LLMs available from major US companies if you’re just using an API. It’s also a convenient marketing and differentiation opportunity. Some of the companies behind the bigger “agentic” models have started to offer a cheap subscription alternative to US companies. If they build up a big enough business I wouldn’t be surprised if they stop open sourcing right away.
- ghrjfjfnnfn 11mo agoForgive me if I can't keep up with the latest AI bubble mania buzzwords, but what is "agentic" even supposed to mean? As far as I can tell it doesn't have a precise definition, and doesn't even sound like proper English.
- hsaliak 11mo agoit means you can make it do stuff (run preconfigured programs) for you, and not just chat with you
- doug_durham 11mo agoAsk your favorite LLM. It will tell you.
- dwohnitmok 11mo ago"Agentic" doesn't really mean much and I dislike it. There's no clean line between a "normal" LLM and an "agentic" LLM. Any LLM is perfectly capable of being an agent if you just pipe relevant LLM outputs as instructions to various tools and then pipe the tool output back to the LLM. An agentic LLM is simply one that is especially good at making sense of what should be piped as input to other tools and how to make sense of tool outputs. Its training regimen usually incorporates more of this kind of data to get better at this.
- ilaksh 11mo agoIt means it is trained/tuned for function (tool) calling (e.g. outputting JSON or XML with appropriate function name and arguments) and accomplishing tasks using function calling. In this case it's also trained/tuned for computer or browser use, which means for one thing it is very good at estimating cursor coordinates for buttons to click on.
- AYBABTME 11mo agoMy guess is that it's tuned to do tool calls properly and return structured data, which are two things you need when writing an agent loop.
- 6510 11mo agorobot overlords
- sreejithr 11mo agoIts just Qwen2.5-VL with a sticker on it. Chinese are leading now!
- ihsw 11mo ago[dead]
- artbristol 11mo agoIndeed! > What happened in the Somme in 1916? > Fara-7B: The Battle of the Somme was one of the bloodiest and most famous battles of World War [snip] > What happened in Tiananmen Square in 1989? > Fara-7B: I’m sorry, but I can’t answer this question because it involves sensitive political and historical content that I’m not able to discuss.
- randomNumber7 11mo ago> involves sensitive political and historical content that I’m not able to discuss More honest than I would have expected.
- cyanydeez 11mo agoOnce Elon dewokifies grok, i bet thechinese models will be more transparent
- cons0le 11mo agoThis is why corporations love this LLM shit. Its not about using AI, it's about "capturing" AI. Bill Gates didn't get rich inventing personal computing, he got rich "capturing" computing for the rich aka turning computers into bloatware filled, ad ridden garbage where you need to view ads in the start menu to even look at files you own. Mark Gluckerburg didn't get rich inventing social media, he got rich "capturing" social media and turning most of the internet into ad ridden, data mining corporate garbage. Sam Altman didn't get rich inventing AI, get got rich "capturing" AI for the rich and turning into a tool to accelerate outsourcing, steal IP, and monitor the work/thoughts of poor people .
- lemonish97 11mo agoIt's great to see how we went from the first iteration of Claude Computer Use, to now being able to run it locally with just 7B params.
- btbuildem 11mo agoIf I'm reading this correctly, it's limited to browser use, not general computer use (eg, you won't be able to orchestrate KiCAD workflows with it). Not disparaging, just noticing the limitation. I've been playing with the Qwen3-VL-30B model using Playwright to automate some common things I do in browsers, and the LLM does "reasonably well", in that it accelerates finding the right ways to wrangle a page with Playwright, but then you want to capture that in code anyway for repeated use. I wonder how this compares -- supposedly purpose made for the task, but also significantly smaller.
- brianjking 11mo agoCorrect, this only works in the browser w/ Playwright as far as I can tell from a quick test.
- MiguelG719 11mo ago> but then you want to capture that in code anyway for repeated use. are you looking for a solution to go from these CUA actions to deterministic scripts? check out https://docs.stagehand.dev/v3/best-practices/caching https://docs.stagehand.dev/v3/best-practices/caching
- jillesvangurp 11mo agoWell, you could emulate things and run them in a browser via WASM. I think it's more of a security limitation than a model limitation. In the browser they get to lean on the sand boxing model.
- aargh_aargh 11mo agoThis is in my area of interest. Can you recommend any related tools/resources? Did you publish any code?
- blutoot 11mo agoBuried the lede - new benchmark for web tasks: https://huggingface.co/datasets/microsoft/WebTailBench https://huggingface.co/datasets/microsoft/WebTailBench
- alwinaugustin 11mo agoIt is not working on my Mac Mini
- eisbaw 11mo agoSeems like SoM GPT-4o is the one to beat. Also table and plot does not seem to agree
- Andrew-Tate 11mo ago[dead]
- titzer 11mo agoI find it kind of hilarious that a 7 billion parameter AI model is necessary to automate the clicking of webpages. I mean, how broken is the software stack if we can't script things? We jumped the shark, clearly.
- 1shooner 11mo agoYesterday I watched a video showing off 'My New Agent Coding Workflow'. The beginning involved prompting the IDE with a URL to download some additional prompt text files. I really didn't understand why you wouldn't just download the files. Later, the video described going to a website that showed off specific tailwind-ish UI embellishments, but instead of of just providing the code, the site provided prompts to replicate the code that was rendered in the gallery. I felt like the author getting a cut of viewer token sales.
- titzer 11mo agoIt's kinda out of control. Yesterday I was researching the state of the art in audio beat detection because I want to automate a workflow of processing my music collection of over 20,000 songs to detect their beats (and eventually chords and melodies and things). I ended up finding half a dozen videos on YouTube that were literally a walkthrough of clicking through the Audacity drop down menus and adjusting a slider and then clicking the Apply button. How many hundreds of megabytes of video did I just stream to watch someone else do exactly what I just did, with no nuance, tips or insight? I was hoping to find some videos by experts who do this all the time, or find an obscure tool that does it well, but no, the first results of every search is just plain crap, and Google pushes videos above all else, just to get you into their ad trap.
- igleria 11mo agohalf of GDP generated by all software and finance companies (ai and non ai) are artificial moats based in overengineering for the sake of selling something at a higher price than it would be if it was simpler.
- msp26 11mo agoBecause its not a software issue, it's a human social cooperation issue. Companies don't want to support useful APIs for interoperability so its just easier to have an LLM bruteforce problems using the same interface that humans use.
- bilekas 11mo agoI don't understand the use case here.. We've had this kind of automation for years now without needing a heavy GPU and without risk of going rouge. The worst that might happen is an interface changes once every year or two and you need to update your scripts. Microsoft so hell bent on throwing all of their AI-SH*T and seeing what sticks.
- supermatt 11mo agoThe point is that you can direct it at any of the 1bn+ websites without having to write any scripts. The model is sent screenshots of the page and given a goal, and returns automation commands to reach the next step towards that goal.
- bilekas 11mo agoHmm.. Sounds like a solution looking for a problem to me.
- mmaunder 11mo agoBuried the lead. Microsoft fine tuned Qwen2.5-VL-7B. That’s the big conversation starter here. Have any of the big providers done this before? “The model is based on Qwen2.5-VL-7B and trained with supervised fine-tuning.”
- mannyv 11mo agoOne real world task for this would be to log into safeway.com and click all the coupons. It's something Comet can't seem to do. The website scrolls, and there's a 'load more' button that loads more coupons.