14 ms·
Darkbloom – Private inference on idle Macs
- DeathArrow 6mo agoWhy only Macs? If we think of all PCs and mobile phones running idle, the potential is much larger.
- stryakr 6mo agosimple first target, PCs have more variability
- btown 6mo agoFrom the paper: https://github.com/Layr-Labs/d-inference/blob/master/papers/dginf-private-inference.pdf https://github.com/Layr-Labs/d-inference/blob/master/papers/... > Apple’s attestation servers will only generate the FreshnessCode for a genuine device that checks in via APNs. A software-only adversary cannot forge the MDA certificate chain (Assumption 3). Com- bined with SIP enforcement (preventing binary replace- ment) and Secure Boot (preventing bootloader tampering), this provides strong evidence that the signing key resides in genuine Apple hardware.
- saagarjha 6mo agoI am not entirely sure they understand that System Integrity Protection and Secure Boot can be turned off.
- deleted 6mo ago[deleted]
- btown 6mo agoMy understanding from the paper is that doing so should cause certain things in Apple's hardware security enclaves to break a signing chain, and a server-side MDM system integrated with Apple servers can detect this. But I'm not familiar with the underlying technology, so not sure if underlying assumptions are incorrect.
- saagarjha 5mo agoAFAIK that just ensures the SEP is present but perhaps they are signing the boot state now
- nl 6mo agoThey use the Apple TEE which they claim also protects GPU memory (I wasn't aware of this). NVidia data center GPUs have a similar path, but not their consumer ones. Not sure about the NVidia Spark. It's possible AMD Strix Halo can do this, but unlikely for any other PC based GPU environments.
- MrDrMcCoy 6mo agoEpyc has that VM encrypted memory thing, which comes pretty close. It does raise an interesting question, though: would a PCIe card passed through to a VM be able to DMA access the memory of neighboring devices?
- rvz 6mo agoShould have called it “Inferanet” with this idea. Away this looks like a great idea and might have a chance at solving the economic issue with running nodes for cheap inference and getting paid for it.
- nl 6mo agoThey use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE) The key question here is how they avoid the outside computer being able to view the memory of the internal process: > An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communication channels that could be observed, with optional hypervisor mem- ory isolation that extends protection from software- enforced to hardware-enforced via ARM Stage 2 page tables at zero performance cost.[1] I was under the impression this wasn't possible if you are using the GPU. I could be misled on this though. [1] https://github.com/Layr-Labs/d-inference/blob/master/papers/dginf-private-inference.pdf https://github.com/Layr-Labs/d-inference/blob/master/papers/...
- flockonus 6mo agoWhile they do make this argument, realistically anyone sending their prompt/data to an external server should assume there will be some level of retention. And more so in particular, anyone using Darkbloom with commercial intents should only really send non-sensitive data (no tokens, customer data, ...) I'd say only classification tasks, imagine generation, etc.
- joelthelion 6mo agoThere's a difference between trusting Anthropic and trusting random mac owners.
- flockonus 5mo agoI know where my answer lies in that; but i don't claim to be an objective truth. For example OpenAI has been caught sharing data with the gov. agencies.
- ramoz 6mo agoMacs do not have an accessible hardware TEE. Macs have secure enclaves.
- kennywinker 6mo agoI have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?
- znpy 6mo agoBeing the middleman is often way more profitable
- foota 6mo agoCapital and availability?
- kennywinker 6mo agoI guess if it only works at scale capital is maybe the answer. Like enough cash to buy 5 or 10 or even 100 minis seem doable - but if the idea only works well when you have 10,000 running - that makes some sense.
- gleenn 6mo agoPower and racking are difficult and expensive?
- kennywinker 6mo agoHow difficult? Is running 1000 minis worth $1,000,000/month of effort? I feel like it is.
- runako 6mo agoThere are many people who do not have ready access to a million dollars to purchase said Mac minis, much less the operating capital to rack & operate them. Very smart play to build a platform, get scale, and prove out the software. Then either add a small network fee (this could be on money movement on/off platform), add a higher tier of service for money, and/or just use the proof points to go get access to capital and become an operator in your own pool.
- chaoz_ 6mo agoThat solution actually makes great sense. So Apple won in some strange way again? Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. I think batch-evals for non-sensitive data has great PMF here.
- rvz 6mo agoYes. They never needed to participate in the AI race to zero. Because they were already at the finish line with Apple Silicon. > I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. The whole inference is end-to-end encrypted so none of the nodes can see the prompts or the messages.
- chaoz_ 6mo agoFun question: can some (part of it) be a crypto token that I can buy? :)) That would finally be a crypto thing which is backed by value I believe in.
- 59nadir 6mo ago> So Apple won in some strange way again? Heh, what did they win exactly? This is just a way for another company to extract value out of the single region of the world where Apple is a relevant vendor, and it happens to be the one where it's the easiest to pull people into schemes.
- bentt 6mo agoI thought this was Apple’s plan all along. How is this not already their thing?
- TuringNYC 6mo agoI'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute. Any suggestions from anyone? We currently run ollama but manually manage the pools
- damezumari 6mo agohttps://github.com/exo-explore/exo https://github.com/exo-explore/exo
- utopiah 6mo agoSeems like so much more work than "just" paying for https://huggingface.co https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token.
- TuringNYC 6mo agoIt is much more work because for many workloads you have geographic ringfencing and cannot send it out to the cloud
- utopiah 6mo agoDoubt this kind of workloads would agree to send data then to a cloud of randos devices, precisely when cloud providers to certify they aren't looking at clients data (Customer-managed encryption keys, CMEK).
- TuringNYC 5mo ago>> Doubt this kind of workloads would agree to send data then to a cloud of randos devices, Totally agree, which is why i said "I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute."
- zozbot234 6mo agoIf you set CPUSchedulingPolicy=idle Nice=19 IOSchedulingClass=idle in the ollama server configuration it should run in the background with lowest priority.
- pants2 6mo agoCool idea. Just some back-of-the-envelope math here (not trusting what's on their site): My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output. That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7. Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs. Could be good for lunch money every once in a while! Around $700/yr.
- MrDrMcCoy 6mo agoDon't forget to factor in cooling costs.
- pants2 6mo agoOr saved heating costs in the winter!
- todotask2 6mo agoOpenAI has only about 5% paying customers, how does it generate revenue? I don’t think this is a sustainable business model. For example, Cubbit tried to build decentralised storage, but I backed out because better alternatives now exist, and hardware continues to improve and become cheaper over time. Your electricity and ownership are going to get lower return and does not actually requce CO2.
- chaoz_ 6mo agoGenuinely curious, is there any way to estimate amortization of Mac? I’d imagine 1 year of heavy usage would somehow affect its quality.
- pants2 6mo agoYeah, only way to get there is assuming they're not giving prompt caching discounts while my laptop is getting prompt caching benefits, with very many large prompts. So yes I am skeptical of their numbers.
- xendo 6mo ago
- BingBingBap 6mo agoGenerate images requested by randoms on the internet on your hardware. What could possibly go wrong?
- pants2 6mo agoYou might not even know it as a user but the payment/distribution here is all built on crypto+stablecoins. This is a great use case for it.
- rvz 6mo agoGood. Another great non-speculative use-case for crypto and stablecoins.
- kennywinker 6mo agoAmazing! Let me see, doing the math r/n… carry the one, yup that makes the total number of non-speculative uses for crypto and stablecoin: 1 ;P
- rvz 5mo agoIt always has been payments. x402 and Stripe Tempo makes the use case more than 1.
- ramoz 6mo agoUnfortunately, verifiable privacy is not physically possible on MacBooks of today. Don't let a nice presentation fool you. Apple Silicon has a Secure Enclave, but not a public SGX/TDX/SEV-style enclave for arbitrary code, so these claims are about OS hardening, not verifiable confidential execution. It would be nice if it were possible. There's a lot of cool innovations possible beyond privacy.
- geon 6mo agoEvery hardware key will be broken if there is enough incentive to do so. Their claims read like pure hubris.
- znnajdla 6mo agoWho cares about AI privacy? Most people don’t. If you do, run locally.
- znnajdla 6mo agoAs if you get privacy with the inference providers available today? I have more trust in a randomly selected machine on a decentralized network not being compromised than in a centralized provider like OpenAI pinky promising not to read your chats.
- ramoz 6mo agoInference providers don't claim private inference. However, they must uphold certain security and legal compliances. You have no guarantees over any random connected laptop connected across the world.
- deleted 6mo ago[deleted]
- znnajdla 6mo agoI would say the chances of OpenAI itself getting hacked and your secrets in logs getting leaked are about the same or less as the chances of a randomly selected machine on a decentralized network being reverse-engineered by a determined hacker. There's no risk-free option, every provider comes with risks. If you care about infosec you have to do frequent secret rotation anyway.
- dr_kiszonka 6mo ago"These are estimates only. We do not guarantee any specific utilization or earnings. Actual earnings depend on network demand, model popularity, your provider reputation score, and how many other providers are serving the same model. When your Mac is idle (no inference requests), it consumes minimal power — you don't lose significant money waiting for requests. The electricity costs shown only apply during active inference. Text models typically see the highest and most consistent demand. Image generation and transcription requests are bursty — high volume during peaks, quiet otherwise."
- dcreater 6mo agoI cant buy credits - says page could not load
- stuxnet79 6mo agoSo basically ... Pied Piper.
- JaggerJo 6mo agofinally!
- tgma 6mo agoI installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.
- thatxliner 6mo agoand I don't think they ever will unless they're highly competitive (hopefully that price they have stays? at least for users) I was thinking of building this exact thing a year ago but my main stopper was economics: it would never make sense for someone to use the API, thus nobody can make money off of zero demand. I guess we just have to look at how Uber and Airbnb bootstrapped themselves. Another issue with my original idea was that it was for compute in general, when the main, best use-case, is long(er)-running software like AI training (but I guess inference is long running enough). But there already exist software out there that lets you rent out your GPU so...
- tgma 6mo agoPeople underestimate how efficient cost/token is for beefy GPUs if you are able to batch. Unlikely for one off consumer unit to be able to compete long term.
- starkeeper 6mo agoWhat's a good place to do this?
- koliber 6mo agoApple should build this, and start giving away free Macs subsidized by idle usage.
- jboggan 6mo agoIs this named after the 2011 split album with Grimes and d'Eon?
- gndp 6mo agoThey are almost claiming FHE, isn't it just a matter of creating the right tool to get the generated tokens from RAM before it gets encrypted for transfer. How is it fundamentally different than chutes?
- resonanormal 6mo agoI could imagine this working for the openclaw community if the price is right
- Havoc 6mo agoThat was my first thought too especially for talks that aren’t particularly important like daily digests of online things
- 0xelpabl0 6mo ago[dead]
- 0xbadcafebee 6mo agoI'm not sure how the economics works out. Pricing for AI inference is based on supply/demand/scarcity. If your hardware is scarce, that means low supply; combine with high demand, it's now valuable. But what happens if you enable every spare Mac on the planet to join the game? Now your supply is high, which means now it's less valuable. So if this becomes really popular, you don't make much money. But if it doesn't become somewhat popular, you don't get any requests, and don't make money. The only way they could ensure a good return would be to first make it popular, then artificially lower the number of hosts.
- utkarsh_apoorva 6mo agoLike the concept. This is not a business - should be an open source GitHub repo maybe. They lost me with just one microcopy - “start earning”. Huge red signal.
- Hamuko 6mo agoBut why would I donate my Mac Studio's idle time if I couldn't "start earning"?
- jaylane 6mo agolatest (v0.3.8) tar doesn't contain image-bank or gRPCServerCLI dependencies so installer fails.
- amdivia 6mo agoUntil we have breakthroughs in homomorphic encryption compute, I won't trust such privacy claims
- jiusanzhou 6mo ago[dead]
- jstlykdat 6mo ago[dead]
- woadwarrior01 6mo agoI won't install some random untrusted binary off of some website. I downloaded it and did some cursory analysis instead. Got the latest v0.3.8 version from the list here: https://api.darkbloom.dev/v1/releases/latest https://api.darkbloom.dev/v1/releases/latest Three binaries and a Python file: darkbloom (Rust) eigeninference-enclave (Swift) ffmpeg (from Homebrew, lol) stt_server.py (a simple FastAPI speech-to-text server using mlx_audio). The good parts: All three binaries are signed with a valid Apple Developer ID and have Hardened runtime enabled. Bad parts: Binaries aren't notarized. Enrolls the device for remote MDM using micromdm. Downloads and installs a complete Python runtime from Cloudflare R2 (Supply chain risk). PT_DENY_ATTACH to make debugging harder. Collects device serial numbers. TL;DR: No, not touching that.
- WatchDog 6mo agoI installed two models, but it just always reports: Available models (2): CohereLabs/cohere-transcribe-03-2026 (4.6 GB) flux_2_klein_9b_q8p.ckpt (20.2 GB) ... Advertising 0 model(s) (only loaded models) Also the benchmark just doesn't work. Interesting idea, but needs some work.
- eddie-wang 6mo ago[dead]
- gleenn 6mo agoYou have to install their MDM device management software on your computer. Basically that computer is theirs now. So don't plan on just handing over your laptop temporarily unless you don't mind some company completely owning your box. Still might be a validate use for people with slightly old laptops lying around, but beware trying to share this computer with your daily activities if you e.g. use a bank on a browser on this computer regularly. MDM means they can swap out your SSL certs level of computer access, please correct me if I'm wrong.
- mirashii 6mo agoMDMs on macOS are permissioned via AccessRights, and you can verify that their permission set is fairly minimal and does not allow what you've described here (bits 0, 4, 10). That said, their privacy posture at the cornerstone of their claims is snake oil and has gaping holes in it, so I still wouldn't trust it, but it's worth being accurate about how exactly they're messing up.
- mike_hearn 6mo agoEdit: deleted post. I see your other post now. You are right - the "nonce binding" the paper uses doesn't seem convincing. The missing link is that Apple's attestation doesn't bind app generated keys to a designated requirement, which would be required to create a full remote attestation.
- mirashii 6mo ago> If you can prove a public key is generated by the SEP of a machine running with all Apple's security systems enabled, then you can trivially extend that to confidential computing because the macOS security architecture allows apps to block external inspection even by the root user. It only effectively allows this for applications that are in the set of things covered by SIP, but not for any third-party application. There's nothing that will allow you to attest that arbitrary third-party code is running some specific version without being tampered with, you can only attest that the base OS/kernel have not been tampered with. In their specific case, they attempt to patch over that by taking the hash of the binary, but you can simply patch it before it starts. To do this properly requires a TEE to be available to third-party code for attestation. That's not a thing on macOS today.
- egorfine 6mo agoI really want this to succeed
- NiloCK 6mo agoInteresting to see an offering with this heritage [1] proposing flat earnings rates for inference operators here, rather than trying to sell a dynamic marketplace where operators compete on price in real-time. Right now the dashboards show 78 providers online, but someone in-thread here said that they spun one up and got no requests. Surely someone would be willing to beat the posted rate and swallow up the demand? I expect this is a migration target, but a tactical omission from V1 comms both for legitimate legibility reasons (I can sell x for y is easier to parse than 'I can participate in a marketplace') and slightly illegitimate legibility reasons (obscuring likely future price collapse). Still - neat project that I hope does well. [1] Layer Labs, formerly EigenLayer, is company built around a protocol to abstract and recycle economic security guarantees from Ethereum proof of stake.
- Inferlane 6mo ago[dead]
- v9v 6mo agoThey could consider registering as a provider on something like OpenRouter if they aren't getting enough inference requests on their own site.
- smooth968 6mo ago[dead]
- Jn2G3Np8 6mo agoLove the concept, with some similarity to folding@home, though more personal gain. But trying it out it still needs work, I couldn't download a model successfully (and their list of nodes at https://console.darkbloom.dev/providers https://console.darkbloom.dev/providers suggests this is typical). And as a cursory user, it took me some digging to find out that to cash out you need a Solana address (providers > earnings).
- miki123211 6mo ago> Operators cannot observe inference data. Is there some actual cryptography behind this, or just fundamentally-breakable DRM and vibes?
- grvbck 6mo agoBroken calculator or am I missing something here? Macbook Air M2 8GB 12h/day -> $647/month Mac Mini M4 32GB 12h/day -> $290/month I mean, I'd be happy to buy a few used M2 Airs with minimal specs and start printing money but…
- puttycat 6mo ago> Every request is end-to-end encrypted Afaik you will need to decrypt the data the moment it needs to be fed into the model. How do they do this then?
- mr_mitm 6mo agoThe system hosting the model must be one of the ends. Remember, all encryption is E2EE if you're not picky about the ends.
- subpixel 6mo agoWhy isn’t a MacBook Air M5 on the hardware list?
- chakintosh 6mo agono fans
- ianpurton 6mo agoBecause the model that generated that list was trained before the M5 came out.
- heddycrow 6mo agoI think it’s important that systems like this exist, but getting them off the ground is non-trivial. We’ve been building something similar for image/video models for the past few months, and it’s made me think distribution might be the real bottleneck. It’s proving difficult to get enough early usage to reach the point where the system becomes more interesting on its own. Curious how others have approached that bootstrap problem. Thanks in advance.
- haspok 6mo agoHaving strong SETI@Home vibes from 25 years ago, except of course, this is not for the greater good of humanity, but a for-profit project. Problem is, from a technical point of view, what kind of made sense back then (most people running desktops, fans always on, energy saving minimal) is kind of stupid today (even if your laptop has no fan, would you want it to be always generating heat?)... I definitely want my laptops to be cool, quiet and idle most of the time.
- vorticalbox 6mo agoI some times play about with local models via ollama/comfyui and more recently ace-step to generate music. This is short bursts of heat 5-10 m during the render I would not be happy with that for multiple hours a day. I am sure that would have a negative effect on battery health.
- kamranjon 6mo agoMy m4 max mbp with 128gb of memory is constantly training 24/7 on weekends- it’s why I bought the thing.
- Fokamul 6mo agoThanks, if this takse off. I have finally some motivation to do exploitation in kernel. :)
- eigengajesh 6mo agohey guys! i'm the creator. let me know if you have any questions.
- 0xc133 6mo agoHey Gajesh! I sent you an email with some of the teething problems I ran into trying to get started as a provider. Hope it didn't end up in your spam folder!
- daniel_iversen 6mo agoHi! Others have noted this too but you can't seem to buy credits right now, it says "This page couldn’t load" in a custom error page when you've selected the amount and click continue to checkout. Congrats on launching such a cool project that's getting people excited, thinking and discussing :)
- SlavikCA 6mo agoPlease offer new clients try it: at least let us to send few requests in the chat.
- drob518 6mo agoSeems like an interesting way for those people that purchased a Mac Mini to run OpenClaw to pay off the hardware, since mostly it’s now idle.
- bojangleslover 6mo ago[dead]
- bprasanna 6mo agoLike Fold@home but for profit!
- MicBook56 6mo agoI like the idea but it wont take off until Homomorphic Encryption for inference becomes a thing that's efficient and anyone can be a node.
- PiersonMarks 5mo agoThis is what I was confused about like if they own their device they can see what's happening there until we solve this
- alexpotato 6mo agoWasn't there an idea about 15 years ago where you would open your browser, go to a webpage and that page would have a JavaScript based client that would run distributed workloads? I believe the idea was that people could submit big workloads, the server would slice them up and then have the clients download and run a small slice. You as the computer owner would then get some payout. Intersting to see this coming back again.
- thekid314 6mo agoOr SETI which would search for signs of alien life.
- willquack 6mo agoI used to work at Distributive (formerly "Kings Distributed Systems") on its DCP compute platform" which is entirely what you're describing. You can deploy a JS/WASM based workload, and it will be "sliced" and served to browser-based compute nodes. With WebGPU you can sort of have inference executing in the browser too. Incredible people there with an awesome project I added Python execution support via Pyodide (cpython compiled to wasm) and worked on a bunch of other random stuff like WebLLM inferencing during my time there. Apart from Distributive, there's also the "Golem network", "Salad", "Koii" and various other similar projects. --- I'm not sure if I'm convinced by the "Uber for compute" use case with compute buyers and compute workers (sellers), but if you're a university and you have 1000 Windows machines across all your computer labs, it'd be nice to leverage that compute for running research or something idk - especially with the price of ram / cloud offerings these days...
- alexpotato 5mo ago> but if you're a university and you have 1000 Windows machines across all your computer labs, it'd be nice to leverage that compute for running research or something idk - especially with the price of ram / cloud offerings these days... This reminds me of the DevOps guy who made the developer laptops part of a Jenkins "swarm" under the thought that the machines were beefy and underutilized most of the time.
- ripped_britches 6mo agoHow does the inference work correctly if the payloads are encrypted?
- ponyous 6mo agoWhy does M1 Max project significantly higher revenue than M3 Max with double the ram?
- podviaznikov 6mo agoI've tried to install it on my mac, but not sure what macOS version it should support. on 15.1 it failed to serve models. updated to latest 15.5 and it fails to run binary.
- Schiendelman 6mo agoI think macOS has jumped to 26, right?
- jonhohle 6mo ago> That is not a technology problem. It is a marketplace problem. I cringe every time I see this sentence structure. I know the joke is about emdashes, but the “Its not …. It’s ….” drives me crazy.
- nnevatie 6mo agoOnly an em dash missing in between to be chefs-kiss-perfect.
- sergiusignacius 6mo agoI see it everywhere now, it's even in videos and how people talk.
- rustyhancock 6mo agoMaybe it's a Baader-Meinhoff phenomena and even a small difference in frequency feels overwhelming because you can't help but notice when it happens without noticing when it doesn't occur.
- parasubvert 6mo agoIt learned it from humans....
- biztos 6mo agoIt's not a sentence-structure problem. It's an effort problem. But the real solution is to do this other thing. If you'd like I can give you the three-step guide to fixing the thing you asked me to fix. /s
- projektfu 6mo agoIn a little while, AI will reverse the order and everyone will be happy. Tired: That is not a technology problem. It is a marketplace problem. Wired: This is a marketplace problem, not a technology problem.
- rustyhancock 6mo ago
- jaffee 6mo agoclient side of this kind of needs to be open source unless I'm running it on a dedicated machine and firewalling it from the rest of my network. Or the company needs to have a very strong reputation and certifications. curlbash and go is a pretty hard sell for me
- dangoodmanUT 6mo agoThis feels like defi... de-ai
- dgacmu 6mo ago@eigengajesh - Your cost estimator lists Mac Mini M4 Pro with only 24 or 48GB options, but the M4 Pro mini can also be configured with 64GB. At least, I hope so, as I'm typing this on one. ;-) Oh, also, you seem to have some bugs: Gemma: WARN [vllm_mlx] RuntimeError: Failed to load the default metallib. This library is using language version 4.0 which is not supported on this OS. library not found library not found library not found cohere: 2026-04-16T14:25:10.541562Z WARN [stt] File "/Users/dga/.darkbloom/bin/stt_server.py", line 332, in load_model 2026-04-16T14:25:10.541614Z WARN [stt] from mlx_audio.stt.models.cohere_asr import audio as audio_mod 2026-04-16T14:25:10.541643Z WARN [stt] ModuleNotFoundError: No module named 'mlx_audio.stt.models.cohere_asr' Trying to download the flux image models fails with: curl: (56) The requested URL returned error: 404 darkbloom earnings does not work your documentation is inconstent between saying 100% of revenue to providers vs 95% I think .. this needs a little more care and feeding before you open it up widely. :) And maybe lay off the LLM generated text before it gets you in trouble for promising things you're not delivering.
- canarias_mate 6mo ago[flagged]
- sharts 6mo agoToo much to read.
- throwatdem12311 6mo agoActually more useful than Bitcoin. Brilliant idea.
- deleted 6mo ago[deleted]
- dchuk 6mo agoInteresting concept. Two sided marketplaces are hard to bootstrap but maybe just enough curiosity would get the flywheel going. Hell they should just try and convince people to enroll as providers but then also use the service even if it’s hitting their own machines until there’s some degree of supply and demand pressure then try and get only providers to sign up. Or set up some way to encourage providers to promote others to use the service (the 100% rev share kind of breaks that concept but anything can change). I wish this was self hostable, even for a license fee. Many businesses have fleets of Macs, sometimes even in stock as returned equipment from employees. Would allow for a distributed internal inference network, which has appeal for many orgs who value or require privacy.
- deleted 6mo ago[deleted]
- smooth968 6mo ago[dead]
- logicallee 6mo agoIt's a good project that makes sense. I recommend adding a contractual layer as well, since it's free and makes sense. Operators could legally sign that they will not look into the inference layer. After all, the operators already have a financial relationship with this provider, so it makes sense to add a contract to it and keep operators from looking into other people's data that way, too. I wish this project a lot of success.
- frankfrank13 6mo agoThis is one of those ideas I think makes perfect sense, but requires so much operational change for the entire stack, that it would be very difficult to scale: - Convincing labs to run distributed, burst-y inference - Convincing people to run their Mac all day, hoping to make a little profit - Convincing users to trust a distributed network of un-trusted devices I had a similar idea, pre-AI, just for compute in general. But solving even 1 of those 3 (swap AI lab for managed-compute-type-company, eg Supabase, Vercel) is nearly impossible.
- afcool83 5mo ago...or convincing operators that jobs sent to their machines are legal, legitimate, and non-nefarious. I could not find disclosure on their site about the guard-railing or safety-systems at the point the prompt is gathered from users which would intercept, log & prevent bad actors from inadvertently involving me in something illegal or immoral as an operator. Perhaps that disclosure exists and I just need to be linked to it; that would be welcome.
- auslegung 6mo agoHow can one do this safely? If I create a new, non-sudo user, can I install the MDM profile only for that user? I don't understand how this all works obviously so maybe this is a very dumb question
- czk 6mo agothe MDM profile requirement is suspect though I get why they are doing it. but it doesn't inspire confidence to see that their profile is unsigned and still using the default micromdn scep challenge...
- jzig 6mo ago[fix: remove hardcoded API_KEYS ](https://github.com/Layr-Labs/d-inference/pull/39/changes https://github.com/Layr-Labs/d-inference/pull/39/changes)
- creamyhorror 6mo agooh boooy, it's a benchmarking script, but still...
- poorman 6mo agoAs one of the only people running a Mac Studio M3 Ultra with 512 GB of RAM on the network, I can tell you at sustained 100% GPU utilization I am measuring 250 watts max (at the power outlet). My solar panels are easily producing this. The power calculation goes away once you connect a solar panel. You can get a 400 watt solar panel on Amazon for $300.
- qurren 6mo ago> You can get a 400 watt solar panel on Amazon for $300. Too expensive. It's probably producing 200 watts average for 8 hours a day. That's 1600 watt hours, which is about $1.60 at PG&E prices. That would take 187 days to recoup the cost of just the panel. If you include installation costs and "what PG&E steals if you wire it to the same grid" it's probably more like 4x that, which is too long. Tell me when we can have 400 watt solar panels for $50. Stupid capitalism literally forces solar panel prices to make it unprofitable. People should never have to take out loans for solar. Solar should be subsidized and forced by the government to be so cheap that it repays for its cost within a month. Then we're talking. Most things I buy to save money, I expect them to repay within a month. Maybe 2 months max.
- poorman 6mo agoIf you are worried about a $300 solar panel you are not going to like the cost of a Mac Studio M3 Ultra 512 GB! haha
- qurren 6mo agoI'm not "worried" about that cost, I would rather just pay PG&E electricity if the solar panel cost $300. Just pointing out why capitalism + solar is a failure. Capitalism reprices the good thing to be equally expensive to the bad thing, so that nobody buys the good thing anymore.
- dgacmu 6mo agoYou can get them used for that price, and new for $107 if you buy qty 10+. See signature solar as one example. Installation costs and inverters not included, however.
- MyUltiDev 6mo agoThe hardware-attested privacy path is the interesting part of this, but the economic side has a quieter risk the thread has not named: the load tax per request. MiniMax M2.5 239B from your catalog still has to load all 239B weights even though only 11B are active — that is roughly 120GB at Q4_K_M, and cold load from SSD on Apple Silicon is measurable in tens of seconds. Even the Qwen3.5 122B MoE lands around 65GB cold. If the coordinator routes request number two to a different idle Mac than request number one, or if the owner's machine spun the model out to free memory in between, each request pays that cold load before the first token. Keeping the model resident 24/7 solves the latency but eats into the power budget the operator is trying to amortize in the first place. How does the coordinator decide which provider to keep warm for which model? A 16GB or 32GB home Mac cannot host Qwen3.5 122B MoE at all, and the Mac Studios that can are a much smaller slice of the 100M machine estimate.
- dkroy 6mo agoCool idea, though hats off to anyone who got cohere-transcribe to show up as serving the model. I could get device to show up, but kept having issues getting their server to properly serve the model though it could just be the device I tested.
- poorman 6mo agoYeah I think there's a dependency issue going on there. Something isn't installed that needs to be.
- autodidacticon 6mo agobittensor has something to say about this
- AustinDev 6mo agoI'm unable to download FLUX.2 models from `darkbloom models`
- deleted 6mo ago[deleted]
- jdironman 6mo agoReminds me a lot of when I used to deploy this (DataseamGrid) on K12 computers. I was actually just discussing this scenario with a friend. https://www.dataseam.org/research/ https://www.dataseam.org/research/
- zv-io 5mo agoYou're printing everyone's serial numbers publicly. https://console.darkbloom.dev/providers https://console.darkbloom.dev/providers then "Security Verification" for any machine and then "Verify this device independently" -- all of this can be scraped.
- matt-attack 5mo agoAnyone know why not est earnings are shown for Mac Airs? Also they don’t list the M5 air.
- Xx_crazy420_xX 5mo ago"Debugger attachment is blocked. Memory inspection is blocked." - reminds me old crackme challenges. Everything they mention can be bypassed, so determined person can start stealing data from the network. For me this is a killer of such distributed compute ideas, but who knows, maybe the non-enteprise users will be desperate enough for cheap compute to make this idea valid.
- TheHalfDeafChef 5mo agoHad it for 3 days running with nary a request. Perhaps it's the chosen model that I am serving (Gemma 4 26b)? I did see some WARN logs right after startup but the description don't suggest any problems that would block accepting requests or processing them.