5 ms·
I released a new llm-gemini plugin with support for the Gemini 2.0 Flash model, here's how to use that in the terminal: llm install -U llm-gemini llm -
by simonw 2y ago
I released a new llm-gemini plugin with support for the Gemini 2.0 Flash model, here's how to use that in the terminal:
llm install -U llm-gemini
llm -m gemini-2.0-flash-exp 'prompt goes here'
LLM installation: https://llm.datasette.io/en/stable/setup.html https://llm.datasette.io/en/stable/setup.html
Worth noting that the Gemini models have the ability to write and then execute Python code. I tried that like this:
llm -m gemini-2.0-flash-exp -o code_execution 1 \
'write and execute python to generate a 80x40 ascii art fractal'
Here's the result: https://gist.github.com/simonw/0d8225d62e8d87ce843fde471d1432cc https://gist.github.com/simonw/0d8225d62e8d87ce843fde471d143...
It can't make outbound network calls though, so this fails:
llm -m gemini-2.0-flash-exp -o code_execution 1 \
'write python code to retrieve https://simonwillison.net/ and use a regex to extract the title, run that code'
Amusingly Gemini itself doesn't know that it can't make network calls, so it tries several different approaches before giving up: https://gist.github.com/simonw/2ccfdc68290b5ced24e5e0909563c62a https://gist.github.com/simonw/2ccfdc68290b5ced24e5e0909563c...
The new model seems very good at vision:
llm -m gemini-2.0-flash-exp describe -a https://static.simonwillison.net/static/2024/pelicans.jpg
I got back a solid description, see here: https://gist.github.com/simonw/32172b6f8bcf8e55e489f10979f8f085 https://gist.github.com/simonw/32172b6f8bcf8e55e489f10979f8f...
- rafram 2y ago> Some pelicans have white on their heads, suggesting that some of them are older birds. Interesting theory!
- smackay 2y agoBrown Pelican (Pelecanus occidentalis) heads are white in the breeding season. Birds start breeding aged three to five. So technically the statement is correct but I wonder if Gemini didn't get its pelicans and cormorants in a muddle. The mainland European Great Cormorant (Phalacrocorax carbo sinensis) has a head that gets progressively whiter as birds age.
- pcwelder 2y agoCode execution is okay, but soon runs into the problem of missing packages that it can't install. Practically, sandboxing hasn't been super important for me. Running claude with mcp based shell access has been working fine for me, as long as you instruct it to use venv, temporary directory, etc.
- UltraSane 2y agoIs there a guide on how to do that?
- pcwelder 2y agoFor building mcp server? The official docs do a great job https://modelcontextprotocol.io/introduction https://modelcontextprotocol.io/introduction My own mcp server could be an inspiration on Mac. It's based on pexpect to enable repl session and has some tricks to prevent bad commands. https://github.com/rusiaaman/wcgw https://github.com/rusiaaman/wcgw However, I recommend creating one with your own customised prompts and tools for maximum benefit.
- threecheese 2y agoLove the name of your mcp server! What indeed.
- UltraSane 2y agoThanks. I'm a network engineer so I'm going to work on integrating the output of show commands to Claude using MCP.
- stavros 2y agoI wrote a program that can do more or less the same thing, if you only care about the LLM running commands to help you do something: https://github.com/skorokithakis/sysaidmin https://github.com/skorokithakis/sysaidmin
- mnky9800n 2y agoCan it run ipython? Then you could use ipython magic to pip install things: https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-pip https://ipython.readthedocs.io/en/stable/interactive/magics....
- simonw 2y agoPublished some more detailed notes on my explorations of Gemini 2.0 here https://simonwillison.net/2024/Dec/11/gemini-2/ https://simonwillison.net/2024/Dec/11/gemini-2/
- bravura 2y agoQuestion: Have you tried using this for video? Alternately, if I wanted to pipe a bunch of screencaps into it and get one grand response, how would I do that? e.g. "Does the user perform a thumbs up gesture in any of these stills?" [edit: also, do you know the vision pricing? I couldn't find it easily]
- simonw 2y agoPrevious Gemini models worked really well for video, and this one can even handle steaming video: https://simonwillison.net/2024/Dec/11/gemini-2/#the-streaming-api-is-next-level https://simonwillison.net/2024/Dec/11/gemini-2/#the-streamin...
- bravura 2y agoWow this is amazing. It just gave me critique on my bodyweight squat form. But I also found it hard to prompt to tutor in French or Portuguese; the accents were gruesomely bad.
- famouswaffles 2y agoFor some reason, the realtime api is using TTS for speech output. Not sure if that's temporary
- andy_ppp 2y agoI can't wait to see these models are able to do video <-> video, just so that I can be in ten standup meetings at once :^)
- lngnmn2 2y ago[dead]