6 ms·
This post is about a talk I gave at a data journalism conference, but it's also a demo of a whole bunch of projects I've been working on over the past couple of
by simonw 3y ago
This post is about a talk I gave at a data journalism conference, but it's also a demo of a whole bunch of projects I've been working on over the past couple of months/years:
- Claude 3 Haiku generation via phone or laptop camera: https://tools.simonwillison.net/haiku https://tools.simonwillison.net/haiku
- A new Datasette plugin for creating tables by pasting in CSV/TSV/JSON data: https://github.com/datasette/datasette-import https://github.com/datasette/datasette-import
- A plugin that lets you ask a question in English and have that converted into a SQL query (also using Claude 3 Haiku): https://github.com/datasette/datasette-query-assistant https://github.com/datasette/datasette-query-assistant
- shot-scraper for scraping web pages from the command-line: https://shot-scraper.datasette.io/en/stable/javascript.html https://shot-scraper.datasette.io/en/stable/javascript.html
- Datasette Enrichments for applying bulk operations to data in a SQLite database table: https://enrichments.datasette.io/en/stable/ https://enrichments.datasette.io/en/stable/ - demonstrating both a geocoder enrichment https://github.com/datasette/datasette-enrichments-opencage https://github.com/datasette/datasette-enrichments-opencage and a GPT-powered one: https://github.com/datasette/datasette-enrichments-gpt https://github.com/datasette/datasette-enrichments-gpt
- The LLM command-line tool: https://llm.datasette.io/ https://llm.datasette.io/ - including a preview of the image support, currently in a branch
- The datasette-extract plugin for extracting structured data into tables from unstructured text and images: https://www.datasette.cloud/blog/2024/datasette-extract/ https://www.datasette.cloud/blog/2024/datasette-extract/
- The new datasette-embeddings plugin for calculating and storing embeddings for table content and executing semantic search queries against them: https://github.com/datasette/datasette-embeddings https://github.com/datasette/datasette-embeddings
- Datasette Scribe by Alex Garcia: a tool for transcribing YouTube video audio into a database, diarizing speakers and making the whole thing searchable: https://simonwillison.net/2024/Apr/17/ai-for-data-journalism/#datasette-scribe https://simonwillison.net/2024/Apr/17/ai-for-data-journalism...
- doctorpangloss 3y agoBut Simon, you are a strongly opinionated guy whose opinions are interesting. Why don't you make a data journalism piece? Why don't you submit an article to the NYTimes, WSJ, etc.? They will publish it! What are you waiting for?
- simonw 3y agoSounds like a lot of work! I have been thinking that I need to do some actual reporting myself at some point though, ultimate version of dogfooding for all of my stuff.