11 ms·
Anthropic Claude 3.5 can create icalendar files, so I did this
- LarsDu88 2y agoThis is why I love HackerNews. Life hacks without the endless meme spam found in reddit.
- wenc 2y agoYou just have to double check the results whenever you tell Claude to extract lists and data. 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. But LLMs can get things ever slightly wrong when it comes to long lists/tables. I've been bitten by this before. Trust but verify. (edit: if the answer is "machine verifiable", one approach to ask an LLM to write a Python validator which it can execute internally. ChatGPT can execute code. I believe Sonnet 3.5 can too, but I haven't tried.)
- thelittleone 2y agoCan solve that to have a similar or lower error probability as a human by running the results through a verification agent.
- jph00 2y agoSonnet on Claude.ai can not execute code, although it often pretends otherwise.
- disillusioned 2y agoThe "off by one" predilection of LLMs is going to lead to this massive erosion of trust in whatever "Truth" is supposed to be, and it's terrifying and going to make for a bumpy couple of years. (Or the complete collapse of objective knowledge, on a long enough time horizon.) It's one thing to ask an LLM when George Washington was born, and have it return "May 20, 2020." It's another thing to ask it, and have it matter-of-factly hallucinate "February 20, 1733." At first glance, that... sounds right, right? President's Day is in February, and has something to do with his birthday? And that year seems to check out? Good enough! But it's not right. And it's the confidence and bravado with which LLMs report these "facts" that's terrifying. It just misstates information, calculations, and detail work, because the stochastic model compelled it to, and there wasn't sufficient checks in place to confirm or validate the information. Trust but verify is one of those things that's so paradoxical and cyclical: if I have to confirm every fact ChatGPT gives me with... what I hope is a higher source of truth like Wikipedia, before it's overrun with LLM outputs... then why don't I just start there? If I have to build a validator in Python to verify the output then... why not just start there? We're going to see some major issues crop up from this sort of insidious error, but the hard part about off-by-ones is that they're remarkably difficult to detect, and so what will happen is data will slowly corrupt and take us further and further off course, and we won't notice until it's too late. We should be so lucky that all of LLMs' garbage outputs look like glue on pizza recommendations, but the reality is, it'll be a slow, seeping poisoning of the well, and when this inaccurate output starts sneaking into parts of our lives that really matter... we're probably well and truly fucked.
- stavros 2y agoThis is semi-offtopic, but "trust but verify" is an oxymoron. Trusting something means I don't have to verify whether it's correct (I trust that it is), so the saying, in the end, is "don't verify but verify".
- cromka 2y agoA better phrase would be “Use it but verify”, simply.
- stavros 2y agoYes, which boils down to "verify".
- EGreg 2y agoI always wondered about that
- bilal9684 2y ago[flagged]
- cj 2y agoIt's possible to trust (or have faith) in my car being able to drive another 50k miles without breaking down. But if I bring it to a mechanic to have the car inspected just in case, does that mean I never had trust/faith in the car to begin with? "I trust my coworkers write good code, but I verify with code reviews" -- doing code reviews doesn't mean you don't trust your coworker. Yet another way to look at it: people can say things they believe to be true but are actually false (which isn't lying). When that happens, you can successfully trust someone in the sense that they're not lying to you, but the absence of a lie doesn't guarantee a truth, so verifying what you trust to be true doesn't invalidate your trust.
- 2y ago
- nabla9 2y agoThey are just human.
- IshKebab 2y agoThey aren't the same as humans. They definitely work differently. Also they've been trained to say something as plausible as possible. If it happens to be true, then that's great because it's extra plausible. If it's not true, no big deal. While I have worked with one awful human in the past who was like that, most thankfully aren't!
- mmahemoff 2y agoYes, probably better to get the LLM to write the script. Example, I was trying out two podcast apps and wanted to get a diff of the feeds I had subscribed to. I initially asked the LLM to compare the two OPML files but it got the results wrong. I could have spent the next 30 minutes prompt engineering and manually verifying results, but instead I asked it to write a script to compare two LLMs, which turned out fine. It's fairly easy to inspect a script and be confident it's _probably_ accurate compared to the tedious process of checking a complex output.
- sorokod 2y agoIsn't asking an LLM "to write a Python validator" suffers from the 99.9% (or whatever the error rate for validators written by Claude) problem?
- bonzini 2y agoThe difference is that you're asking it to perform one intellectual task (write a program) instead of 100 menial tasks (parse a file). To the LLM the two are the same level of complexity, so performing less work means less possibility of error. Also, the LLM is more likely to fail spectacularly by hallucinating APIs when writing a script, and more likely to fail subtly on parsing tasks.
- dbaupp 2y agoIn addition to what you say, it can also be easier for a (appropriately-skilled) human to verify a small program than to verify voluminous parsing output, plus, as you say, there's the semi-automated "verification" of a very-wrong program failing to execute.
- jessekv 2y agoAll tests have this problem. We still write them for the same reasons we do double-entry bookkeeping.
- Foobar8568 2y agoWell for my case, I have 0 trust in chatgpt ( or local LLM) to extract properly data from a PDF file, especially if it's over a few pages.
- dan-robertson 2y agoIf it’s correct 99.9% of the time, and the piano lessons are every two weeks, that’s one error in piano lesson scheduling over 40 years. That sounds good enough to me to not verify.
- MattGaiser 2y agoHave two LLMs do the task and compare.
- campers 2y agoOr three or four or five! https://openreview.net/pdf?id=zj7YuTE4t8 https://openreview.net/pdf?id=zj7YuTE4t8
- surfingdino 2y agoGoogle's Gemini dev docs contain a few of such warnings. For a good reason, these models make stuff up and the domain where ROI is positive is small.
- websap 2y agoIt would be interesting to see how well does a human do it? Are they correct more than 99.9% of the time?
- huijzer 2y agoI completely agree. If correctness matters, then it’s probably better to use LLMs to write the code than to let LLMs be the code.
- agumonkey 2y agoCan Claude tdd itself ? Lean-Claude
- wanderingmind 2y agoKind of a noob question, is it possible to design a GAN type network with LLM, where one (or many) LLMs generate outputs, while a few other LLMs validate or discriminate them and thus improving generator LLMs accuracy.
- oneshtein 2y agoYes, you can use AI to spot errors in AI output. Done before with good results, but it requires to run 2 different, but equally good, AI models in parallel, which is way more expensive than 1 model.
- ada1981 2y ago"which is way more expensive than 1 model." In this case "way more" means exactly 2x the cost.
- onlyrealcuzzo 2y ago> 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. That's far better than I would do on my own. I doubt I'd even be 99% accurate. If it's really 99.9% accurate for something like this - I'd gladly take it.
- croes 2y agoThe problem is that people could be impressed and use it for things where 0.01% could lead to people getting hurt or even get killed.
- onlyrealcuzzo 2y agoYeah, but that's probably not your own calendar reminders...
- taberiand 2y agoI call that problem "Doctor, doctor! It hurts when I do this!" If the risk exists with AI processing this kind of data, it exists with a human processing the data. The fail-safe processes in place for the human output need to be used for the AI output too, obviously - using the AI speeds up the initial process enormously though
- croes 2y agoThe problem is liability. Who is liable if the AI makes errors?
- deleted 2y ago[deleted]
- thegabriele 2y agoIt depends on the richness of the AI
- troupo 2y ago> You just have to double check the results whenever you tell Claude to extract lists and data. There's also the problem that they are tuned to be overly helpful. I tried a similar thing described in the OG article with some non-English data. I could not stop Claude from "helpfully" translating chunks of data into English. "include description from the image" would cause it to translate it, and "include description from the image, do not translate or summarize it" would cause it to just skip it.
- bhl 2y agoI wonder if tool-calling to output schema'd json would have a low error rate here. For each field, you could have a description of what is approximately right, and that should anchor the output better than a one-off prompt.
- stocknoob 2y agoHave N separate chats, and take the result that they all agree on. N=1 is like RAID0.
- sussmannbaka 2y ago> Trust but verify. This is why I don’t bother with GenAI. It’s a lot harder to verify data from a machine trained to generate plausible looking data (that happens to be correct most of the time, sure) than from real data. Ofc this is getting harder as “real data” gets drowned in GenAI output, which drives the animosity against GenAI in my circle.
- damnever 2y agoVerification comes with an additional cost. This makes me wonder: if we can verify, why use a LLM at all? An LLM might even cost more.
- pplante 2y agoI just did this with my kids various school calendars all of which are locked up in gross PDF files. Claude only made one tiny mistake which it was able to correct when I asked it why it made the mistake. It actually pointed out the confusing formatting used by the author. I tried to do the same with ChatGPT a few weeks ago and was unable to get very far.
- zacharyozer 2y agoI did this last year with my kids various activities using ChatGPT and the results were just OK. I'm looking forward to trying this out with Claude this year!
- pvsnp 2y agoFunny, I just learned this this week too and had it generate an ics file from an excel file. Then I realized there might be bugs, so I asked it to write program instead.
- buro9 2y agoI'm finding Claude to hallucinate less than ChatGPT, and to be far more accurate at coding than CoPilot. Pleasantly surprised on both counts. Example hallucinations from ChatGPT include researching the dates of historical events for the company I work at, trivially verifiable by me but I was being lazy... ChatGPT told me about blog posts that never existed and I could prove never existed, Claude was spot on with dates and source links (but only appeared to have data through to the end of last year). On coding, CoPilot came up with reasonable suggestions but Claude was able to take a file as an input and match the style of the code within the repo. Claude, for me, is starting to hint at what a highly productive assistant can achieve.
- romeros 2y agoFor some reason I felt Claude was Yahoo to ChatGPT's Google in the initial days. It just felt like ChatGPT had an insurmountable moat. I played with Claude and it is just insanely superior compared to ChatGpt as of now. Someone on HN commented the same thing a few months back but I did not take it very seriously. But, Claude is just superior to Chatgpt.
- zoover2020 2y agoI've felt this for a long time, been using Claude for over a year now internally - super impressed w/ it from the start and the 3.5 model is blazingly fast. Which version of C-GPT did you compare it to?
- wenc 2y agoI pay for both Claude and ChatGPT. Claude 3.5 is now my daily driver but it still refuses to answer certain questions especially about geopolitics so I still have to go back to ChatGPT. I don’t agree that Claude is insanely superior to ChatGPT though. It still has trouble with LaTeX and optimization formulations. Its coding abilities are good but sometimes ChatGPT does better. This is why I keep both subscriptions. At $40/math I get the best of both worlds.
- BoorishBears 2y ago
- dukeofdoom 2y agoYou can also ask AI to make a markdown check list. Useful for ToDo's you can paste into markdown editors.
- low_tech_love 2y agoDoes that mean we’re getting the functional PDAs that were promised to us for decades by the sci-fi? That’s be amazing.
- wannabag 2y agoI had the exact same use case two weeks ago but I had received a pdf file from school and was sitting at a cafe with only my phone. I use ChatGPT, and while the article is correct to say that it will claim that it cannot generate .ics files directly in the code interpreter it is however very much capable of solving this particular problem. I did the following (all on my android phone): 1. Had it extract all the useful dates, times and comments from the pdf 2. Prompted it to generate the ics file formatted content as code output 3. Prompted it to use the code interpreter to put this content into a file and save it as a .ics extension It complied through and through and I could download and open the file with the gcal app on my phone to import all appointments.. For completeness, the claim that the code interpreter cannot "generate" ics files is because the python environment in which it runs doesn't have a specific library for doing so. ICS files are just text files with a specific format, so definitely not out of reach.
- gw5815 2y agoYeah, Claude.ai just gave me the text that I simply saved as a my.ics file. I wonder if I could reword the prompt to get chatgpt to do it.
- wannabag 2y agoYes, just rewording is enough in this case. It will fail if it tries to be clever with the code interpreter as mentioned. I used: "Extract the important dates and times from this pdf in Swedish" [..] "Great now generate the content of an ics formatted file, just the text, with the schedule above" [..] "Now use the code interpreter to create a file with the above content then save the file as an .ics extension"
- weissi 2y agoInteresting, maybe ChatGPT got more reluctant to spit out ics? Didn't have to try very hard back in May: https://chatgpt.com/share/0848349d-4b0b-40f8-9d24-e9c4ffc065c0 https://chatgpt.com/share/0848349d-4b0b-40f8-9d24-e9c4ffc065...
- samstave 2y ago
- cdrini 2y agoThis is phenomenal! And also highlights a design pattern which I'm hoping becomes even more popular because of AI: interoperable human readable file formats! Things like .ICS files. I'd love to see more websites/apps prioritise supporting these sorts of things, since it make it really easy to plug/play and use AI, but also just empowers more people to get creative/build extensions. There's been a bit of a trend to more silo'd data, and a deemphasis on files entirely in the last ~decade in the name of "simplicity". I'd love to see that backtrack a bit.
- pietz 2y agoReally cool! It's also really good at creating all kinds of mermaid diagrams. I have a prompt to create learning resources based on YouTube transcripts including visualizations in mermaid. It's awesome.
- solaarphunk 2y agoCan also be used to generate Google calendar links!
- sen 2y agoI did a similar thing with ChatGPT-4o giving it a txt file list of movie names, and it returned the release date of each one, then gave me an iCal file of all those movies with their release dates as recurring yearly anniversary events so I can watch my favorite movies on their anniversary. I’ve done it for a few friends as well now and it’s got a 100% success rate so far, across over 100 total movie names.
- stavros 2y agoDid you go back and check all the dates?
- sen 2y agoYup, we tried scraping IMDB for dates too to compare which method was easiest/best and it was spot on (and easier!). I guess the release dates are mentioned enough around the web that it has enough reference. It does use the US release dates for everything though which was one issue, but that’s almost always the first release for major movies anyway.
- stavros 2y agoOh nice! Though I'd probably ask it to write some parsing code for the page instead, that way I'm more sure that the results are OK...
- orhmeh09 2y agoThat is a really fun idea, I love it.
- dotancohen 2y agoTo bad that wouldn't work for good music albums. For many years (still, maybe?) albums would come out on Tuesdays. So one day of the week you'd have them all bunched up together. That's how I remember that September 11, 2001 was a Tuesday. Album day, and there was a good one that day, too.
- 2y ago
- yakorevivan 2y agoI have myself been using chatgpt/sonnet-3.5 to clean data, extract data, heck, even generate sample sql insert statements for a given table schema... small things, but, when done repeatedly, saves soooo damn amount of time and frustrations. I have been using these tools to generate sooo many small scripts to automate or do things that otherwise I either wouldn't have done it, or, it would have taken significant amount of time. These tools are tooo good now to not use them. Also, as someone else have already pointed out, these things work correctly 99.99% of the time. But that remaining 0.01%... that's what becomes major issue since it is so small of an error, that, unless you verify, you'll end up missing. When using LLM's..."Trust, but verify".
- durdn 2y agoI've been using ChatGPT for similar visual recognition things. Recently I took a video of a car because I really liked its color. I upladed the video to ChatGPT and asked to extract which paint color I'd need to specify to a modding garage to put a foil on my car. ChatGPT really impressed me, extracted a screenshot, did a color analysis of the paint, found the palettes from paint vendors and found for me the exact paint code to tell the garage. I was speechless.
- yas_hmaheshwari 2y agoWow! Simply wow! I had a similar requirement a few days back, but I stopped at Google lens. TIL
- jlarks32 2y agoYeah for what it's worth, ChatGPT with custom instructions can definitely do this! This was something that I did pretty early on: https://chatgpt.com/g/g-6mfNb9Hys-calendar-creator https://chatgpt.com/g/g-6mfNb9Hys-calendar-creator
- imarkphillips 2y agoThis is quite useful. Apartment renters can now create their own ical feeds for booking platforms.
- cromka 2y agoThat’s been implemented for ages, hasn’t it? I had my imported Airbnb calendar always up to date.
- aussieguy1234 2y agoI used a screenshot of my internet banking transaction list with GPT4-o to get a list of transactions for my company tax return. Visually verified the results and they were correct. Earlier, a product manager sent me a list of company id's to turn on a feature flag for as a screenshot. Rather than enter them all in manually, I used that to get GPT4-o to generate a comma separated list from the screenshot. Again, worked perfectly.
- layer8 2y agoOpenAI must be happy to get all that confidential information. ;)
- aussieguy1234 2y agoI used their paid API and the open-webui frontend. They say that information sent through their API is not used for training, unlike the official free chatgpt interface. As a paid service however it's been insanely cheap compared to the time it's saved me.
- layer8 2y agoThey might use it for more worrying purposes than training.
- jillesvangurp 2y agoI've been doing similar things with gpt 4o to simplify data entry. Right now, it's kind of useful to use but people are not systematically doing much yet with LLMs. I think that's going to change because it's so obviously useful to do. Any work involving entering data into some form is something that can and will be automated now. Especially if you have the information in some printed or printable way. Just point your camera at the thing and quickly review the information. Sucking up information that is out there on signs, posters, etc. can be amazingly useful. People put a lot of effort and money into communicating a lot of information visually that often does not exist in easily accessible digital form.
- visarga 2y agoIn my tests, if you operate over diverse forms or documents the information extraction rate is around 90%. It's especially hard for complex forms.
- jillesvangurp 2y agoThat's pretty good already for a model that isn't specialized or finetuned. I'm sure things will get better.
- flmaki 2y agohmm
- asimovfan 2y agoIve been doing this with chatgpt.. sometimes makes mistakes, i have to try a few times but better than by hand..
- hganesan 2y agoAI for calendar invites is a real winner, I did a related thing here, but ical is even simpler! https://hareeshganesan.com/2024/07/14/baby-calendar https://hareeshganesan.com/2024/07/14/baby-calendar
- albert_e 2y agoMany folks like schools, employers, etc still publish calendars in PDF format. They spend more time on branding and visual formatting, than trying to create the same in formats that we can import into our calendar apps and actually use it practically. I wonder if there is a two step process we can follow that will more robustly generalize ... 1. Read any document and convert into into a simple table that tries to tabulate: date, time (+timezone), venue, url, notes, recurrence 2. Read a table that roughly has the above structure and use that to create google/ical/ics files or links We may be able to fine tune a model or agent separately to do each step very well.
- marbletiles 2y agoYou badly overestimate the technical chops of the parent-guardian cohort and underestimate the practicality of a printable PDF with key dates.
- albert_e 2y agoThis is not either/or. We can have both. Any organization in 2024 creating a calendar that impacts 500-1000 people should have the awareness that people use digital calendars / apps too. They should make an effort to publish both PDF and ical. Since that has yet to catch on in many places -- including the HR departments of large enterprises that publish holiday calendars and the like -- these alternatives ideas are explored. Once we have the right tools ... It just takes one tech-savvy parent to do this 10 minute exercise and publish an online calendar that other busy parents can subscribe to.
- bronco21016 2y agoAgreed. If my child’s elementary school can manage, then most larger organized activities should be able to manage as well. I’m thinking along the lines of like organized rec athletic leagues, organized after school activities. Etc. I do find that most of these do provide some method of getting a calendar link but we’re not quite to 100% yet.
- blsummer 2y agoI tried the prompts in Gemini 1.5 Pro, it works too. Use `List all the dates` or LLM will be lazy by listing only 3 results
- dkga 2y agoThat’s a great step forward. I use a combination of the Reminders and Calendar apps to mark entries of academic conferences I participate. One interesting thing for me personally would be to find a way to read the appointments in the Reminders app (presumably a sqlite3 db somewhere in iCloud) and create corresponding dates in the calendar, such that whenever I add a reminder to submit papers to a conference, this would automagically block that calendar slot for the dates of the conference such that I know not to schedule in person meetings with anyone in those dates.
- franze 2y ago> I tried the exact same thing in ChatGPT -- it identified the dates in the images, but it was unable to create an .ics file. However, it did give me some Python code to create the ics file for me. well i constantly let ChatGPT create my calendar entries, either via Google Calendar links or ics files or QR codes. Don't know what went wrong there.
- TeMPOraL 2y agoBack when I last did it with GPT-4-Turbo, asking for an ICS file would get it to refuse on the grounds that it cannot make a file. Asking for encoding in iCal fomat, however, would make it output into a Markdown code block, which I'd then copy-paste into Notepad and save as ICS. These days, you may also try asking it to "use Python" for the purpose, in which case it will make you the file (by writing code that writes the file and executing it), but it's likely to make more mistakes because it has to encode the result into a Python string or array.
- 1oooqooq 2y agoHN devolved into a cheap product review. this forum now is like reddit being taken by nothing but fake accounts hyping one mobile phone or game console or movie. so tiring.
- fmbb 2y agoMaybe useful in some cases, but this calendar has a heading saying the lessons are "every other week" so you don't have to make 13 calendar entries. It takes a few seconds but you can make a calendar entry on September 13, tell your calendar app that it repeats every two weeks, and then you delete the six "vacation" slots. This is much faster.
- wodenokoto 2y agoAnd this is where Siri failed in my opinion. Even things like "Create an event that starts on Monday and continues for 90 days" is and always has been a crapshoot. All the things you didn't want to do manually yourself, Siri couldn't do either. Not much of an assistant then.
- greenthrow 2y agoSo let's see all the power and copyright violations that are required to build an LLM and it can be used to replace a ~20 line bash script. Awesome guys.
- ndr_ 2y agoI use this trick for book announcements on Amazon: some ambitious book releases never get released, so I am not a fan of buying before the release. With LLM support, I'll add the release date given by Amazon to my calendar - quickly. The file download feature of my Workbenches helps with that: https://ndurner.github.io/chatbots-update https://ndurner.github.io/chatbots-update
- losvedir 2y agoHey, I did a similar thing last week with my preschooler's paper schedule they handed out, using GPT-4o. But I asked it to output in a certain CSV format, which Google Calendar can import. Easier to review, fewer tokens.
- millzlane 2y agoThis was my first use of ChatGPT maybe 1.0 I don't really remember but it was early last year sometime. I just started college for the first time and had a list of due dates for assignments. I asked chatGPT "Can you make me an ics file from this list of dates" It did it instantly and told me to save the file in a text editor as an .ics file. But it messed up and put the assignments and discuissions on the same week. I ask "Do the same thing but move the discussion questions to the following tuesday, and put the assignment details in the details of the meeting. It spit out a perfect .ics file.
- hereme888 2y agoMonths ago I did this with ChatGPT. It was unable to create the .ics directly at first, but at least it wrote the python script that created the .ics. I first ask the LLM to write out the calendar events in a markdown table to double-check it read everything correctly, and then create the file. A different prompting strategy later on, ai think with example .ics formatting, resulted in a correct .ics. I just had to *emphasize* the time zone.
- shade 2y agoI did something similar recently with my daughter's school calendar and ChatGPT with gpt-4o; her school has a ton of closures/teacher work days, I fed the PDF of the calendar in and asked it for all the dates that impacted 2nd grade, then asked it to create an icalendar file for them. Oddly, it didn't want to create the actual file, but gave me a Python script for doing so; you just need to be sure to tell it what time zone you are working in and that you want it to be an all-day event, or you'll get suboptimal results.
- CraigJPerry 2y agoWhen I long press on any date in the grainy image listing the dates, iOS OCRs the text, interprets it as a date and because I long-pressed, it offers a popup context menu to create an appointment or a reminder, while showing a day view of my calendar for that date.
- bmarswalker 2y agoI tried this in ChatGPT 3.5 times and it repeatedly said "I am a language model and can't do that". Good to know it can do it now.
- htadashi 2y agoI made a Chrome extension that does this for textual input: https://github.com/htadashi/AI-Event-Scheduler https://github.com/htadashi/AI-Event-Scheduler