17 ms·

Xkcd 1425 (Tasks) turns ten years old today

1003 points by ulrischa 2y ago

cung 2y ago
It took a bit over five years, but now checking if it’s a photo of a bird is the easier task.
- consp 2y ago
  At what confidence level are we talking about? With these over simplified questions (as in the xkcd) my guess would be the asker assumes 100%.
  - shiroiushi 2y ago
    You're not going to get 100% confidence with either problem. The GPS one might be easier to get high confidence with, but even here you have to worry about 1) the accuracy of the GPS coordinates from your camera/phone, which isn't that good, and 2) calculating the exact boundaries of the park from the public data. So you could probably calculate with nearly 100% confidence that you're, for example, within 5km of a park, but if you take the photo from a location close to the park's boundary, the confidence will go way down. If you're a meter or two from the boundary, forget it.
    - vinibrito 2y ago
      How could the gps position and the park boundary not be exact? Phones GPS give a 2 meters accuracy, and a park boundary is a well defined hard line polygon. Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that. Asking because I'm afraid I may be missing something here due to this being something I already worked on.
      - shiroiushi 2y ago
        >Phones GPS give a 2 meters accuracy Well I already pointed out that if you're within a couple meters of the boundary, you won't have good confidence because of this fact. >and a park boundary is a well defined hard line polygon. Is it? I'm no expert on parks, but surely some of them have borders along rivers. Many US states have such borders. >Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that. That doesn't account for the 2m accuracy. What if I'm standing exactly 1m from the boundary when I take the photo? You have no idea if I'm really in the park or not from the GPS data. I also have serious doubts about your 2m accuracy claim, based on personal experience. Maybe if you're standing in a wide-open desert with nothing around you, but anywhere else, the accuracy isn't that great, especially around buildings. GPS accuracy is terrible in cities.
        ddoeth 2y ago
        > Is it? I'm no expert on parks, but surely some of them have borders along rivers. Many US states have such borders. Depending on the country, but Australia has some [1]. I still think that there is a set of polygons that can be used to describe this border. Not to argue against your point (I rarely get less than 4m of accuracy), but luckily > but anywhere else, the accuracy isn't that great, especially around buildings. GPS accuracy is terrible in cities. cities are (almost?) never in national parks. [1]: https://www.nationalparks.nsw.gov.au/-/media/npws/maps/pdfs/parks/border-ranges-national-park/border-ranges-national-park-map.pdf https://www.nationalparks.nsw.gov.au/-/media/npws/maps/pdfs/...
        shiroiushi 2y ago
        >cities are (almost?) never in national parks. Sometimes they are. See Washington, DC. Anyway, the requirement is for determining if a photo was taken in a park or not. The resolution wasn't stated, however: just how accurate do we need to be? If I'm in a canoe in a river that borders a park, but the river isn't part of the park, but the shoreline a few meters away is, our algorithm might claim I'm in the park, when I'm really not. The requirement wasn't "somewhere near a park", but "in a park". Rivers change their courses over time, so some polygons aren't going to accurately describe this border.
      - emj 2y ago
        If you need to know 100% that the bird is in the park at that precise moment it can be tricky. If you need to identify a Bird-of-prey in the Alpha quadrant you can understand the Klingon proverb a sharp knife is nothing without a sharp eye.
      - deleted 2y ago
        [deleted]
    - vincnetas 2y ago
      What does it even mean "YOU are in the park". What is YOU? If you standing on the boundary, are YOU in or out? Details mater :D
      - shiroiushi 2y ago
        Exactly! What if you're standing on the boundary, with one foot inside and the other outside? These specifications are far too vague.
- marricks 2y ago
  For a person to set up but definitely not how many cpu cycles are burned
- qwertox 2y ago
  Is it? I assume that you are thinking of using a 3rd-party API endpoint to which you upload the image so that the service decides for you if it is a bird and which kind of bird it is. Or you use something like Firebase. Because if that is the way you'd solve this problem, then just sending lat/lon to a service to determine if it is in a national park is even easier, as it's just a GET request. I'm still unsure about what would be harder to set up locally.
  - llm_trw 2y ago
    https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/ https://tech.amikelive.com/node-718/what-object-categories-l... Bird is in the default COCO dataset. I haven't look for birds in images, but for people yolov10 is fewer lines of code to detect if there is someone in the frame than it is to setup a flask server for the API calls.
  - a_t48 2y ago
    Using a service for this would kind of suck if you're _in_ the national park, due to cell service. The ML portion of this is probably still a bit harder than the GPS bit. ML: Grab yolov8 (Optional?) Fine tune it on bird pictures Convert it to CoreML, do whatever iOS stuff is needed in XCode to run it GPS: Get https://www.nps.gov/lib/npmap.js/4.0.0/examples/data/national-parks.geojson (TIL this exists! thanks federal govt!) Stuff it into the app somehow Get the coordinates from the OS Use your favorite library for point+polygon intersection to decide if you're in a national park Bonus: use distance from polygon instead to account for GPS inaccuracy, keeping in mind lat and long have different scales. ...actually the ML one might be easier, nowadays. Now I kind of want to try this.
    - xxs 2y ago
      > Use your favorite library for point+polygon intersection to decide if you're in a national park it's like 50ish lines of code in C, just iterating over the points (with the polygon represented by arrays of points). The algorithm is linear with regards to the points.
- munchler 2y ago
  Your phone already does both automatically, so I’d call it a draw.
- unsigner 2y ago
  "easier tasks" is arguable and arguably wrong "task about which you will find more easy-looking tutorials hiding the complexity under a blanket of 3rd party code and services" is better
  - YurgenJurgensen 2y ago
    You just described all computational tasks that don’t open with ‘first, dig up some sand and some copper ore’.
    - aiaiaiaiaiai 2y ago
      Except all the stuff millions of software engineers work on for billions of hours per year.
    - unsigner 2y ago
      checking whether coordinates fall inside a national park is an exercise in computational geometry, surveying (the earth is not flat, what are coordinates on a sphere), databases, access to government data. detecting birds is an exercise in gathering properly labeled training sets, neural networks and their topologies, matrix multiplication performance and/or orchestration of rented GPUs. both of which cover interesting tasks, worthy things to learn, and are by no means easy. "easy" is the bird recognition where you do an API call to a totem pole of third party services.
- thaumasiotes 2y ago
  > but now checking if it’s a photo of a bird is the easier task. That depends on whether you care about getting the answer right. If you don't, it was always the easier task. If you do, Seek by iNaturalist still can't do this job, and that's the only thing Seek is supposed to be able to do.
ml_basics 2y ago
It's quite remarkable how much the goal posts have shifted when it comes to what is impressive with AI/ML. Things like this are a good reminder. 10 years ago the GAN paper came out and everyone was excited how amazing the generated image quality was (https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/1406.2661) The amount of progress we've made is mind boggling.
- fouronnes3 2y ago
  Arguably the goal post for AGI has moved about as much, if not more. One wonders if Turing reading a 2024 LLM chat transcript would say "but it's not really thinking!".
  - audunw 2y ago
    Passing the Turing test has always been a non-binary thing. Chat bots have been able to pass off as a human for a short time under certain circumstances. Now they can pass off as human for a longer time under more circumstances. But I don’t think you can claim that they can pass any variation of a Turing test you can come up with. Has the AGI goal post been shifted? Or are we just forced to refine what exactly those goals are, in more detail, now that it’s actually possible to run these tests with interesting results?
    - sigmoid10 2y ago
      But isn't this exactly the goalpost moving the other comment claimed? If you pass any version of the turing test and then someone comes along and makes it harder that is exactly the problem. At what point do things like "oh, the test wasn't long enough" or "oh, the human tester wasn't smart enough" stop being moving goalposts and instead become denial that AI could replace the majority of humans without them noticing? Because that's where we're headed and it's also where the real danger is. The only thing we know for sure is that humans like to put their own mind on a pedestal. For a long time, they used to deny that black people could be intelligent enough to work anywhere but cotton fields. In the same way they used to deny that women could be smart enough to vote. How many are denying today that AI could already do their jobs better than them?
weinzierl 2y ago
One could say, "This didn't age well." but I think the real point of "it can be hard to explain the difference between the easy and the virtually impossible" is only reinforced by an almost ironic twist that switched the hard and easy around. Who would have thought ten yeas ago?
- 0x006A 2y ago
  Its worth looking at the alt text of the xkcd itself: In the 60s, Marvin Minsky assigned a couple of undergrads to spend the summer programming a computer to use a camera to identify objects in a scene. He figured they'd have the problem solved by the end of the summer. Half a century later, we're still working on it.
  - bonoboTP 2y ago
    It's a cute anecdote but of course reality is more boring. The assignment wasn't to solve human level vision in a summer, just to do quite constrained object recognition. But even that turned out to be quite hard. But Minksy wasn't that fully miscalibrated.
    - acdha 2y ago
      I think it’s worth reading the original assignment: https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100.pdf?sequence=2&isAllowed=y https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100.... If you want to argue that “likely objects” is weaker than Munroe intended, I think that’s a valid position but also that we’re certainly overthinking it.
      - bonoboTP 2y ago
        It's specific-object identification, not object categorization. Also in front of a controlled background. They had a limited set of known objects. There was no generalization expected. It asked if it's looking at one specific ball or one specific hammer, not the general category of bird in the wild. In fact, that scope was solved fairly fast, using techniques like Canny edge detection and Minkowski fractal dimension features, Hu moment features on Otsu thresholding etc.
p0w3n3d 2y ago
The only thing hard here nowadays will be no mobile signal in a national park
- left-struck 2y ago
  The bird checking function could run locally and GPS doesn’t require mobile phone signal. It will be much less responsive though, and you need all the relevant offline data
- ryzvonusef 2y ago
  Starlink mini has eliminated that.
  - p0w3n3d 2y ago
    Quite expensive option for occasional birdwatching
- mike_hearn 2y ago
  That's not hard. You just ship the polys of all the national parks as data files in the mobile app and you're done.
  - p0w3n3d 2y ago
    I mean that running LLM on your phone will kill its CPU and memory
dvh 2y ago
Just detect if photo contains common bird color and ship it. We'll fix it later when we decimate the competitors.
jessekv 2y ago
This one always felt off to me. Humans spent millennia working out the navigation problem. The comic exists in this brief window of time where one task was finally "solved" and the other one was just getting started. I'll add that if you think training models takes a lot of energy, try launching fleets of rockets to maintain an artificial satellite constellation.
- roomey 2y ago
  That's a really good point. It only makes sense if we ignore the "standing on the shoulders of giants bit".
  - aiaiaiaiaiai 2y ago
    "Self made millionaire" "Your parents made you; money had to be invented"
- scarface_74 2y ago
  I was just listening to a Planet Money podcast about GPS (https://overcast.fm/+AAYs-52QVys https://overcast.fm/+AAYs-52QVys) while it wouldn’t be “global”, the US did have a terrestrial alternative location system until 2010 (https://en.m.wikipedia.org/wiki/Loran-C https://en.m.wikipedia.org/wiki/Loran-C). With today’s technology, enhanced LORAN would have been more accurate if funded as a backup to GPS
- m463 2y ago
  Also another window... The National Park Service was started August 25, 1916 (only 108 years ago) :)
  - deleted 2y ago
    [deleted]
- josefx 2y ago
  > Humans spent millennia working out the navigation problem. And you think we spend any less time trying to identify food animals that lay tasty eggs?
ryzvonusef 2y ago
Iirc, Flickr had implement bird detection within a few months of this xkcd coming out? EDIT: A month, https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/ https://code.flickr.net/2014/10/20/introducing-flickr-park-o... It's so weird them explaining 'Deep Networks'. Language on AI has definitely changed in the past ten years. Also, hilariously, the page they created to demonstrate this (http://parkorbird.flickr.com/ http://parkorbird.flickr.com/) no longer works. Oh, how time flies. explain XKCD page for good measure: https://www.explainxkcd.com/wiki/index.php/1425:_Tasks https://www.explainxkcd.com/wiki/index.php/1425:_Tasks
llm_trw 2y ago
For those wondering what the state of the art is, I did this in 10 min between meetings. The most challenging thing was getting the right indexing on the lists since it could be an empty list, a list with a single value, or any number of values. from ultralytics import YOLO model = YOLO("yolov10x.pt") results = model("birds/*", classes=[14]) for result in results: if 14 in list(result.boxes.cls): print("Bird!") else: print("Not Bird") I will now take my 5 billion dollar please. Ironically enough the meeting I'm going in to is to explain why I need to use $100,000 worth of hardware for three months before I can answer the business side if what they want to do is possible or not.
- CardenB 2y ago
  Probably better to use openclip
  - llm_trw 2y ago
    You're welcome to provide the code.
    - CardenB 2y ago
      Here’s some code: https://github.com/mlfoundations/open_clip?tab=readme-ov-file#usage https://github.com/mlfoundations/open_clip?tab=readme-ov-fil...
      - llm_trw 2y ago
        You're as helpful as the original gpt2.
- hbossy 2y ago
  Very nice. Now run it on a 2014 phone.
  - ta1243 2y ago
    With 2014 code
m463 2y ago
should be here: https://www.xkcd.com/1425/ https://www.xkcd.com/1425/
1f60c 2y ago
One thing I’ve always felt is that the relative difficulty of each task seems to have flipped? I could write a bird classifier in my sleep using fastai, but I have no idea how to do a GIS lookup.
- moffkalast 2y ago
  An LLM will tell you how to do the GIS lookup, but ironically as privacy laws become better it will genuinely become a harder and harder task unless the user explicitly wants you to do it.
  - ehnto 2y ago
    I don't think that makes the task harder. You should just not do the task at all if the user doesn't want it. Flashback to when apple rolled out their enhanced privacy tools and when I said "that data is blocked unless the user gives us permissions" the project manager un-ironically anguished "But how will we track them?!" Won't someone please think of the commercial interests?! /s
- brazzy 2y ago
  That has nothing to do with the difficulty of the task and everything with what APIs you are personally familiar with.
- solardev 2y ago
  It's not too bad. Just pop up a dialog for the user. "Are you in a national park?"
DevScout 2y ago
It's interesting how problems that were once difficult, like bird detection, have become routine with AI advancements. However, the increasing complexity of AI systems brings new challenges, particularly in areas like privacy and decision-making.
dools 2y ago
> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive. Case in point: the other day my daughter was doing a presentation and she said "Dad can you help me find a picture of the word HELLO spelled out in vegetables?" I was like "CAN I!!?!?! This sounds like a job for ChatGPT". I'll tell you what: ChatGPT can give you a picture of a cat wearing a space suit drinking a martini but it definitely cannot give you the word HELLO spelled out in vegetables. I ended up getting it to give me each individual letter of the alphabet constructed with vegetables and she pasted them together to make the words she wanted for her presentation.
- gpmcadam 2y ago
  Grok did OK? (Grok 2 Mini Beta) https://i.imgur.com/mvnusFd.jpeg https://i.imgur.com/mvnusFd.jpeg
  - dools 2y ago
    Not bad! I just tried with 4o and ChatGPT is still failing hard. Have I just inadvertently invented a new benchmark!?
    - randomdata 2y ago
      No. Had you asked for it to spell "strawberry", though...
    - thaumasiotes 2y ago
      > Not bad! Well, it's not a great example of spelling "HELLO"...
      - dools 2y ago
        Hey we would have been happy with a case insensitive but otherwise correct answer ...
  - card_zero 2y ago
    What kind of vegetable is the first 'l' made out of? It's like tiny chicken nuggets.
riiii 2y ago
Back in the day I had a manager that didn't understand programming. To him, it was just one button that would open this small info window. Just one button. Just one window. It took him weeks to understand that we didn't have the data ready he wanted to show. We could do it, but it would take weeks of research and development.
liquidise 2y ago
Software is a house. The more you lean into the analogy, the better. I often tell clients "The first thing you asked me to do was to move a dining room chair into the living room. Then you asked me to do the same with the toilet. The latter only works if we tear out all the plumbing." Non-coders seem to understand these analogies intuitively.
- amelius 2y ago
  I'm currently building a skyscraper on the foundations of a bikeshed. Anyway, a nice idea for generative AI could be to take source code and turn it into a corresponding image of a building so managers can see what they're doing wrong.
  - ivolimmen 2y ago
    What? Are you working on my project???
  - ornornor 2y ago
    Wait until they tell you it needs an Olympic swimming pool in the penthouse
    - jermaustin1 2y ago
      On the cantilevered terrace of the penthouse. The residents of the penthouse aren't going to give up living space for a pool!
  - antonhag 2y ago
    > I'm currently building a skyscraper on the foundations of a bikeshed. Not sure if you are joking or not, but I often hear similar things and I believe that it misses the point. What constitutes a good foundation in software is very subjective - and just saying "foundation bad" does not help a non-technical person understand _why_ it is bad. It's better to point at that one small rock (some ancient perl-script that no-one longer understands) which holds up the entire thing. Which might be fine until someone needs to move that rock. Or something surrounding it.
ta1243 2y ago
1425 came out in 2014. "I'll need a research team and 5 years" In 2020 The BBC had this blog about cameras detecting not just "is it a bird or superman", but what types of birds https://www.bbc.co.uk/rd/blog/2020-06-springwatch-artificial-intelligence-remote-camera https://www.bbc.co.uk/rd/blog/2020-06-springwatch-artificial... I guess Cueball got the team together.
vanillaicesquad 2y ago
[dead]
zizicat 2y ago
[flagged]
- bhaak 2y ago
  Another problem that would be trivially solved 100% correctly with proper data tagging. Pareto solution: look up that person’s Twitter account which often also include pronouns if they care about that.
  - tmtvl 2y ago
    'Here's a picture of a person, find their Twitter profile' seems like a good way of enabling all kinds of identity fraud, cyberstalking, et cetera.
thih9 2y ago
Note that we are still within Randall’s expectations - the initial estimate for the project at the time was five years and ten years later there is a publicly available solution. It would have been interesting to see the reverse - the problem becoming trivial in less time then the project’s estimate.
- afro88 2y ago
  And even then it doesn't work 100%
- Davidzheng 2y ago
  Convnets had the problem solved around time the comic was released. Recall a post but flickr or something but it wasn't hard by then either
  - deleted 2y ago
    [deleted]
sva_ 2y ago
Btw, Jeremy Howard implements this classifier within the first 10 or so minutes of his excellent diffusion course (with reference to the comic.) https://nitter.poast.org/jeremyphoward/status/1518038001292435456?lang=de https://nitter.poast.org/jeremyphoward/status/15180380012924...
stefanos82 2y ago
@simonw, in case you read this, can I kindly ask you a tiny favor please? Would it be too much to ask you to start livestreaming any coding of yours that can be shared publicly? I would love to learn so many things from you, especially around your current ecosystem, that is Python, SQLite (data), and JavaScript.
- appendix-rock 2y ago
  I’d here to hear what your definition of a non-tiny favour is!
rlalpq 2y ago
Unfortunately, there is limited interest in bird classifiers. There is intense interest in classifiers for military applications: https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_Gaza_Strip https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G... The good thing about selling to the government is that it does not care whether the product is snake oil or not. People on the other hand get tired of AI quite quickly.
cubefox 2y ago
> They're computer systems that are terrible at maths and that can't reliably lookup facts! It seems plausible OpenAI's most recent model is better at math and googling than the average human.
- tomw1808 2y ago
  I agree. But then, a TI-30 is also better at math than the average human. Can't google though...
  - a96 2y ago
    I'm sure somebody must have hacked their TI to google stuff.
  - cubefox 2y ago
    It's not better at math. It can only compute some operations better, but there is much more to math than that. Otherwise, this wouldn't be considered cheating: https://news.ycombinator.com/item?id=41550907 https://news.ycombinator.com/item?id=41550907
- mirrorlake 2y ago
  I've seen math PhDs mess up addition and subtraction on a whiteboard, though. Beating the 99th percentile human at any subject should not be difficult when the LLM training is equivalent to living thousands of lifetimes spent reading and nearly memorizing every book ever written on every university subject. The fact that it only just barely beats humans feels hollow to me. For those who've seen it, imagine if at end of Groundhog Day everyone in the crowd went, "Wow, he's slightly better than average at piano!"
jokoon 2y ago
I want to ask the question: if software engineers had 1000 times or 1 millions times, or 1^12, or 1^30 more computer power and storage, would they be able to make advance in AI or go toward general intelligence? I would say probably not. Unless computer scientists try to understand what biological intelligence really is, AI will not advance.
- int_19h 2y ago
  It's not a given that there is even a coherent definition of what "biological intelligence" is.
Almondsetat 2y ago
GPS is a ready-made infrastructure that took decades of hard work to build and maintain. When the comic was made, image recognition didn't didn't have the same done for it, but now with pre-trained models everyone can do it in 5 minutes too.
roymurdock 2y ago
Wow, really puts the amazing progress over the last decade into perspective. Thanks for sharing
EGreg 2y ago
These brand new AI-assisted proto-programmers are having a crash course in this easy-v.s.-hard problem. Ha! Problem? Just ask ChatGPT. Come on, that one was easy… ps: I am only somewhat joking this time. I think AI agents using just the current state of the art can reliably come up with solutions which the top 1% of humans can come up with. We just need to build them properly. And then deploying swarms at scale, we can scale up 1000x. Scale is all you need at this point. I don’t mean the model parameters, I mean “consensus of expert agents arguing” pps: if you doubt me, look up the talk about “centaurs” beating a computer and a human alone, after Kasparov was bested by Deep Blue. Most of that talk died down 10 years later (Kasparov still believed it after everyone else stopped). No humans in the loop is going to come about quicker than you can predict.
iamyourcanary 2y ago
[dead]
mhb 2y ago
Great, but "check whether the photo is of a bird" should be "center text in three columns".
spaceman_2020 2y ago
Vision LLM is really remarkable Had a project that involved describing and cataloging over 20,000 images. Traditional method using real people would take months and crap load of money (the descriptions have to be customer-readable) OpenAI’s vision API does it for cents per image. Must have spent under $200 for the whole thing
- forrestthewoods 2y ago
  Did you have humans verify OpenAI’s descriptions were accurate?
  - lolinder 2y ago
    For a lot of applications it doesn't matter—you can check the descriptions as you use them as long as everyone knows they may be flawed, and checking 20k descriptions at point of use is still orders of magnitude easier than writing them from scratch.
    - atrettel 2y ago
      I suppose at that scale you may not have to worry as much as errors like this, but I still think the point that there needs to be a human in the supply chain shows how much of AI is in fact just laundering of human intelligence. There's even an XKCD on this [1]! [1] https://xkcd.com/1897/ https://xkcd.com/1897/
      - lolinder 2y ago
        Is the fact that there needs to be a human QC person at a factory evidence that all this purported automation in factories just a laundering of human labor?
        danielbln 2y ago
        Yes, it is, but it's not the slight that grandparent makes it out to be.
joseangel_sc 2y ago
i’d love to read more about the artifact thing that failed because i don’t understand it
itslennysfault 2y ago
I did this tutorial series to try to get some context/foundation in deep learning, and the first lesson was building the bird thing from this comic. It was really easy and fun. The whole course is great. Highly recommend for anyone who has a programming background and wants to get a solid intro to deep learning. https://course.fast.ai/ https://course.fast.ai/
BWStearns 2y ago
I wonder if we'll ever hit a critical mass of technical literacy where this kind of misunderstanding largely disappears. Ten years ago I would have said yes. Now I think the advances in UX/UIs and the appification of everything have insulated the median person from the details. That's good as far as individual products go, but in aggregate might lead to unrealistic expectations. I've heard younger folks ask questions about "why doesn't x just do y" that I previously could only have imagined my very non-technical parents' cohort asking. At least in the 80s, when computers roughly equalled magic for much of the population (looking at you Wargames!), most people didn't really have to interact with it. Their expectations about computers were roughly as important as my expectations about alien life. But I'm afraid that magical thinking about tech will be of greater consequence both individually and societally.
the_af 2y ago
In my opinion a lot of commenters are focusing on current LLM and computer vision capabilities, but missing the point of that particular xkcd. It wasn't "computer vision is really hard!". The point was: for laypeople, it's hard to understand why a particular problem is hard or easy for computers. Some things that seem hard are actually easy (people in comments here mention "standing on the shoulders of giants", but I find that's also missing the point) and some things that seem easy are hard. And it's sometimes difficult to explain to laypeople why something can be done in a couple of hours while other tasks would require a 5-year long research project, when to the external observer both tasks are roughly equally complex. Really, that's it. That's the point of the joke/observation. It wasn't truly an observation about the status of computer vision.
ulrischa 2y ago
I submitted this because I fully agree. The dev world is getting easier but also more complex when it comes to guessing the expense for software
soufron 2y ago
Technological short-sight at its best. The GIS took decades to be developped and become functionnal. The tools allowing to do a GIS lookup also took decades to be developped. The tools allowing to encode the geodata within a picture file idem. Not mentioning the development of the necessary hardware to take the pictures. At the time the comics was being written, GIS lookup had been made easily available since what? 10 years? And 10 years later, another layer is now easily available - also after decades of collective research and development. It's not about "the difference between easy and hard challenges in software". It's about the maturity of a software and its ecosystem, and understanding how even some small incremental changes can have an important impact.
- minkzilla 2y ago
  The point of the comic that that from the outside no one knows how far along hard problems are, not that in and of themselves they are hard. Both the tasks in the comic rely on decades of foundational work in computing and even photography, the author is not saying all of that was easy up until GIS.
  - soufron 2y ago
    I totally agree. The comics is right - it's Munroe after all, but it's the analysis of the blog author that I am divergent with.
    - minkzilla 2y ago
      Ah that makes sense. I must admit I did not even read the analysis of the comic.
Dave_Rosenthal 2y ago
What no-one is pointing out is that LLMs have made almost as much progress on the first part of the request as the second! ChatGPT writes me a is_point_in_national_park function and points me to the relevant shapefile in like ~30 seconds. That's a few hundred times speedup of the "few hours" referenced in the comic.
ezoe 2y ago
Many of the futuristic items I read on manga(especially Doraemon) back in the day I was a child became a reality.
- ProllyInfamous 2y ago
  My mother — daughter of a literal NASA rocket engineer — lost her father around c~2000. She said to me c~2012 "I sure wish my father had lived long enough to see the iPhone's invention: watching scifis he'd tell me I'd live long enough to see telecommunicators even more capable than these movie-props." This mother died six months before ChatGPT's debut, and I thought "sure wish Mom had lived long enough to see LLMs." It wasn't until her first anniversary, flipping through some of our letters, that I realized she had tasted GPT-2 while we played around with ThisWordDoesNotExist.com (made-up words & their "definitions"). Maybe I'll live long enough to see telecommunicators even more capable than these pretrained-transformers.
righthand 2y ago
> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive. That’s because the idea of it being a super human intelligence (an undefined metric) is being sold. So you have to lie and say “it’s amazing, it’s going to change everything”. If I tell you “it’s okay and is often wrong” you wouldn’t buy my product would you? This is just to say I can’t blame that on easy/hard task agency, specifically. === addendumb === “it’s okay and is often wrong” Sounds like working with my junior coworker who I don’t enjoy pairing with. If I said “it’s impressive how the results are to the level of a junior engineer” you sell me on your product.
temporallobe 2y ago
One is using an existing “off-the-shelf” technology, the other is not.
hiergiltdiestfu 2y ago
aged badly in a very good way :D
amp108 2y ago
To be fair, 10 years ago the programmer said "I'll need a research team and five years".
- deleted 2y ago
  [deleted]
smitty1e 2y ago
Years ago--I think he was an IBM Fellow at the time--I had the privilege of hearing Jeff Jonas[1] speak. He was discussing literal spatial indices, scanning the sky for stuff, and how one would break up the problem to make it computable. So I trolled him and asked if he'd take on the weather. [1] https://www.jeffjonas.com/ https://www.jeffjonas.com/
haaryball 2y ago
Made me think of a different one, Constructive https://xkcd.com/810/ https://xkcd.com/810/ Which basically is RLHF :)
maybeben 2y ago
and still not funny

1003 points by ulrischa 2y ago

cung 2y ago

It took a bit over five years, but now checking if it’s a photo of a bird is the easier task.

consp 2y ago
At what confidence level are we talking about? With these over simplified questions (as in the xkcd) my guess would be the asker assumes 100%.
- shiroiushi 2y ago
  You're not going to get 100% confidence with either problem. The GPS one might be easier to get high confidence with, but even here you have to worry about 1) the accuracy of the GPS coordinates from your camera/phone, which isn't that good, and 2) calculating the exact boundaries of the park from the public data. So you could probably calculate with nearly 100% confidence that you're, for example, within 5km of a park, but if you take the photo from a location close to the park's boundary, the confidence will go way down. If you're a meter or two from the boundary, forget it.
  - vinibrito 2y ago
    How could the gps position and the park boundary not be exact? Phones GPS give a 2 meters accuracy, and a park boundary is a well defined hard line polygon. Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that. Asking because I'm afraid I may be missing something here due to this being something I already worked on.
    - shiroiushi 2y ago
      >Phones GPS give a 2 meters accuracy Well I already pointed out that if you're within a couple meters of the boundary, you won't have good confidence because of this fact. >and a park boundary is a well defined hard line polygon. Is it? I'm no expert on parks, but surely some of them have borders along rivers. Many US states have such borders. >Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that. That doesn't account for the 2m accuracy. What if I'm standing exactly 1m from the boundary when I take the photo? You have no idea if I'm really in the park or not from the GPS data. I also have serious doubts about your 2m accuracy claim, based on personal experience. Maybe if you're standing in a wide-open desert with nothing around you, but anywhere else, the accuracy isn't that great, especially around buildings. GPS accuracy is terrible in cities.
      - ddoeth 2y ago
        > Is it? I'm no expert on parks, but surely some of them have borders along rivers. Many US states have such borders. Depending on the country, but Australia has some [1]. I still think that there is a set of polygons that can be used to describe this border. Not to argue against your point (I rarely get less than 4m of accuracy), but luckily > but anywhere else, the accuracy isn't that great, especially around buildings. GPS accuracy is terrible in cities. cities are (almost?) never in national parks. [1]: https://www.nationalparks.nsw.gov.au/-/media/npws/maps/pdfs/parks/border-ranges-national-park/border-ranges-national-park-map.pdf https://www.nationalparks.nsw.gov.au/-/media/npws/maps/pdfs/...
        shiroiushi 2y ago
        >cities are (almost?) never in national parks. Sometimes they are. See Washington, DC. Anyway, the requirement is for determining if a photo was taken in a park or not. The resolution wasn't stated, however: just how accurate do we need to be? If I'm in a canoe in a river that borders a park, but the river isn't part of the park, but the shoreline a few meters away is, our algorithm might claim I'm in the park, when I'm really not. The requirement wasn't "somewhere near a park", but "in a park". Rivers change their courses over time, so some polygons aren't going to accurately describe this border.
    - emj 2y ago
      If you need to know 100% that the bird is in the park at that precise moment it can be tricky. If you need to identify a Bird-of-prey in the Alpha quadrant you can understand the Klingon proverb a sharp knife is nothing without a sharp eye.
    - deleted 2y ago
      [deleted]
  - vincnetas 2y ago
    What does it even mean "YOU are in the park". What is YOU? If you standing on the boundary, are YOU in or out? Details mater :D
    - shiroiushi 2y ago
      Exactly! What if you're standing on the boundary, with one foot inside and the other outside? These specifications are far too vague.
marricks 2y ago
For a person to set up but definitely not how many cpu cycles are burned
qwertox 2y ago
Is it? I assume that you are thinking of using a 3rd-party API endpoint to which you upload the image so that the service decides for you if it is a bird and which kind of bird it is. Or you use something like Firebase. Because if that is the way you'd solve this problem, then just sending lat/lon to a service to determine if it is in a national park is even easier, as it's just a GET request. I'm still unsure about what would be harder to set up locally.
- llm_trw 2y ago
  https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/ https://tech.amikelive.com/node-718/what-object-categories-l... Bird is in the default COCO dataset. I haven't look for birds in images, but for people yolov10 is fewer lines of code to detect if there is someone in the frame than it is to setup a flask server for the API calls.
- a_t48 2y ago
  Using a service for this would kind of suck if you're _in_ the national park, due to cell service. The ML portion of this is probably still a bit harder than the GPS bit. ML: Grab yolov8 (Optional?) Fine tune it on bird pictures Convert it to CoreML, do whatever iOS stuff is needed in XCode to run it GPS: Get https://www.nps.gov/lib/npmap.js/4.0.0/examples/data/national-parks.geojson (TIL this exists! thanks federal govt!) Stuff it into the app somehow Get the coordinates from the OS Use your favorite library for point+polygon intersection to decide if you're in a national park Bonus: use distance from polygon instead to account for GPS inaccuracy, keeping in mind lat and long have different scales. ...actually the ML one might be easier, nowadays. Now I kind of want to try this.
  - xxs 2y ago
    > Use your favorite library for point+polygon intersection to decide if you're in a national park it's like 50ish lines of code in C, just iterating over the points (with the polygon represented by arrays of points). The algorithm is linear with regards to the points.
munchler 2y ago
Your phone already does both automatically, so I’d call it a draw.
unsigner 2y ago
"easier tasks" is arguable and arguably wrong "task about which you will find more easy-looking tutorials hiding the complexity under a blanket of 3rd party code and services" is better
- YurgenJurgensen 2y ago
  You just described all computational tasks that don’t open with ‘first, dig up some sand and some copper ore’.
  - aiaiaiaiaiai 2y ago
    Except all the stuff millions of software engineers work on for billions of hours per year.
  - unsigner 2y ago
    checking whether coordinates fall inside a national park is an exercise in computational geometry, surveying (the earth is not flat, what are coordinates on a sphere), databases, access to government data. detecting birds is an exercise in gathering properly labeled training sets, neural networks and their topologies, matrix multiplication performance and/or orchestration of rented GPUs. both of which cover interesting tasks, worthy things to learn, and are by no means easy. "easy" is the bird recognition where you do an API call to a totem pole of third party services.
thaumasiotes 2y ago
> but now checking if it’s a photo of a bird is the easier task. That depends on whether you care about getting the answer right. If you don't, it was always the easier task. If you do, Seek by iNaturalist still can't do this job, and that's the only thing Seek is supposed to be able to do.

ml_basics 2y ago

It's quite remarkable how much the goal posts have shifted when it comes to what is impressive with AI/ML. Things like this are a good reminder. 10 years ago the GAN paper came out and everyone was excited how amazing the generated image quality was (https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/1406.2661) The amount of progress we've made is mind boggling.

fouronnes3 2y ago
Arguably the goal post for AGI has moved about as much, if not more. One wonders if Turing reading a 2024 LLM chat transcript would say "but it's not really thinking!".
- audunw 2y ago
  Passing the Turing test has always been a non-binary thing. Chat bots have been able to pass off as a human for a short time under certain circumstances. Now they can pass off as human for a longer time under more circumstances. But I don’t think you can claim that they can pass any variation of a Turing test you can come up with. Has the AGI goal post been shifted? Or are we just forced to refine what exactly those goals are, in more detail, now that it’s actually possible to run these tests with interesting results?
  - sigmoid10 2y ago
    But isn't this exactly the goalpost moving the other comment claimed? If you pass any version of the turing test and then someone comes along and makes it harder that is exactly the problem. At what point do things like "oh, the test wasn't long enough" or "oh, the human tester wasn't smart enough" stop being moving goalposts and instead become denial that AI could replace the majority of humans without them noticing? Because that's where we're headed and it's also where the real danger is. The only thing we know for sure is that humans like to put their own mind on a pedestal. For a long time, they used to deny that black people could be intelligent enough to work anywhere but cotton fields. In the same way they used to deny that women could be smart enough to vote. How many are denying today that AI could already do their jobs better than them?

weinzierl 2y ago

One could say, "This didn't age well." but I think the real point of "it can be hard to explain the difference between the easy and the virtually impossible" is only reinforced by an almost ironic twist that switched the hard and easy around. Who would have thought ten yeas ago?

0x006A 2y ago
Its worth looking at the alt text of the xkcd itself: In the 60s, Marvin Minsky assigned a couple of undergrads to spend the summer programming a computer to use a camera to identify objects in a scene. He figured they'd have the problem solved by the end of the summer. Half a century later, we're still working on it.
- bonoboTP 2y ago
  It's a cute anecdote but of course reality is more boring. The assignment wasn't to solve human level vision in a summer, just to do quite constrained object recognition. But even that turned out to be quite hard. But Minksy wasn't that fully miscalibrated.
  - acdha 2y ago
    I think it’s worth reading the original assignment: https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100.pdf?sequence=2&isAllowed=y https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100.... If you want to argue that “likely objects” is weaker than Munroe intended, I think that’s a valid position but also that we’re certainly overthinking it.
    - bonoboTP 2y ago
      It's specific-object identification, not object categorization. Also in front of a controlled background. They had a limited set of known objects. There was no generalization expected. It asked if it's looking at one specific ball or one specific hammer, not the general category of bird in the wild. In fact, that scope was solved fairly fast, using techniques like Canny edge detection and Minkowski fractal dimension features, Hu moment features on Otsu thresholding etc.

p0w3n3d 2y ago

The only thing hard here nowadays will be no mobile signal in a national park

left-struck 2y ago
The bird checking function could run locally and GPS doesn’t require mobile phone signal. It will be much less responsive though, and you need all the relevant offline data
ryzvonusef 2y ago
Starlink mini has eliminated that.
- p0w3n3d 2y ago
  Quite expensive option for occasional birdwatching
mike_hearn 2y ago
That's not hard. You just ship the polys of all the national parks as data files in the mobile app and you're done.
- p0w3n3d 2y ago
  I mean that running LLM on your phone will kill its CPU and memory

dvh 2y ago

Just detect if photo contains common bird color and ship it. We'll fix it later when we decimate the competitors.

jessekv 2y ago

This one always felt off to me. Humans spent millennia working out the navigation problem. The comic exists in this brief window of time where one task was finally "solved" and the other one was just getting started. I'll add that if you think training models takes a lot of energy, try launching fleets of rockets to maintain an artificial satellite constellation.

roomey 2y ago
That's a really good point. It only makes sense if we ignore the "standing on the shoulders of giants bit".
- aiaiaiaiaiai 2y ago
  "Self made millionaire" "Your parents made you; money had to be invented"
scarface_74 2y ago
I was just listening to a Planet Money podcast about GPS (https://overcast.fm/+AAYs-52QVys https://overcast.fm/+AAYs-52QVys) while it wouldn’t be “global”, the US did have a terrestrial alternative location system until 2010 (https://en.m.wikipedia.org/wiki/Loran-C https://en.m.wikipedia.org/wiki/Loran-C). With today’s technology, enhanced LORAN would have been more accurate if funded as a backup to GPS
m463 2y ago
Also another window... The National Park Service was started August 25, 1916 (only 108 years ago) :)
- deleted 2y ago
  [deleted]
josefx 2y ago
> Humans spent millennia working out the navigation problem. And you think we spend any less time trying to identify food animals that lay tasty eggs?

ryzvonusef 2y ago

Iirc, Flickr had implement bird detection within a few months of this xkcd coming out? EDIT: A month, https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/ https://code.flickr.net/2014/10/20/introducing-flickr-park-o... It's so weird them explaining 'Deep Networks'. Language on AI has definitely changed in the past ten years. Also, hilariously, the page they created to demonstrate this (http://parkorbird.flickr.com/ http://parkorbird.flickr.com/) no longer works. Oh, how time flies. explain XKCD page for good measure: https://www.explainxkcd.com/wiki/index.php/1425:_Tasks https://www.explainxkcd.com/wiki/index.php/1425:_Tasks

llm_trw 2y ago

For those wondering what the state of the art is, I did this in 10 min between meetings. The most challenging thing was getting the right indexing on the lists since it could be an empty list, a list with a single value, or any number of values. from ultralytics import YOLO model = YOLO("yolov10x.pt") results = model("birds/*", classes=[14]) for result in results: if 14 in list(result.boxes.cls): print("Bird!") else: print("Not Bird") I will now take my 5 billion dollar please. Ironically enough the meeting I'm going in to is to explain why I need to use $100,000 worth of hardware for three months before I can answer the business side if what they want to do is possible or not.

CardenB 2y ago
Probably better to use openclip
- llm_trw 2y ago
  You're welcome to provide the code.
  - CardenB 2y ago
    Here’s some code: https://github.com/mlfoundations/open_clip?tab=readme-ov-file#usage https://github.com/mlfoundations/open_clip?tab=readme-ov-fil...
    - llm_trw 2y ago
      You're as helpful as the original gpt2.
hbossy 2y ago
Very nice. Now run it on a 2014 phone.
- ta1243 2y ago
  With 2014 code

m463 2y ago

should be here: https://www.xkcd.com/1425/ https://www.xkcd.com/1425/

1f60c 2y ago

One thing I’ve always felt is that the relative difficulty of each task seems to have flipped? I could write a bird classifier in my sleep using fastai, but I have no idea how to do a GIS lookup.

moffkalast 2y ago
An LLM will tell you how to do the GIS lookup, but ironically as privacy laws become better it will genuinely become a harder and harder task unless the user explicitly wants you to do it.
- ehnto 2y ago
  I don't think that makes the task harder. You should just not do the task at all if the user doesn't want it. Flashback to when apple rolled out their enhanced privacy tools and when I said "that data is blocked unless the user gives us permissions" the project manager un-ironically anguished "But how will we track them?!" Won't someone please think of the commercial interests?! /s
brazzy 2y ago
That has nothing to do with the difficulty of the task and everything with what APIs you are personally familiar with.
solardev 2y ago
It's not too bad. Just pop up a dialog for the user. "Are you in a national park?"

DevScout 2y ago

It's interesting how problems that were once difficult, like bird detection, have become routine with AI advancements. However, the increasing complexity of AI systems brings new challenges, particularly in areas like privacy and decision-making.

dools 2y ago

> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive. Case in point: the other day my daughter was doing a presentation and she said "Dad can you help me find a picture of the word HELLO spelled out in vegetables?" I was like "CAN I!!?!?! This sounds like a job for ChatGPT". I'll tell you what: ChatGPT can give you a picture of a cat wearing a space suit drinking a martini but it definitely cannot give you the word HELLO spelled out in vegetables. I ended up getting it to give me each individual letter of the alphabet constructed with vegetables and she pasted them together to make the words she wanted for her presentation.

gpmcadam 2y ago
Grok did OK? (Grok 2 Mini Beta) https://i.imgur.com/mvnusFd.jpeg https://i.imgur.com/mvnusFd.jpeg
- dools 2y ago
  Not bad! I just tried with 4o and ChatGPT is still failing hard. Have I just inadvertently invented a new benchmark!?
  - randomdata 2y ago
    No. Had you asked for it to spell "strawberry", though...
  - thaumasiotes 2y ago
    > Not bad! Well, it's not a great example of spelling "HELLO"...
    - dools 2y ago
      Hey we would have been happy with a case insensitive but otherwise correct answer ...
- card_zero 2y ago
  What kind of vegetable is the first 'l' made out of? It's like tiny chicken nuggets.

riiii 2y ago

Back in the day I had a manager that didn't understand programming. To him, it was just one button that would open this small info window. Just one button. Just one window. It took him weeks to understand that we didn't have the data ready he wanted to show. We could do it, but it would take weeks of research and development.

liquidise 2y ago

Software is a house. The more you lean into the analogy, the better. I often tell clients "The first thing you asked me to do was to move a dining room chair into the living room. Then you asked me to do the same with the toilet. The latter only works if we tear out all the plumbing." Non-coders seem to understand these analogies intuitively.

amelius 2y ago
I'm currently building a skyscraper on the foundations of a bikeshed. Anyway, a nice idea for generative AI could be to take source code and turn it into a corresponding image of a building so managers can see what they're doing wrong.
- ivolimmen 2y ago
  What? Are you working on my project???
- ornornor 2y ago
  Wait until they tell you it needs an Olympic swimming pool in the penthouse
  - jermaustin1 2y ago
    On the cantilevered terrace of the penthouse. The residents of the penthouse aren't going to give up living space for a pool!
- antonhag 2y ago
  > I'm currently building a skyscraper on the foundations of a bikeshed. Not sure if you are joking or not, but I often hear similar things and I believe that it misses the point. What constitutes a good foundation in software is very subjective - and just saying "foundation bad" does not help a non-technical person understand _why_ it is bad. It's better to point at that one small rock (some ancient perl-script that no-one longer understands) which holds up the entire thing. Which might be fine until someone needs to move that rock. Or something surrounding it.

ta1243 2y ago

1425 came out in 2014. "I'll need a research team and 5 years" In 2020 The BBC had this blog about cameras detecting not just "is it a bird or superman", but what types of birds https://www.bbc.co.uk/rd/blog/2020-06-springwatch-artificial-intelligence-remote-camera https://www.bbc.co.uk/rd/blog/2020-06-springwatch-artificial... I guess Cueball got the team together.

vanillaicesquad 2y ago

[dead]

zizicat 2y ago

[flagged]

bhaak 2y ago
Another problem that would be trivially solved 100% correctly with proper data tagging. Pareto solution: look up that person’s Twitter account which often also include pronouns if they care about that.
- tmtvl 2y ago
  'Here's a picture of a person, find their Twitter profile' seems like a good way of enabling all kinds of identity fraud, cyberstalking, et cetera.

thih9 2y ago

Note that we are still within Randall’s expectations - the initial estimate for the project at the time was five years and ten years later there is a publicly available solution. It would have been interesting to see the reverse - the problem becoming trivial in less time then the project’s estimate.

afro88 2y ago
And even then it doesn't work 100%
Davidzheng 2y ago
Convnets had the problem solved around time the comic was released. Recall a post but flickr or something but it wasn't hard by then either
- deleted 2y ago
  [deleted]

sva_ 2y ago

Btw, Jeremy Howard implements this classifier within the first 10 or so minutes of his excellent diffusion course (with reference to the comic.) https://nitter.poast.org/jeremyphoward/status/1518038001292435456?lang=de https://nitter.poast.org/jeremyphoward/status/15180380012924...

stefanos82 2y ago

@simonw, in case you read this, can I kindly ask you a tiny favor please? Would it be too much to ask you to start livestreaming any coding of yours that can be shared publicly? I would love to learn so many things from you, especially around your current ecosystem, that is Python, SQLite (data), and JavaScript.

appendix-rock 2y ago
I’d here to hear what your definition of a non-tiny favour is!

rlalpq 2y ago

Unfortunately, there is limited interest in bird classifiers. There is intense interest in classifiers for military applications: https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_Gaza_Strip https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G... The good thing about selling to the government is that it does not care whether the product is snake oil or not. People on the other hand get tired of AI quite quickly.

cubefox 2y ago

> They're computer systems that are terrible at maths and that can't reliably lookup facts! It seems plausible OpenAI's most recent model is better at math and googling than the average human.

tomw1808 2y ago
I agree. But then, a TI-30 is also better at math than the average human. Can't google though...
- a96 2y ago
  I'm sure somebody must have hacked their TI to google stuff.
- cubefox 2y ago
  It's not better at math. It can only compute some operations better, but there is much more to math than that. Otherwise, this wouldn't be considered cheating: https://news.ycombinator.com/item?id=41550907 https://news.ycombinator.com/item?id=41550907
mirrorlake 2y ago
I've seen math PhDs mess up addition and subtraction on a whiteboard, though. Beating the 99th percentile human at any subject should not be difficult when the LLM training is equivalent to living thousands of lifetimes spent reading and nearly memorizing every book ever written on every university subject. The fact that it only just barely beats humans feels hollow to me. For those who've seen it, imagine if at end of Groundhog Day everyone in the crowd went, "Wow, he's slightly better than average at piano!"

jokoon 2y ago

I want to ask the question: if software engineers had 1000 times or 1 millions times, or 1^12, or 1^30 more computer power and storage, would they be able to make advance in AI or go toward general intelligence? I would say probably not. Unless computer scientists try to understand what biological intelligence really is, AI will not advance.

int_19h 2y ago
It's not a given that there is even a coherent definition of what "biological intelligence" is.

Almondsetat 2y ago

GPS is a ready-made infrastructure that took decades of hard work to build and maintain. When the comic was made, image recognition didn't didn't have the same done for it, but now with pre-trained models everyone can do it in 5 minutes too.

roymurdock 2y ago

Wow, really puts the amazing progress over the last decade into perspective. Thanks for sharing

EGreg 2y ago

These brand new AI-assisted proto-programmers are having a crash course in this easy-v.s.-hard problem. Ha! Problem? Just ask ChatGPT. Come on, that one was easy… ps: I am only somewhat joking this time. I think AI agents using just the current state of the art can reliably come up with solutions which the top 1% of humans can come up with. We just need to build them properly. And then deploying swarms at scale, we can scale up 1000x. Scale is all you need at this point. I don’t mean the model parameters, I mean “consensus of expert agents arguing” pps: if you doubt me, look up the talk about “centaurs” beating a computer and a human alone, after Kasparov was bested by Deep Blue. Most of that talk died down 10 years later (Kasparov still believed it after everyone else stopped). No humans in the loop is going to come about quicker than you can predict.

iamyourcanary 2y ago

[dead]

mhb 2y ago

Great, but "check whether the photo is of a bird" should be "center text in three columns".

spaceman_2020 2y ago

Vision LLM is really remarkable Had a project that involved describing and cataloging over 20,000 images. Traditional method using real people would take months and crap load of money (the descriptions have to be customer-readable) OpenAI’s vision API does it for cents per image. Must have spent under $200 for the whole thing

forrestthewoods 2y ago
Did you have humans verify OpenAI’s descriptions were accurate?
- lolinder 2y ago
  For a lot of applications it doesn't matter—you can check the descriptions as you use them as long as everyone knows they may be flawed, and checking 20k descriptions at point of use is still orders of magnitude easier than writing them from scratch.
  - atrettel 2y ago
    I suppose at that scale you may not have to worry as much as errors like this, but I still think the point that there needs to be a human in the supply chain shows how much of AI is in fact just laundering of human intelligence. There's even an XKCD on this [1]! [1] https://xkcd.com/1897/ https://xkcd.com/1897/
    - lolinder 2y ago
      Is the fact that there needs to be a human QC person at a factory evidence that all this purported automation in factories just a laundering of human labor?
      - danielbln 2y ago
        Yes, it is, but it's not the slight that grandparent makes it out to be.

joseangel_sc 2y ago

i’d love to read more about the artifact thing that failed because i don’t understand it

itslennysfault 2y ago

I did this tutorial series to try to get some context/foundation in deep learning, and the first lesson was building the bird thing from this comic. It was really easy and fun. The whole course is great. Highly recommend for anyone who has a programming background and wants to get a solid intro to deep learning. https://course.fast.ai/ https://course.fast.ai/

BWStearns 2y ago

I wonder if we'll ever hit a critical mass of technical literacy where this kind of misunderstanding largely disappears. Ten years ago I would have said yes. Now I think the advances in UX/UIs and the appification of everything have insulated the median person from the details. That's good as far as individual products go, but in aggregate might lead to unrealistic expectations. I've heard younger folks ask questions about "why doesn't x just do y" that I previously could only have imagined my very non-technical parents' cohort asking. At least in the 80s, when computers roughly equalled magic for much of the population (looking at you Wargames!), most people didn't really have to interact with it. Their expectations about computers were roughly as important as my expectations about alien life. But I'm afraid that magical thinking about tech will be of greater consequence both individually and societally.

the_af 2y ago

In my opinion a lot of commenters are focusing on current LLM and computer vision capabilities, but missing the point of that particular xkcd. It wasn't "computer vision is really hard!". The point was: for laypeople, it's hard to understand why a particular problem is hard or easy for computers. Some things that seem hard are actually easy (people in comments here mention "standing on the shoulders of giants", but I find that's also missing the point) and some things that seem easy are hard. And it's sometimes difficult to explain to laypeople why something can be done in a couple of hours while other tasks would require a 5-year long research project, when to the external observer both tasks are roughly equally complex. Really, that's it. That's the point of the joke/observation. It wasn't truly an observation about the status of computer vision.

ulrischa 2y ago

I submitted this because I fully agree. The dev world is getting easier but also more complex when it comes to guessing the expense for software

soufron 2y ago

Technological short-sight at its best. The GIS took decades to be developped and become functionnal. The tools allowing to do a GIS lookup also took decades to be developped. The tools allowing to encode the geodata within a picture file idem. Not mentioning the development of the necessary hardware to take the pictures. At the time the comics was being written, GIS lookup had been made easily available since what? 10 years? And 10 years later, another layer is now easily available - also after decades of collective research and development. It's not about "the difference between easy and hard challenges in software". It's about the maturity of a software and its ecosystem, and understanding how even some small incremental changes can have an important impact.

minkzilla 2y ago
The point of the comic that that from the outside no one knows how far along hard problems are, not that in and of themselves they are hard. Both the tasks in the comic rely on decades of foundational work in computing and even photography, the author is not saying all of that was easy up until GIS.
- soufron 2y ago
  I totally agree. The comics is right - it's Munroe after all, but it's the analysis of the blog author that I am divergent with.
  - minkzilla 2y ago
    Ah that makes sense. I must admit I did not even read the analysis of the comic.

Dave_Rosenthal 2y ago

What no-one is pointing out is that LLMs have made almost as much progress on the first part of the request as the second! ChatGPT writes me a is_point_in_national_park function and points me to the relevant shapefile in like ~30 seconds. That's a few hundred times speedup of the "few hours" referenced in the comic.

ezoe 2y ago

Many of the futuristic items I read on manga(especially Doraemon) back in the day I was a child became a reality.

ProllyInfamous 2y ago
My mother — daughter of a literal NASA rocket engineer — lost her father around c~2000. She said to me c~2012 "I sure wish my father had lived long enough to see the iPhone's invention: watching scifis he'd tell me I'd live long enough to see telecommunicators even more capable than these movie-props." This mother died six months before ChatGPT's debut, and I thought "sure wish Mom had lived long enough to see LLMs." It wasn't until her first anniversary, flipping through some of our letters, that I realized she had tasted GPT-2 while we played around with ThisWordDoesNotExist.com (made-up words & their "definitions"). Maybe I'll live long enough to see telecommunicators even more capable than these pretrained-transformers.

righthand 2y ago

> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive. That’s because the idea of it being a super human intelligence (an undefined metric) is being sold. So you have to lie and say “it’s amazing, it’s going to change everything”. If I tell you “it’s okay and is often wrong” you wouldn’t buy my product would you? This is just to say I can’t blame that on easy/hard task agency, specifically. === addendumb === “it’s okay and is often wrong” Sounds like working with my junior coworker who I don’t enjoy pairing with. If I said “it’s impressive how the results are to the level of a junior engineer” you sell me on your product.

temporallobe 2y ago

One is using an existing “off-the-shelf” technology, the other is not.

hiergiltdiestfu 2y ago

aged badly in a very good way :D

amp108 2y ago

To be fair, 10 years ago the programmer said "I'll need a research team and five years".

deleted 2y ago
[deleted]

smitty1e 2y ago

Years ago--I think he was an IBM Fellow at the time--I had the privilege of hearing Jeff Jonas[1] speak. He was discussing literal spatial indices, scanning the sky for stuff, and how one would break up the problem to make it computable. So I trolled him and asked if he'd take on the weather. [1] https://www.jeffjonas.com/ https://www.jeffjonas.com/

haaryball 2y ago

Made me think of a different one, Constructive https://xkcd.com/810/ https://xkcd.com/810/ Which basically is RLHF :)

maybeben 2y ago

and still not funny