31 ms·
Vesuvius Challenge 2023 Grand Prize awarded: we can read the first scroll
- sekai 3y ago> We estimate that the scrolls we have in Naples contain more than 16 megabytes of text. Some members of our papyrology team say that revealing this text will be the greatest revolution in the classics since the Renaissance Amazing achievement, let's hope the Italian government allows for additional excavation of the villa.
- riffraff 3y agothey likely would, Pompeii and Herculaneum are _still_ being excavated after two centuries, it's not like things are still. But we have only read 5% of this scroll and there are a ton more already excavated, it will probably take years before we manage to process what we already have.
- seydor 3y ago> it will probably take years In the direction things are going ... maybe a few months :)
- pimlottc 3y agoThere’s more to processing than scanning, it has has to be reviewed, transcribed and translated by linguistics experts, and then analyzed and studied by academics and researchers who can put in context and integrate it with what we currently know about history, cultural, philosophy, etc of the time.
- kristopolous 3y agoThe biggest bottleneck is getting the experts to read it. You need a decade or so of graduate level education and interpreting things takes apparently quite a long time. Maybe that's another AI application.
- digging 3y agoAs much as I'd like to endorse study of the classics, I'm almost certain that AI will be better at interpreting the texts than humans very soon.
- ffgjgf1 3y agoGPT 4.0 isn’t even remotely close to being useful at all in this case so I have some doubts about ‘very soon’
- bluGill 3y agoI doubt it. Most universities have some professor in this area and a few graduate students. There is very little else for them to read (the entire of all classic books we have fits on a shelf), and most of them can read the original language. They will read this as soon as it is available just because it they don't have much else to read that is a primary source material. Thus everything new is of great interest because there is so little to work with.
- kristopolous 3y agoThe Oxyrhynchus Papyri was discovered in the 1800s, has been fully digitized and experts expect it to take I think 150 additional years before they can all be read to the level of scholarship that's required I found some conversation on the difficulty here https://www.reddit.com/r/AcademicBiblical/comments/16dj68i/how_many_oxyrhynchus_papyri_are_left/ https://www.reddit.com/r/AcademicBiblical/comments/16dj68i/h...
- bluGill 3y agoThanks for the link. However the problems there do not seem to apply here. There the problem seems to be lots of little fragments that need to be fit together to figure out the scroll. Here the scrolls are complete in one piece, so we have a much better idea of how the parts fit together (not perfect since this is layers that have to be combined, but close). There just getting the text is hard, here we mostly have the text (though we will need to error correct, they are at least readable)
- Digory 3y agoIf you can automate the input, you can probably automate much of the basic analysis (things that would be "revolutionary" to undergrads). "ChatGPT, give me the highlights of these ancient Greek scrolls ..."
- wl 3y agoThe big problem is that the Villa of the Papyri is underneath modern buildings. That doesn't mean that excavation without demolition is impossible (see the Scavi underneath the Basilica of Saint Peter), but it makes things far more difficult.
- BlueTemplar 3y agoIf the prospect is very high to multiply by several times the total remaining classical works, I doubt that the money will be particularly hard to find ?
- wl 3y agoDavid W. Packard (HP heir) has been trying to throw money at doing this for years, so the money isn't as much of an issue as you'd think. The larger issue is that the locals don't want digging underneath their buildings, no matter how careful the excavators are. Also, all the money that would be necessary to excavate has made the project a target for the mafia who wants to get their share.
- jobs_throwaway 3y ago> the money isn't as much of an issue as you'd think. The larger issue is that the locals don't want digging underneath their buildings, no matter how careful the excavators are This sounds like the money IS a huge issue. How expensive can it be to buy out the locals? We're talking about priceless cultural artifacts
- jacquesm 3y ago> Also, all the money that would be necessary to excavate has made the project a target for the mafia who wants to get their share. I wonder how far Italy will go once - if - they get rid of the mafia, it is like trying to drive a car with the handbrake on.
- alach11 3y agoThis is the coolest thing I've read this year. It reads like science fiction. Who would even imagine it's possible to read text from a 2000 year-old rolled up burnt-crisp paper?
- dougmwne 3y agoIt’s a 270 year archaeological and technological culmination. The scrolls were dug up in 1752. It took the collective developments of the Industrial Revolution, the sciences all our engineering and manufacturing prowess to discover, preserve and scan the scrolls. Then the final cherry on top of the current AI revolution that can create inferences and connections that are beyond the human mind to even understand. And out pops 2000 year-old wisdom of the ancients.
- bee_rider 3y agoIt is a bit fitting that it turns out the scroll is about the relationship between enjoyment and abundance. It looks like, from what we can gather, the author decides that should something be hard to get, that doesn’t lead to greater enjoyment. But, it seems that the archaeologists have found an awful lot of joy in how “rare” access to these scrolls is!
- jfengel 3y agoOne other thing that was required: the patience not to ruin it. So much has been lost to well-meaning archaeologists who dug up and threw away things that they didn't think were important. They tried cleaning and preservation techniques on artifacts without testing, sometimes ruining them in the process. They ripped things out of context, and "restored" them based on guesses that were sometimes flagrantly wrong. Of course they couldn't be expected to know everything that would come in the future, so blame can sometimes perhaps be muted. But it's especially positive that they extricated these particular objects very carefully and just waited for a way to extract information that they could hardly have hoped for.
- linsomniac 3y agoSpeaking of "reading like sci-fi", what's that book where they scan an entire library of books, descructively, by feeding them into a "book chipper" like device that chops the books up into little pieces, vacuums those pieces up and scans the pieces as they flow through, reconstructing the original text by putting the scanned results together like so many jigsaw puzzles? It was a subplot of the book, but I can't for the life of me remember what book it was.
- kretaceous 3y agoWhen I first came across this project on HN (early last year), I was taken aback by how impossible the project looked and how smart were people working on this. Despite seeing a few intelligent names behind the project, I subconsciously believed that this would at least take 5-10 years before a breakthrough. Today I sit with the same amazement, taken aback again, appreciating how ridiculously awesome this is. Congratulations to the winners and everyone involved!
- qingcharles 3y agoSo many things that look insane are becoming a reality. You look at those scrolls burnt to a crisp and the idea of reading them is nonsense. The fact I have a computer writing flowery alt text descriptions of my photos with unnerving accuracy is something I would not have predicted for another 20 years. But, here we are...
- Keyframe 3y agoRight? Imagine trying to explain some of it to one of the ancients - so, you have this quartz sand, see?...
- p-e-w 3y agoThat would be like explaining a baking recipe by starting with protons and electrons. State-of-the-art machine learning architectures aren't actually that complex. Diffusion models and transformers can be explained to a bright high schooler. I'm sure Archimedes and Euclid would have no problem understanding them. What they might have a problem understanding (or even imagining) is the mind-boggling amount of computation required to make those systems do anything useful. Getting Llama to produce a single token of text takes more calculations than all of humanity did by hand during all of Classical Antiquity.
- hnfong 3y agoI think the quartz sand metaphor was to illustrate how advanced our silicon-based technology has become, not just the ML parts. Imagine all the stuff... transistors, Turing/Von Neumann machines, lithography, theoretical computer science, OS and compilers, the Internet... and lastly there's modern day machine learning that builds on top of all the above. The base level stuff isn't exactly protons and electrons, but given the nanometer scale of our chips, it's not that far away from the truth, and we (humanity) has somehow built amazing stuff on top of that.
- jtchang 3y ago“Any sufficiently advanced technology is indistinguishable from magic.” Absolutely insane the level of wizardry being applied here to turn a lump of blackened, charred scrolls into readable text. Having only cursory knowledge with machine learning are some of the techniques used in the article only recently discovered or have they been around for a while? Is it due to us having reached an inflection point with these types of algorithms that they have become more popular and thus we are seeing new ways to apply them to old problems?
- kortex 3y agoThere has definitely been a virtuous cycle between GP-GPU processing capability, algorithms, libraries and software that use that hardware, and researchers working with those tools.
- echelon 3y ago> Absolutely insane the level of wizardry being applied here to turn a lump of blackened, charred scrolls into readable text. Imagine what we'll be able to do to brains, dead or alive, in 100 years. And in 10,000, maybe we'll be reconstructing the light cone. Maybe that's what we are right now. (Not serious, but it's a fun thought experiment.)
- xvector 3y agoThis is why I am going for cryopreservation if I ever have the luxury of choosing the way I die.
- echelon 3y agoWell, that could go lots of ways. Maybe some rich trillionaire buys you and spawns you into an endless horror simulation. They might be into torture and get off on it. ("No real humans harmed.") But if the future can reverse the light cone, nobody is immune to that fate. Who knows what the future holds. These are just sci-fi flights of fancy.
- 3y ago
- countrymile 3y agoQuite incredible work, with the original breakthrough model being trained on a 1070: https://twitter.com/LukeFarritor/status/1754532281690243339 https://twitter.com/LukeFarritor/status/1754532281690243339
- bglazer 3y agoOne aspect of archaeology that I really find fascinating is the practice of leaving certain artifacts unexplored. The original discoverers of the scrolls tried to unroll a few, apparently found it was impossible without completely destroying the scroll, and then just left the rest undisturbed. Rather than pushing forward and destroying everything, they left these as a mystery for a future age. Two centuries (!!) later we can finally begin to understand these, with the aid of technology that would be utterly unthinkable to those people who very thoughtfully restrained themselves.
- dmurray 3y agoAn example of the same thing at a macro level: https://www.smithsonianmag.com/smart-news/archaeologists-rebury-first-of-its-kind-roman-villa-180980535/ https://www.smithsonianmag.com/smart-news/archaeologists-reb...
- _a_a_a_ 3y agoBigger yet by far https://en.wikipedia.org/wiki/Mausoleum_of_the_First_Qin_Emperor https://en.wikipedia.org/wiki/Mausoleum_of_the_First_Qin_Emp... He of the terracotta army. Not excavated yet for fear of damage, but I would so love to know...
- cameron_b 3y agoThe feeling you get when you’ve gone into one of those aircraft hanger-size buildings and then you see some of the information they’ve gotten with ground tests ( radar, mercury, etc ) is wild. The site is huge. One of the suppositions is that the main chamber contained a model of his entire kingdom, replete with rivers of mercury. So yes. Archaeology is a bit destructive, and sometimes the destruction can go both ways. Proceed with caution.
- unsupp0rted 3y ago> Rather than pushing forward and destroying everything In the early days they wouldn't have accomplished anything by pushing forward, so it doesn't take all that much restraint. I'm more impressed by people in, say, the 1990s or early 2000s. They might've had a shot but there was still too much risk, so they restrained themselves until it was a safer bet.
- TremendousJudge 3y agoThis is exciting, it's very unusual for new texts to be added to our collection of ancient literature.
- jdminhbg 3y agoHere is the link to their "master plan" to read all of the excavated scrolls: https://scrollprize.org/master_plan https://scrollprize.org/master_plan It looks like there are two main bottlenecks to reading more: the need for manual intervention in segmenting the scanned scrolls, and the cost in scanning new scrolls.
- kilroy123 3y agoWhat a refreshingly clear and thought-out plan. This project honestly gives me a lot of hope.
- Animats 3y agoYes. Now it can be done, but costs too much. Once they get a scanning unit near the scrolls, it will be much cheaper. The data reduction will probably get cheaper, too.
- s0rce 3y agoScanning unit? Seems like the scanning was done using a synchrotron beamline. Maybe there is a suitable beamline at Elettra. I haven't looked closely why the synchrotron is needed. A Sigray instrument might work here, or even something simpler.
- sbierwagen 3y agoVesuvius Challenge used the DLS I12 "High resolution imaging camera" mode: https://www.diamond.ac.uk/Instruments/Imaging-and-Microscopy/I12/Detectors-at-I12.html https://www.diamond.ac.uk/Instruments/Imaging-and-Microscopy... https://drive.google.com/file/d/1I6JNrR6A9pMdANbn6uAuXbcDNwjk8qZ2/view?pli=1 https://drive.google.com/file/d/1I6JNrR6A9pMdANbn6uAuXbcDNwj... The 2019 scans were done at 7.91 micrometer resolution with 88KeV monochromatic x-rays. Synchrotrons like DLS can also use their uniquely coherent x-rays to do diffraction imaging but I see no evidence they've tried that with Vesuvius scrolls. (Because the object is too thick?) That scan is 5.5 terabytes of .tif files. The 2023 scan apparently is at 3.24 micrometer resolution and four separate scans of 53/70/88/105KeV each, which should produce a vast, data processing-chokingly wad of data. If all x thousand scrolls need to be scanned at that detail then the Vesuvius Challenge people are going to be juggling a lot of hard drives.
- pimlottc 3y agoThis is really cool! Is the output resolution limited by the granularity of the CT scans?
- codeulike 3y agoOver 2000 years ago a chap called Philodemus sits in the library of a luxurious villa owned by a rich guy who likes collecting art and writing. He writes his thoughts on pleasure, and the relationship between the quantity of something and the pleasure that might derive from it. The scroll goes on the shelves with the others. He writes lots. At some point later the villa is covered by lava from mt vesuvius. 2000 years later we scan the carbonised scrolls with (basically) magic rays and use thinking machines to reconstruct what Philodemus wrote. I wish we could tell him. Sounds like he was a thinker, he would really appreciate it.
- clawoo 3y agoI have too wild imagination sometimes. I picture him with a shocked look on his face, similar to what one of the modern 'thinkers' would have if they forgot to clear their browser history and somehow someone restored it in the future. "You recovered... uh... everything?"
- bee_rider 3y agoThat recovery is much easier though, it is using the device as intended. And of course, rm just unlinks, doesn’t actually delete, so even going a step further and recovering deleted content is hardly magic. This is more like if, sometime in the future, they somehow successfully reconstructed a snapshot of our computers’ volatile memory by examining the power supply, or something ridiculous like that.
- nindalf 3y agoBig if. We’ll lose a lot of digital data simply because we won’t have the means to read it. CD-readers aren’t manufactured anymore in volume. It’s easy to imagine society in 40 years not having any CD readers handy but having a bunch of CDs they want to read. Now multiply that by all the funny storage formats we’ve created over the years.
- bornfreddy 3y ago
- nerdo 3y agoIncredible effort. Text of scroll is essentially "drink more Ovaltine" which is on theme.
- leoc 3y agoApparntly the bad news is that the remaining scrolls most likely contain yet more Epicurean philosophy, maybe largely from not-top-rated guys like Philodemus. (Apparently it's possible that the library actually is, or incorporates, Philodemus' personal library.) https://twitter.com/DrFrancisYoung/status/1754536306456027542 https://twitter.com/DrFrancisYoung/status/175453630645602754...
- dougmwne 3y agoThat’s quite premature, but even if we are looking at a personal library containing only personal writings, you’d be looking at a massive increase of information on the ancient world, like a neural map of a single ancient mind that contained all their experiences and thoughts. The worse case would be that it was 800 copies of the same scroll waiting to be sold off to other libraries.
- Leo_Germond 3y agoAll the 847 chapters of Philodemus fan fiction of MLP (my little Plato)
- r_klancer 3y agoThat's not necessarily bad news. Everyone interested in this story should read Stephen Greenblatt's The Swerve (https://www.pulitzer.org/winners/stephen-greenblatt https://www.pulitzer.org/winners/stephen-greenblatt). It traces the story of a Renaissance humanist who tracked down and translated the Epicurean philosopher/poet Lucretius' De Rerem Natura, which Greenblatt describes as portraying a strikingly modern way of seeing the world. In particular Lucretius and the Epicureans denied the existence of supernatural causes, were opposed to religious fear, and posited the ideas of atomism and biological evolution. Of course they're better known for their approach to living life, which Greenblatt shows is more sophisticated than sometimes caricatured, and which he portrays as a breath of fresh air compared to the oppressive moralism and hypocrisy of the Church at the time. (Jefferson and many of the American Founders described themselves as Epicureans.) He goes on to imply that Epicureanism was influential and widespread in the ancient world but suppressed by the early Church, so that we now know little of it. Anyone, one of the tantalizing parts of the book is where he describes the carbonized and unreadable Herculaneum scrolls, since they were the private library of a wealthy patron of the Epicureans. I think he thinks being able to read the scrolls will really change our understanding of the ancient world. And remember: if they hadn't been carbonized, they would have crumbled to dust. That's why we only have the texts that managed to get copied. (Anthony Doerr's Cloud Cuckoo Land is a novel about the survival and 21st century rediscovery of an imaginary Greek play, and ... I'll let you read it yourself - https://www.anthonydoerr.com/books/cloud-cuckoo-land https://www.anthonydoerr.com/books/cloud-cuckoo-land) (Apologies for any errors above, as basically all I know about this subject is what I read in the book!)
- dang 3y agoRelated: First word discovered in unopened Herculaneum scroll by CS student - https://news.ycombinator.com/item?id=37857417 https://news.ycombinator.com/item?id=37857417 - Oct 2023 (207 comments) The Vesuvius Challenge - https://news.ycombinator.com/item?id=35322809 https://news.ycombinator.com/item?id=35322809 - March 2023 (32 comments) Vesuvius Challenge - https://news.ycombinator.com/item?id=35169869 https://news.ycombinator.com/item?id=35169869 - March 2023 (32 comments)
- dang 3y agoFrom today there's also this article, which maybe goes into more background (I haven't checked): Can AI Unlock the Secrets of the Ancient World? - https://news.ycombinator.com/item?id=39261465 https://news.ycombinator.com/item?id=39261465 - Feb 2024 (1 comment) and this tweet which presumably covers the same ground as OP: The $700k Vesuvius Challenge prize has been won - https://news.ycombinator.com/item?id=39261933 https://news.ycombinator.com/item?id=39261933 - Feb 2024 (2 comments)
- low_tech_punk 3y agoLack of information retrieval technology ≠ loss of information.
- ImageXav 3y agoI was ridiculously excited when I first read about this in October (if I remember correctly) last year, when a few of the first results were beginning to pop out. I found the methodology fascinating. First of all the digital unwrapping of the scrolls, then the recognition that crackling in the paper was the sign of ink, and finally putting together a model to detect it, piece by piece. I need to look into the final repository to understand what exactly they did, but they seem to have used a TimeSFormer. I'm confused by this choice as I thought it was for video. How did they apply this to images? In the end though, what a wonderful day for archeology. These young minds deserve a huge round of applause for what they have achieved.
- deleted 3y ago[deleted]
- marcyb5st 3y agomy understanding is that the scan they did on the scrolls returned the layers themselves. Like so: ``` xxxxxxxxxx <- The surface of the scroll xxxxxxxxxx ... xxxxxxxxxx <- The bottom of the scroll ``` So, by tiling the image on the surface you get data that is size_x * size_y * n_layers. So, it can be seen as a video stream with size_x * size_y * 1 channel * n_layers where the layers replace the temporal dimension.
- namaria 3y agoThey explain it on the methodology sections. The scans result in a stack of tiff images that can be rendered as videos of the scan or as 3d models.
- dan_mctree 3y agoThe scans used for the grand prize look like this : https://scrollprize.org/img/grandprize/scroll1.mp4 https://scrollprize.org/img/grandprize/scroll1.mp4 It's a cut through the scroll, with the time dimension in this video representing the location of the cut along the scroll lengthwise. As you can see from the mess it's far from trivial to find the surface of any of the sheets in the scroll, often they layers are blended together messes. You may be thinking of the scans they used of an unwrapped sheet, those were as you describe and were used to help figure out methods for the real challenge.
- ComputerGuru 3y agoBetter but less technical writeup: https://www.bloomberg.com/features/2024-ai-unlock-ancient-world-secrets/?accessToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb3VyY2UiOiJTdWJzY3JpYmVyR2lmdGVkQXJ0aWNsZSIsImlhdCI6MTcwNzE0MTc1MCwiZXhwIjoxNzA3NzQ2NTUwLCJhcnRpY2xlSWQiOiJTOERZV0lUMEFGQjQwMCIsImJjb25uZWN0SWQiOiI0Q0RGQzk1QkQ5NjI0RjAyOEYwREMxOTNDRDM2RDBGMiJ9.1FLLhYSi9d9XWAt6vzeuo9zV0hxAsheISAImpNGdfLA https://www.bloomberg.com/features/2024-ai-unlock-ancient-wo...
- iandanforth 3y agoI was recently reading the Greek Myths series by Stephen Fry and he makes a point of how there are so many stories we know of but have been lost from ancient texts. Stories and authors that were famous enough to be mentioned by multiple other authors but which themselves have been lost. This collection of scrolls could contain some of those lost stories and the possibility of that is terribly exciting.
- colechristensen 3y agoNot just works of fiction or mythology but also histories, and works of science and philosophy.
- eszed 3y agoI want Aristotle's treatise on Comedy, and if I'm allowed to be terribly greedy, just one more play by Aeschylus. This is the most exciting thing in the world to me right now, these scrolls, along with the thought that there might be literally thousands more still in the ground.
- CamperBob2 3y agoMight want to put on a pair of gloves and a respirator if you find that Aristotle volume. Some people were pretty offended by it, I understand.
- arbuge 3y agoAnd those are just the known unknowns. Besides those, to borrow from Rumsfeld, there are the unknown unknowns.
- 0x_rs 3y ago>stories we know of but have been lost from ancient texts So many there's a lengthy list on Wikipedia about it. It's fascinating reading ancients casually referencing works that we otherwise know nothing else about. Without the careful, laborious copying (often imperfect) over the centuries most things would've been lost completely. There's also other works such as maps that did not survive, the Tabula Peutingeriana for example is thought to be a derivative work of one commissioned by Augustus of the known world at the time (to Romans) and of which there's a few mentions in some works by historians at the time. https://en.wikipedia.org/wiki/Lost_literary_work https://en.wikipedia.org/wiki/Lost_literary_work
- thom 3y agoSo excited to see how much text is recoverable and to what extent this can bolster our collection of Epicurean writings!
- yieldcrv 3y agotl;dr The general subject of the text is pleasure, which, properly understood, is the highest good in Epicurean philosophy. In these two snippets from two consecutive columns of the scroll, the author is concerned with whether and how the availability of goods, such as food, can affect the pleasure which they provide. Do things that are available in lesser quantities afford more pleasure than those available in abundance? Our author thinks not: “as too in the case of food, we do not right away believe things that are scarce to be absolutely more pleasant than those which are abundant.” However, is it easier for us naturally to do without things that are plentiful? “Such questions will be considered frequently.”
- cooper_ganglia 3y agoThis is amazing and very, very exciting! It is wild to me, though, that if I have an SSD fail it's essentially unrecoverable, but a 2,000-year old, rolled-up, lava-burnt scroll of Papyrus can be read using Technology™! I love to see it!
- BlueTemplar 3y agoAre you sure it wouldn't be recoverable using something on a similar level than they did (particle accelerator scanning) ?
- jacobwilliamroy 3y agoI don't know if anyone who worked on this project is going to read this but if you are: good job! This looked like it was really hard.
- m3kw9 3y agoPretty cool engineering to pull this off
- dylan604 3y ago"4 passages of 140 characters each, with at least 85% of characters recoverable" oh good grief, even back then, we were limited to this value
- jaredhallen 3y agoI'm sure the original authors didn't expect to be incinerated by a volcano (and I'm also sure that the future legibility of their writings would be the very least of their concerns!), but it really bends the mind to imagine their reaction if they could have known how this would all unfold (unroll? Sorry...)
- laser 3y agoThis is amazing, sci-fi-like-reality. I wonder if they pull off an auto-segmentation breakthrough if the techniques might apply in other areas, like automating neuron mapping (eg. see EyeWire).
- lostlogin 3y agoI was worried it would contain something boring like tax records, but it's even worse than that. Fingers crossed it gets a little more interesting than the average social media post.
- glfharris 3y agoI'm pretty sure there are academics who would bite their own legs off for tax records from this period.
- kilroy123 3y agoIsn't that what most ancient Roman texts we do have are? Tax records.
- ffgjgf1 3y agoDo we have any or any significant number of Roman tax receipts? Most Roman/Greek texts that we have had to survive until ~1000 AD. If a text was available back then there is a reasonably good chance that we have it (or rather a medieval copy of it), why would anyone waste time copying tax receipts though?
- verisimi 3y agoI still think this methodology can be tested by creating equivalent carbonised scrolls, where you write some specific text and check the result against what you wrote. Ie you run your test artifacts through the scanner and software, and the process should return you your test text that you wrote before carbonising the scroll. At that point you can be assured that the process is not making stuff up from noise. But, without that process, you can have no comfort than what is occurring is valid. The software might reliably see a letter in some noise, but so? It doesn't mean the letter is actually there... One can't verify the scroll, and one hasn't verified the process.
- verisimi 3y agoI think this suggestion of mine was downvoted last time too, but I don't get why! One would always want to test stuff in software development, especially if it was fraught and can easily be tested. Mostly there are no tests to be undertaken in history -hence it it's so much hearsay. But here is an opportunity to gain some genuine certainty, in a way that is normally unavailable! The implementors of this method should absolutely test their process!
- ffgjgf1 3y agoAFAIK the scanning process is pretty expensive
- namaria 3y agoYou're being downvoted because they literally did what you suggest already.
- verisimi 3y agoIt does not say that in the article - this one and on a previous HN submission where I read about this. However, you are right - if you go to Tutorials and Scanning there is reference to the creation of a 'campfire scroll'. And now we have some detail..... and the detail is problematic. In tutorial 3, halfway down this page (https://scrollprize.org/tutorial3 https://scrollprize.org/tutorial3) there is a before and after comparison of the scroll. They ask a question "If you look back to the last page of the campfire scroll (before carbonization), can you see which area of the scroll this segment came from?". This is mean to be obvious to answer - and one does have the 2 images to compare. My thoughts on the comparison is - yes at a glance the there is a section that appears to match up - the 'angular squiggle' next to the @ symbol. However, if I look more closely at the 'angular squiggle' I see features and spaces in the generated image that do not correlate with the photo before carbonisation. The troughs are too deep, the spaces are too big. It seems to be a superficial similarity only. I wish I could show what I mean by referencing the images.. But I will provide links for others to see what I mean when I say there is a superficial correlation only. https://scrollprize.org/img/tutorials/vc-segment.png https://scrollprize.org/img/tutorials/vc-segment.png https://scrollprize.org/img/tutorials/campfire-last-page.jpg https://scrollprize.org/img/tutorials/campfire-last-page.jpg Final thought - why not map the 2 images one over the other on the site? Why ask a leading question rather than provide a proof? I hate that kind of presentation - it smacks of providing enough information for someone to make an incorrect snap judgement.
- gmd63 3y agoGiven that AI is able to hallucinate, how can we be convinced the results are accurate? Did they create a new scroll, burn it, and compare the results to what was actually written?
- leeoniya 3y agoread the article
- gmd63 3y agoThanks. From the article, for anyone else skimming who had the same question: Technical reproduction. The Vesuvius Challenge Technical Review Team reproduced the winning submissions manually. We made sure to clearly understand every part of the code, and that when we run it independently we get similar output images. Since all code and training data is now open source, you can do the same! Multiple submissions of the same area. You might have noticed that all submission images above show the same area of the scroll. This is because we released 3d-mapped papyrus sheets within the CT-scan (“segments”) created by our segmentation team, which were then used by all contestants. The resulting output images — created by different ML models and training labels — have produced extremely similar results. This holds not just for the winners and runner ups, but also for the other submissions that we received. Small input/output windows. The ink detection models are not based on Greek letters, optical character recognition (OCR), or language models. Instead, they independently detect tiny spots of ink in the CT scan, the writing appearing later when these are aggregated. As a result, the text appearing in the images is not the imagined output of a machine learning model, but is instead directly tied to the underlying data in the CT scan.
- leeoniya 3y agotl;dr: cross-validation between competing submissions
- WorkerBee28474 3y agoThe ML models were trained without knowledge of the scrolls' language. The models extracted images, and human experts were able to read the images as text. There was no text corpus fed in that could be leaked into the output.
- dekhn 3y agoWe truly live in a golden age of physical and mathematical discovery. To get here required many thousands of years of technological development. The developers of the transformer, as a group, should win some sort of significant prize; it has had more impact in a short time than anything I've seen before. Will we find better architectures in the near future?
- laichzeit0 3y agoJust don’t say something like this out aloud near Gary Marcus.
- EnigmaFlare 3y agoWhy do you say thousands of years? Almost no technology was invented before the industrial revolution. But if you count those meagre dribbles, you might as well go back further and count pre-historic technology. I'd say pretty much all that stuff was trivial compared to what came after and humans spent most of their time working towards more zero-sum objectives than technology.
- ryneandal 3y agoHerculaneum was one of the highlights of my trip to Italy with the wife. I didn't realize the scope of just how much ash and soil had to be removed for excavation. It was dozens of meters [1]. It's an absolute shame that the site is given a fraction of the attention that Pompeii receives, I thought it was vastly better preserved and truly awe-inspiring [2]. I highly recommend spending a few hours wandering the site, it is an absolute wonder. 1: https://www.icloud.com/photos/#08dJAA5eM9jpbhlEa3fzkl5ng https://www.icloud.com/photos/#08dJAA5eM9jpbhlEa3fzkl5ng 2: https://www.icloud.com/photos/#076Pof4FziA7WgcI8hZrGZmzg https://www.icloud.com/photos/#076Pof4FziA7WgcI8hZrGZmzg
- beautron 3y agoI enjoyed the attention given to Herculaneum in a computer game called Rome: Pathway to Power (released in 1992). You start the game as a slave who has to escape Herculaneum before Vesuvius erupts. I loved the game as a kid. It's sort of like an isometric immersive sim (with a clunky interface). It got me interested in ancient Rome. I hope to visit Herculaneum some day.
- inglor_cz 3y agoThe modern Italian town of Ercolano lies just over Herculaneum, so excavations of the rest of the ancient town are a bit tricky. Only about a quarter has been excavated so far, in contrast to Pompeii, which are two-thirds out.
- est31 3y agoThis is great news for preservation efforts as we have a large set of stuff that can't be opened. I wonder what this means for the maya codices, many of which are in similar shape: https://en.wikipedia.org/wiki/Maya_codices#Other_Maya_codices https://en.wikipedia.org/wiki/Maya_codices#Other_Maya_codice...
- mNovak 3y agoThe scroll segmentation looks like it'd be a very manageable task for distributed volunteer work too, at least as a kickstart if that's really the bottleneck now. Just a 100 or 1000 halfway dedicated volunteers (small by internet standards, for a useful and simple task) could make a big dent. Don't know how many scan layers are in a single scroll, but for instance Project Gutenberg's proofreading network has processed millions of pages, one by one.
- imranq 3y agoThe methods are incredible, but it seems like the text is ordinary and nothing to write home about
- s0rce 3y agoNot too surprising, but I like the random ancient texts https://en.wikipedia.org/wiki/Complaint_tablet_to_Ea-n%C4%81%E1%B9%A3ir https://en.wikipedia.org/wiki/Complaint_tablet_to_Ea-n%C4%81...
- f5ve 3y agoThere really is one for everything: https://xkcd.com/2758/ https://xkcd.com/2758/
- hoten 3y agoI dunno, I found it pleasurable.
- namaria 3y agoWe don't decipher ancient texts to have our minds blown about the universe but to know more about what life was like back then and to conduct historical research about it.
- empath-nirvana 3y agoSturgeon's law would imply that the overwhelming likelihood that any random unknown text we find from the ancient world _in situ_ is likely to be not be very good. Most of what we have left from the ancient world is material that people felt worth copying for _centuries_ after they were written. That's a fairly amazing quality filter and gives people a skewed perspective on the overall quality of material written at the time. A selection of work from a random library has also likely to be filtered for quality to an extent, and for the most part, the "good stuff" is going to be works that we _already_ have copies of or fragments from. Anything we don't already have a copy is is most likely going to be something that there weren't many copies made of, and usually there's a reason for that. Which isn't to say that anything we find isn't going to be interesting for other reasons -- even bad writing is going to be incredibly useful for historical research.
- ptelomere 3y agoMy God, once in a while, you need to read something like this, reflect, ask the question "can I do better in my work ?", such an inspiring story of technical feat, persistence, and ingenuity.
- manyty 3y agoNetflix documentary soon please! Big ups for the winners - this is so cool and hopefully can be replicated for deciphering many other lost manuscripts.
- foota 3y agoI could help but wonder while reading this how everyone involved would feel if the "last step" involving the papyrologists was automated as well.
- newzisforsukas 3y agoAmazing? As whomever made it would have likely created an incredibly powerful generally useful tool. That would be much more exciting than probabilistically distinguishing layers of ink in soot.
- darkwater 3y agoEven if I really do think this is an amazing effort, I'll spoil the techno-optimist party here by noting that's a bit sad that this kind of achievements are pulled off only by rich patrons that became millionaires doing another unrelated thing and they just want to scratch a personal itch/curiosity. Yeah, patrons and maecenas have been around forever, but I would really prefer that we as society were mature enough to achieve such things collectively. We did it, at least for a few decades and even if coming from wrong incentives, with space exploration.
- noduerme 3y ago>> “as too in the case of food, we do not right away believe things that are scarce to be absolutely more pleasant than those which are abundant.” However, is it easier for us naturally to do without things that are plentiful? “Such questions will be considered frequently.” This reminded me, since they're scarce but also abundant... Has anyone actually eaten these giant waterbugs at Nue in Seattle? Is that like, a reasonable thing to subject a date to?
- junon 3y agoWow. I remember this being announced and thought it'd be a while or that it wouldn't be possible. Very happy to be wrong!
- kilianbutler 3y agoThe rewriting history comment from Nat Friedman is super interesting. It'll be amazing once this data passes into the hands of historians.
- vjk800 3y agoI first looked at the results and thought "this is kinda cool". Then I proceeded to read about the whole competition and how thoughtfully it was organized and thought that "this is extremely cool". I wonder what else could be achieved with such a well designed incentive program.
- azalemeth 3y agoAn awesome achievement. I don't know if it's possible but I wonder if they could do energy-resolved CT and try to look for a differential absorption edge in that ink. I have no idea what material it is actually made of, but if the IR photo is different, then there's a nonzero chance they could generate quite significant contrast by hitting one of the K-shell edges of a characteristic ion (iron?) present in the ink but not in the papyrus itself. I've been to Diamond in the past but don't know enough about this experiment. Some of them are definitely energy resolved. Utterly brilliant. I'm so glad it appears to be a bit of scholarly writing too, which is what I know most classicists secretly love!
- versteegen 3y agoThe ink is carbon-based, without iron. However there's plenty of contrast when viewed in infrared.
- DeathArrow 3y agoGreat things can be achieved if some smart people organize others to work towards common goals. I think the biggest success isn't the recovered text but organizing such an endeavor with such success. Totally inspiring!
- gadders 3y agoI wish billionaires would spend their money on this, rather than the crap that the WEF comes out with.
- rabbits77 3y agoTIL that I need to now shred my secret documents before throwing them into the burn barrel. Also, possibly, mix the resulting ashes into a slurry and dump into the sea.
- Onlyartist9 3y agoMy initial concern was that the details may have been hallucinated but it seems they accounted for that as well!
- l33tman 3y agoThis is awesome. But it makes me sad to think that our current DIGITAL artifacts might not survive even 20 years, unless explicitly archived. I'm pretty sure a harddrive or SSD thrown on a landfill won't be able to be resurrected in 2000 years even with alien tech, but maybe? Would be cool if someone studied this :) But the main problem is probably not reading thrown out drives - it's that stuff is just too transient nowadays. People put something up on the net and decides to withdraw it next year. archive.org can't store the walled gardens, and even if a social network for example wants to archive stuff they might not be allowed to for legal issues (not that it has prevented them before, but anyway...)
- lnxg33k1 3y agoIt’s weird to see my home town on hacker news, kinda nice considering its historically rich but not technically relevant