5 ms·
EY Canada published a cybersecurity report and most citations were hallucinated
- raro11 4mo agoWhat a horrible page to navigate
- bokkies 4mo agoFeels like my scroll is hallucinating
- umpalumpaaa 4mo agoMy iPhone automatically enabled reader mode - I disabled it to see what you are referring to and I agree…
- snailmailman 4mo agoOn mobile, It’s hijacking my scroll in such a way that I literally cannot move further down the page. And “reader mode” is only showing me the first paragraph or so. I’ll have to try again later on desktop. The content looks interesting but it’s literally impossible to read. I cannot get past the section that introduces Ernst and Young.
- 1000100_1000101 4mo agoOn desktop it keeps adding forced pauses to scrolling, of varying sizes, and you need to scroll down a between 1 and 10 pages worth to begin scrolling again. It might "work" just fine on mobile (or not) but you may have stopped trying before reaching the point of re-scrolling, because it's insane.
- lelandfe 4mo agoI recommend just clicking and dragging the actual scrollbar on desktop for this one. Wild
- snailmailman 4mo agoI eventually managed to get far enough into the article that I thought I saw the main stat - the stat that 26% of the citations were hallucinated. Then the scroll threw me back to the top again and I gave up entirely on reading from my phone. Coming back later on desktop, I see that the percentage keeps climbing the further you manage to make it down the page. The real stat is 60% of the citations were hallucinated.
- kavok 4mo agoVery difficult to use on mobile.
- bbddg 4mo agoI'm usually annoyed by people complaining about scroll hijacking on HN but this site was a new level of bad.
- deleted 4mo ago[deleted]
- nntwozz 4mo agoThis is a whole 'nother level of user hostility, never before have I seen anything like it.
- canyp 4mo agoNon-linear feedback with literal stalls, yikes. Some people should not be allowed to make a website.
- IshKebab 4mo agoThey put a lot of effort in to make it that bad!
- csomar 4mo agoI've stopped reading because of it. I can't scroll. Was this thing vibe-coded? Funny they are picking on EY for not reading their reports but it looks like they didn't test their website.
- chaidhat 4mo agoMaybe they should stop pushing these bankers to do 48 hour shifts…
- 331c8c71 4mo agoThese are not bankers, but the culture is still bonkers
- nilirl 4mo agoSite is gross to scroll on mobile
- cwillu 4mo agoIt's gross to scroll on desktop as well.
- AshamedBadger56 4mo agoIt's gross to scroll on tablets as well.
- ilamont 4mo agoThe problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day. Anyone remember that item a few months back about Amazon now having senior engineers vet generative AI output (https://news.ycombinator.com/item?id=47323017 https://news.ycombinator.com/item?id=47323017)? I had to LOL when I read that. These folks are already slammed. And the idea that Amazon would allow human bottlenecks to multiply across projects and underlying infrastructure development is ridiculous.
- ChrisLTD 4mo ago> the idea that Amazon would allow human bottlenecks to appear across projects and underlying infrastructure is ridiculous. Why?
- SoftTalker 4mo agoAmazon is fairly well known to ruthlessly optimize every process. So if they're having humans proofread what the AI produces, they must have found that to be necessary.
- ChrisLTD 4mo agoYeah, that's what I think too. They aren't going to care about optimizing a process that leads to poor results.
- bluefirebrand 4mo agoAmazon is not immune to making mistakes
- _puk 4mo agoPart of the problem: you get given a complete document to review after it's been fully baked. I'm pushing the need for basic engineering principles across whole organisations. You wouldn't give an engineer 1000 lines of code to review without the original spec of what you're trying to achieve for context (at a minimum, ideally the reviewer was in the room when the work was introduced, and has full context). So, these docs, they're given as an all or nothing. Do you push back on the 39th metric that is defined to the utmost detail? Or just resign yourself to the fact that it is what it is? A one (6 is the goto if we're talking Amazon?!) pager.. "this is what I am proposing" at least gives the skeleton of the idea to push back at the general shape of the idea, refine it, before all the emotional investment of your precious report being complete. Y'know.. the traditional product running through the spec in a SCRUM* environment.. the engineers doing proper code reviews.. * Yes SCRUM is dead, but that's another thing.
- mapontosevenths 4mo agoEY has been quietly laying people off for the last year solid. It's unsurprising that trying to do more with less results in lower quality.
- onlyrealcuzzo 4mo agoThe interesting thing is... There may be a lot of demand for do-nothing services. A lot of corporate work is just do-nothing box-ticking. Boss: get me a report about X, so I can give that report to my boss who won't read it. You: E&Y, please get me a report. Here's $200k.
- bombcar 4mo agoThis underlying much of the non-coding AI revolution (and some of the coding perhaps) - so much corporate activity is write-only and never read.
- fragmede 4mo agoThe trope about external consultants is that your VP brings them in to review the company, and they talk to everybody and write a report on how to improve the business, and the report says exactly what you've been telling your VP but they've been ignoring you.
- 2fff 4mo agoYou are closer to the truth :) they are not simply paid to do nothing. They are paid to do dirty work.
- mapontosevenths 4mo agoThey are paid to justify decisions executives have already made. It's often referred to as due diligence, but in practice these reports mostly just allow executives to tell the board it wasn't their fault if it goes wrong.
- deleted 4mo ago
- Our_Benefactors 4mo agoHoly horrible UI
- cmiles8 4mo agoThis sort of thing is a complete embarrassment to a firm like EY, where people are paying them a lot of money for advice. They’ve basically demonstrated that their market leading research is just someone asking questions to ChatGPT. If you ever needed evidence to not buy “advice” from such outfits, this is exhibit one. Hopefully they at least fired the partner that published this steaming pile of AI slop.
- ralph84 4mo agoExecutives pay them a lot of money to launder blame. If a project fails after consulting EY, well, what can you do. If a project fails without consulting anyone externally, it's obviously a failure of the executive.
- elmomle 4mo agoExactly--they're paid a lot of money for their reputation, which is valuable in offering cover for politically difficult decisions. This was certainly net-negative for E&Y's reputation.
- jimnotgym 4mo agoThe Big Four have become a shadow of their former selves. They have become so risk averse that their advice is already incredibly generic and non-actionable. I think their audit work is in a downwards spiral. Audit has become so competitive that they are struggling to find ways to make it cheaper. They have become slaves to reducing the hours booked, and the rate of those hours. To do this they substitute less experienced people all the time. You used to be able to chat with your partner about an issue you have coming up, now you get their assistant if you are lucky. By chasing 'efficiency' they have lost their value-add. Now the first time the partner has looked at your file is right before the clearance meeting, and they spot issues that should have been picked up earlier and tested on the day you should be signing. So you end up doing it all again. I'm trying to coin a term for the inneficiency caused by chasing efficiency.
- busterarm 4mo ago
- galaxyLogic 4mo agoI don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens? I don't know but I would expect it to be realtively easy for an LLM to detect "hallucinations".
- operatingthetan 4mo ago>I don't quite get it why they can't take another LLM and vet the output of the first with the seond one. I think this may be part of the problem. The actual humans creating the report don't have the expertise to know which one to trust. At least that was what consulting was like in my experience at a similar firm.
- TZubiri 4mo agoBecause they used LLMs to do the work. What you are suggesting is to use the LLMs to create more work, which is counter to the shortcut they were trying to take.
- galaxyLogic 4mo agoGood point with some irony. Thye don't want to do a better job they want to do an easier job. But a company like E&Y should realize shortcuts like these don't work. And their customers are paying them.
- voxl 4mo ago[flagged]
- mindcrime 4mo ago> I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Yes, this technique and its variations[1][2] "work" but it's still not 100% perfect. And it's not as widely used it might be because, among other reason: a. it takes longer to implement b. it costs more (more tokens spread across multiple llm calls) c. higher latency (getting an answer takes longer due to multiple llm calls involved) d. the final answer is probabilistically more likely to be correct, but is still not guaranteed to be error free, so you can never fully escape the need for Human in the Loop. [1]: https://en.wikipedia.org/wiki/LLM-as-a-Judge https://en.wikipedia.org/wiki/LLM-as-a-Judge [2]: https://github.com/karpathy/llm-council https://github.com/karpathy/llm-council
- galaxyLogic 4mo agoI don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens? I don't know but I would expect it to be relatively easy for an LLM to detect "hallucinations".
- gdulli 4mo ago"Why don't they make the whole plane out of the black box???"
- jonwinstanley 4mo agoDid someone hallucinate how scrolling is supposed to work on a web page?
- rao-v 4mo agoWhat’s strange about how things have developed is that this report 12-18 months ago would have been a massive scandal and would have caused durable brand damage. Now nobody will remember or notice.
- deleted 4mo ago[deleted]
- mentalgear 4mo agoThis proves (again) one think for sure: The "Big x" Consulting Firms were always BS - and now them generating all their work themselves using LLMs just profs that their 'clients' can just skip their Million Dollar fees and just ask the LLM directly.
- contingencies 4mo agoBasically the entire consulting industry should die due to AI. Performative executives of yesteryear that constantly need external validation and direction and operate through hive mind and groupthink are weak and will die. I believe some of the biggest problems in today's business leaders are an inability to be open to new information, to think across traditional professional boundaries, or to ask meaningful questions. AI simply exposes this unapologetically. Bad management (this includes most government): up your game or get out of the way. Sycophantic consultant firms: die. The Economist should do an article on this.
- scotty79 4mo agoIf they can't be bothered what they are putting out, do you think that before AI, what they wrote had any merit?
- meibo 4mo agoWow, your mom lets you have TWO scrollbars?
- cwillu 4mo agoIs there any source with just the plain text? The css styling is headache inducing and reader mode doesn't work or has been defeated.
- _tk_ 4mo agoSame goes for lockdown mode on iOS.
- addandsubtract 4mo agoFirefox has a handy "Reader view" (Opt + CMD + R on Mac) that you can activate to get a stripped down view of just the text on the page. Unfortunately, it also removes the images which contain some of the sources they use.
- deleted 4mo ago[deleted]
- solomonxiexie 4mo agoThe scrolling really hurts me, turning to Reading mode broke as well.
- throwrioawfo 4mo agoYou're not actually meant to _read_ these reports.
- jiveturkey 4mo agoBut the chatbots/aggregators do, and they accept the reports as fact. As is noted in the conclusion.
- le-mark 4mo agoThe real comedy is seeing this garbage come down from senior management, clumsy prompting, hallucinated garbage that’s all fluff and zero actionable information, zero real informed analysis. “See this analysis of our support issues from jira, we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix anything. I’ve seen this more than twice now; needs a name. Garbagemaxxing?
- CuriousSkeptic 4mo ago> we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix So a net positive then?
- zb3 4mo agoStop messing with the scroll, I thought there was something wrong with my mouse wheel. Why are you doing this?
- zelphirkalt 4mo agoI wish we could just stop destroying people's jobs and lives using AI. The statistics I have heard quoted say, that merely 25% of the people actually like their job. Meaning they like doing what they do for its own sake, not because it gets them money, which they desperately need to live. I get it, most people don't want to do the work. But can we stop ruining the jobs of people, who are actually dedicated to their job and would like to keep doing their job properly? But I guess since EY is a CYA hedge anyway, no one really cares about whether the reports are hallucinations or not. Someone high up spent money on EY, so that they can justify some decision and won't be held responsible that much, when it turns out the decision was shit. All that matters to them is, that it has the appearance of something genuine and then they can base the decision on what they receive from EY, which better be what they already wanted to hear/read anyway.
- krapp 4mo ago>The statistics I have heard quoted say, that merely 25% of the people actually like their job. Meaning they like doing what they do for its own sake, not because it gets them money, which they desperately need to live. Even people who like their jobs work because they need money to live.
- zelphirkalt 4mo agoMy point is, that people who want to do their job properly are less likely to sling AI slop and be found out to do that, and that I wish we could stop destroying their jobs or lives, to chase stakeholder wet dreams of the companies they invested in letting go almost everyone.
- wg0 4mo ago"All jobs would be gone next month." ~ A greedy, dishonest and unethical capitalist.
- biosboiii 4mo agoI guess this is a great report, but the parallax landing page shenanigans disrupt my reading flow, you cannot easily scroll back to get a overview of the key facts, so I stopped.
- bakrisolo 4mo ago[flagged]
- sourcecodeplz 4mo agoWas the title updated? from "ernst & young" to EY Canada. Why?
- rescripting 4mo agoThey changed their name to from Ernst & Young to EY in 2013.
- smartmic 4mo agoNot by me, but by the mods. They also changed from "full of hallucinations" to "and most citations were hallucinated". Maybe a rep from "EY Global" filed a complain ;)
- atom058 4mo agoErnst & Young officially change their name to just "EY" some years back. Unfortunately, they forgot to check the name beforehand, and it turned out that "EY!" already existed: a gay porn magazine. Yes, this really happened.
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- 0898 4mo agoI did some ghost writing for EY. I wrote cheat sheets about international tax transfer pricing, mining and metals, and life sciences for its then CEO Mark Weinberger. I had no experience and knew absolutely zero about any of those sectors.
- themafia 4mo agoTitle changed to remove "Earnst & Young". Why? It seems deferential to an entity that, in this case, certainly doesn't deserve it.
- FearNotDaniel 4mo agoProbably because (a) that’s not their name any more and (b) when it was, that’s not how you spell it
- themafia 4mo agoWhich is interesting because a) the site title includes it and b) I obviously made a common mistake which has nothing to do with the point at hand.
- henry2023 4mo agoI think it’s important to note that EY report’s overall quality has not been affected by GenAI.
- FearNotDaniel 4mo agoOff topic but: the scroll mechanism on mobile is so horribly irritating and unpredictable that I just can’t be bothered fighting against it to read what sounds like at least a mildly interesting article.
- dragonfax 4mo agoNo, it's like that on desktop too.
- yieldcrv 4mo ago> In late 2025, EY Canada published okay that makes me feel better, I think January's frontier models and beyond are better at this but check your sources folks
- aneutron 4mo agoFix your website. Drop the shitty Javascript animations. Jesus these things were solved in 2014 with D3JS and jQuery.
- s0rce 4mo agoScrolling this page is terribly awkward.
- tipsytoad 4mo agoWho designs a website like this?
- tamimio 4mo agoThose are who rejected you for a job you applied for.. AI amplified the dunning kruger that unfortunately real experts in their field are overlooked now, because a wall of text with numbers sounds and look professional enough. Any person with above average knowledge on a specific topic, can tell when AI starts hallucinating and making things up, or at least introducing new problems due to complexity added rather than solving it, that’s my observation using all top tier ones too, it’s like they are designed to solve a problem regardless so they start making things up or piling workarounds, a person with no deep knowledge in that topic will just copy it all and call it a day. Just yesterday, I asked claude 4.8 on something specific that I know the answer for, it had a long list of solutions that none were close to the real answer, when I replied with the real answer and pushed back, I got the famous quote “you are right, thanks for pushing back”.
- deleted 4mo ago[deleted]
- dwa3592 4mo agowhat an absolute garbage of a website to navigate. zero out of ten.
- jiveturkey 4mo ago> Instead of releasing our results all at once, we're going to focus on one report at a time. This approach both prevents individual examples being overlooked and allows us to illustrate the negative impacts of vibe citing on research quality and public trust. Not to take away from the actually great reporting here, but what they mean is, This approach allows them to milk it for as many clicks as possible.
- bleepblap 4mo agoFucking hell how is this website so unusable.
- ama_built 4mo ago[flagged]
- motohagiography 4mo agoPeople don't get it, this is marketing an example of what they could do for you. They can produce reports that say what you want to say, filtered through third party diligence and E&O policies, then take flak and blowback for tough policy choices. For the client there will be no consequences. It's not just ai slop or garbage, it's what makes them well worth it. Slop signalling may be the new power play. Nothing quite says "FU" like a low effort AI hallucinations.
- deleted 4mo ago[deleted]
- BLKNSLVR 4mo agoErnst & Young again proving they're leading in the race to the bottom. Why would anyone trust these large contractor companies enough to pay them the huge amounts of money to have juniors learning the ropes on their dime? "Customers" were the content the juniors were trained on in the same way that scraped internet data is what LLMs are trained on, except Customers were paying for the privilege of being 'scraped'. Now there are no juniors, just LLMs being asked questions that aren't specific enough, and assuming the answer is one-shot correct. It saves E&Y lots of money though, and their (confusing) reputation will provide a surprising amount of momentum such that plenty of work will keep rolling in for a few years to come.
- emeril 4mo agothis is not surprising at all - much of Big 4 is BS
- ChoGGi 4mo agoVibe coded scrollbar?
- Alifatisk 4mo agoHow does such thing even happen? I know for example in Qwen Chat or Perplexity, they produce citations on at the end of each generated sentence. So I can hover my mouse over each citation and see from which website that was scraped from. Did they just prompt ChatGPT with no web search and copy-pasted it?
- solomonxiexie 4mo agoThere is a big chance the web page itself was vibe coded, and the author was not bothered by it.
- gcollard- 4mo agoI’d also be interested in seeing the rate of hallucinated citations from the pre-ChatGPT era. I’m not entirely convinced this is purely an LLM-related issue. I’ve definitely come across countless misattributed citations in big four reports long before generative AI became widespread.