6 ms·
it seems to me that a 1.3MB file is a perfectly reasonable size to include in a webpage, and much better than a regex for this purpose - a file containing numbe
by notatoad 5y ago
it seems to me that a 1.3MB file is a perfectly reasonable size to include in a webpage, and much better than a regex for this purpose - a file containing numbers is maintainable without having to also maintain a test suite to ensure that your regex is still valid every time the company is assigned a new phone number.
maybe a server-side lookup would have been a little bit nicer than shipping the whole list to the client side, but a static client-side app can be stored in the CDN layer, doesn't require provisioning any server resources, and sidesteps any security concerns about servers processing user input.
it would have been nice if they'd cleaned up the user input before querying the dataset, but overall this looks like a clever solution to release a feature with the smallest risk and maintenance profile possible.
- andy_ppp 5y agoI think with gzip how much is the total < 100kb?
- Recursing 5y agohttps://www.barclays.co.uk/content/dam/json-files/TelephoneNumberChecker_26_03_2021.json https://www.barclays.co.uk/content/dam/json-files/TelephoneN... I see 190kb transferred in chrome dev tools. I fully agree that the criticism in the article is not warranted
- bellyfullofbac 5y agoHow about the cricitism of the tool not caring about +44 or non-numeric characters? It's still lazy programming...
- andy_ppp 5y agoWe’ve no idea how difficult the work environment at Barclays is, I don’t personally feel you should judge other people’s work unless you know exactly what went on there. Clearly there would be changes I would make (in a code review hopefully) but I’m not sure digging people out on hacker news is going to help.
- scrollaway 5y agoSeveral years ago, along with some cofounders, i created a Hearthstone website / database (hsreplay.net). If you look at the card database it's incredibly fast once loaded because all the filtering and displaying etc happens clientside. It downloads a 25-something megabyte JSON (which compresses extremely well), cf. hearthstonejson.com. Downloading a big JSON file locally is underrated. I implemented the exact same strategy on https://overframe.gg https://overframe.gg, a Warframe database. It allows a fantastic user experience ... the tradeoff being the slow loading time (which is invisible to most people on good connections). Overframe was a SPA though and had to lose the approach on most pages because of SEO. The usage were too different. Before that every single page on the site was clientside and available immediately after initial load. For a tool though, especially one that relies on user input (and thus will load in the background before your user starts interacting with it), it's a great approach as long as your data is somewhat bounded.
- aftbit 5y agoOh hey you made Overframe? Thanks for that! I had that open for at least a few minutes every day for about six months during the pandemic.
- scrollaway 5y agoCouldn't have done it without Semlar, the current maintainer (fun fact, i never played Warframe, this was a job for a client/friend). Glad it's of use to you!
- 1123581321 5y agoAgreed. I’ve written a lot of B2B tools and locally storing a copy of lookup data is perfect when appropriate due to the generally smaller dataset sizes. Back in the day I’d even do super fast geo proximity locally by including a json of the locations of first three digits of zip codes and truncating the zip code of the user to make the comparison.
- wwwaldo 5y agoWouldn't it be better to perform a server side lookup to avoid leaking the full list of currently used Barclays numbers in plaintext to fraudsters? The fact that there isn't any user input sanitization makes me inclined to believe that this implementation decision may not have been well-thought-out (which I'm guessing was the author's point exactly)
- notatoad 5y agowhen you're offering a service designed to expose a list of numbers, there's no way to design it in a way that doesn't expose that list of numbers. given enough time, a motivated person will extract all the data. most likely, this page started out as a list of all 74k phone numbers, and at some point somebody built a lookup tool on top of it. there's no reason to assume that keeping the full list secret was ever a design goal.
- mnd999 5y agoSure, but Barclays made it trivial.
- code_duck 5y agothat is mentioned at the end of the article. >why not POST the number to a service which can be updated?
- edent 5y agoI guess you don't pay per MB for data? Or live in an area with poor connectivity? Making life easy for a web developer, rather than saving time and money for users, is unethical behaviour. The developers are putting their (and their company's) comfort over that of the people using their system. Perhaps that's OK on a day-to-day basis. But this is a system designed to help people in a stressful situation trying to ascertain if a caller is genuine. That's not a situation where you want to needlessly pipe a couple of hundred KB over a dodgy 3G connection.
- Retric 5y agoIt’s only 191 kB compressed, which seems like a reasonable thing to send over even a dodgy 3G connection. Not that I think it’s a good idea, but in terms of bank IT I’ve seen far worse.
- deleted 5y ago[deleted]
- playpause 5y agoYou have no real idea of the various numbers involved or the ins and outs of the decision, so to accuse people you don’t know of “unethical behaviour” is ridiculous. Get over yourself.
- vmception 5y agoThe only thing they lose is a few points on a website benchmarker
- fastball 5y agoI don't think anyone in the UK pays per MB of data?
- mrb 5y agoAgreed, this 1.3 MB JSON data file is perfectly reasonable: it compresses to only 191 kB in the HTTP response with standard gzip compression that all browsers have been supporting in the last 20 years (I just checked with Chrome's dev tools)
- matthewmacleod 5y agoI agree that this approach is actually pretty reasonable for checking if a specific string is in a relatively small list of possible values that can't easily be expressed as a simple set of prefixes or whatever. It's must simpler and easier to maintain. The bigger problem with this approach is that it exposes a canonical list of phone numbers that are guaranteed to not be flagged as fake by the tool itself. That isn't likely to be a _massive_ concern here specifically, since it will always be easy to find a valid customer service number without that access. But I'd expect a team looking at dealing with fraud or scams to be pretty conservative about information disclosure generally – it would be a good argument to implement this as a backend service.
- james_in_the_uk 5y agoIn banking, any dislcosure of info hitherto not public is a /potential/ new attack vector for fraudsters. Surprising that they did it this way
- ljm 5y agoI'm struggling to see how a regex would work, either. It's a database lookup, a regex that matches only 74k numbers out of every possible valid phone number in the UK would be absurd.
- staz 5y agoI think the intuition is that a lot of these phone number come in ranges, so it would not be a regex but multiples? But yeah seems easier to just encode the ranges, than some complex system to create the most minimal regex. edit: checked the comments of the original post and they apparently used perl to produce a 11kb regex.
- alexwasserman 5y agoI assumed they meant that a regex here could be used to just extract out the numbers and remove anything extraneous. They mention it fails with trailing characters, or typical punctuation used in numbers (eg. dashes). A little regex to remove out anything non-digit would leave you with just the numbers to compare to the numbers in the list.
- jlg23 5y agoThat is a fine tradeoff until you realize that a single, manual verification of these 1.3MB telephone numbers costs more than developing and maintaining the unit tests...
- tedunangst 5y agoWho verifies the unit tests match the real phone numbers?