22 ms·
Username ending with MIME type format is not allowed
- ncann 5y agoLooks like the fix restricted the check to "usernames that end with dot and MIME type" Still, what is the attack vector?
- boomskats 5y agoThis doesn't look like a security issue, unless I'm missing something.
- pavon 5y agoIndeed the MR template does not have the security box checked.
- kinix 5y agoMy guess is that the username is used in a url somewhere? So browsers might try and interpret it as a file
- numlock86 5y agoThat's what actual MIME types are for. You can serve a .jpg as a .txt file and still have the browser display it correctly as an image if you use image/jpeg. Same for the other way around or any other combination of file and MIME types.
- kinix 5y agoYou're not wrong, but I bet there are lots of things that will assume based on url (rightly or wrongly). e.g. I've seen a lot of nginx reverse proxy setups implement different caching policies based on url suffix (again rightly or wrongly). Maybe just playing it safe?
- numlock86 5y agoYeah, if you for example use the 1001th hip electron based browser (with some hand-crafted curl workers and custom fetchers with an express based proxy-backend included in the background mangled together with some weird mix of 20% javascript and 80% typescript and a combination of at least 400 different npm packages over a commit history of two weekends) to browse the web ... of course, that assumption might break. :^)
- jasonjayr 5y agoDo you remember when MSIE would ignore the MIME type the server sent, and sniffed the contents of the beginning of the file downloaded and overrode what the server sent? Good times ..... (rocking in corner ....)
- LewisVerstappen 5y agoHere's a link to the fix: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954/diffs https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954/...
- tankenmate 5y agoI get the sneaking suspicion that this is a case of Ruby's magic being slightly too magic; it's a problem I have tripped across in the past. Slightly tangentially reminds me of the "More Magic" switch of GLS fame.
- LewisVerstappen 5y ago>Slightly tangentially reminds me of the "More Magic" switch of GLS fame. Can you elaborate on what you're referencing? Tried googling but couldn't find anything.
- robocat 5y agohttp://www.catb.org/jargon/html/magic-story.html http://www.catb.org/jargon/html/magic-story.html
- mdip 5y agoThanks for the link -- you solved a nagging mystery for me[0]. In the '90s, I worked for a tech guy from Texas who referred to the clippers that we used to cut the ends of zip ties off as dikes. Of course, being a teenager, the term often came with a raised eyebrow from me since it was commonly used as a slur for lesbians. He was the only person I've encountered in my life who used that term in that manner and over the years I just assumed it was some bit of obscurity related to where he started working in systems. I guess not! [0] I'm sure the right Gooble query would have gotten me an answer but it teetered on the edge of "I don't care enough to bother" until the answer was presented.
- zorr 5y agoThere is not much ruby magic in that code. This is just a naive use of data from another module. Somewhere in the Gitlab code base there is a MIME_TYPES map with common extensions as the map key. No idea what it is used for but that module is very likely the target of a recent security issue. The first fix to combat the "publicly unknown" vulnerability was to prevent usernames ending with any of the keys in the MIME_TYPES map using a simple "ends_with" strings check. Of course the map keys did not have periods so the ends_with would also match "Asimov" with the "mov" suffix. The second fix in this PR is to extend the ends_with check to add an extra dot. The actual vulnerability is still unknown but I suspect it's something like an intermediate component that performs special handling based on interpreting URLs and that could bypass security/ACL checks.
- nine_k 5y agoIt could maybe potentially be exploited in a very interestingly crafted email, where there's link to download something (e.g. the source tarball, or a build artifact) with an URL containing the username, or being otherwise close by, so that the downloaded file would be interpreted differently. But I'm not creative enough at this hour to suggest a working exploit.
- foota 5y agoI've played with this before. A correctly implemented mail library should handle e.g., subject lines that contain SMTP control characters. I developed a lengthy repro for an email parsing issue in an ancient version of some java email library that contained a truely horrendous parser, only to find out that the library had been updated internally recently :-)
- a-dub 5y agoMIME types are used all over the place: 1) web servers, browsers, proxies 2) graphical os shells 3) email every file a webserver returns has a mime type in the header, and that is how the browser knows how to present it.
- dolmen 5y agoThe ticket here reveals some blacklisting of known file extensions (attached to a know list of MIME type), not MIME types directly.
- mmis1000 5y agoEven webserver returns a mime, browser may sometimes still ignore it and guess it themselves, because server may be wrong. Unless the server ask them `absolutely don't do it` via the header. a.k.a. `X-Content-Type-Options: nosniff` But even there is a header for that, there will still be situations that clients don't care and guess it themselves (probably because software is too old or other reason).
- amenghra 5y agoCorrect. You want to set Content-Type and X-Content-Type-Options. The story is that in some cases, Internet Explorer could be tricked to ignore the Content-Type header. Instead of fixing the core bug, Microsoft decided to add a header (presumably, they didn’t want to break existing websites). A decade later, people discovered two ways to bypass X-Content-Type-Options. This time, Microsoft fixed the core issue instead of adding X-Content-Type-Options-For-Reals.
- paxys 5y agoDefinitely a security issue. - The merge request which originally added this check is inaccessible (https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/1450 https://gitlab.com/gitlab-org/security/gitlab/-/merge_reques...) - In the issue comments the Gitlab employee says "Sorry, I cannot go into details right now. I will link the issue here once it goes public, is it ok?"
- walty8 5y agoBut the in the screen capture of article, the user name is actually 'issac.asimov', i.e. the mime type does not immediately follow the dot.
- ajkjk 5y agoThat was before the fix.
- nobody9999 5y ago>But the in the screen capture of article, the user name is actually 'issac.asimov', i.e. the mime type does not immediately follow the dot. A variation on the Scunthorpe Problem[0] then, eh? [0] https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem
- whizzter 5y agoSomebody probably put in a regexp with .mov$ , however for regexps the dot (.) matches everything (and $ matches end) so the i in asimov is eaten regardless and then the rest of the match succeeds.
- chippiewill 5y agoYou can see the fix they made in the linked MR. It wasn't a regex, they just did a generic "ends with" check.
- iechoz6H 5y agoPerhaps the sub-clause is redundant there? 'The problem was named after an incident in 1996 in which AOL's profanity filter prevented residents of the town of Scunthorpe, Lincolnshire, England, from creating accounts with AOL, because the town's name contains the substring "cunt".'
- nobody9999 5y ago>'The problem was named after an incident in 1996 in which AOL's profanity filter prevented residents of the town of Scunthorpe, Lincolnshire, England, from creating accounts with AOL, because the town's name contains the substring "cunt".' Right. Regardless of the specific pattern matching function, in both cases, the results were both incorrect and unwanted. Which is why I consider this instance to be a variation on the same issue.
- anentropic 5y agoProbably: some web frameworks do content negotiation by appending a content type like .json to the end of the url Not sure if it's an attack vector per se, or just that the behaviour is incompatible with allowing usernames containing . and then having urls where the username is the last segment of the url seems like a badly designed url scheme :)
- jfrunyon 5y ago> some web frameworks do content negotiation by appending a content type like .json to the end of the url This has always disturbed me, considering that HTTP has had content negotiation for ... oh, basically its entire history [https://www.w3.org/Protocols/HTTP/1.0/spec.html#Accept https://www.w3.org/Protocols/HTTP/1.0/spec.html#Accept].
- Macha 5y agoFor non-programmatic usage and verification, the extensions can be easier? On a similar topic HIBP allowed people to request versioning via a custom HTTP Header, a Accept Content-Type, or a version segment in the URL path and approximately everyone went with option 3.
- jfrunyon 5y ago"For non-programmatic usage and verification," it should be served in a human-readable manner, typically including things like formatting, and maybe even syntax coloring if you're feeling nice. Take my `Accept: text/html` and give me my HTML-ified JSON, dangit! ;)
- Macha 5y agoI've actually seen APIs with "pretty=true" or "indent=4" type query parameters to emit formatted JSON for people before, though it doesn't go as far as HTML with syntax colouring.
- gkop 5y ago
- wcoenen 5y agoI'm a little confused about the issue description, because "mov" is not a MIME type. Examples of MIME types: "text/plain", "text/html", "image/png" "application/pdf", "video/quicktime", ... If I was prevented from using the username "wcoenentext/html", then I wouldn't really be bothered by that. (Although I might question the design decisions that would necessitate such a restriction.)
- a-dub 5y agoit seems pretty obvious that they mean any file extension registered to a known MIME type.
- AlfeG 5y agoI guess author of ticket were referring to internal type name mime_type
- gpvos 5y agoIt's easy, basic, and very important for clarity to use the correct terminology in this case. A MIME type is really something different.
- p49k 5y agoIt’s unrealistic to expect everyone to be able to know and use terminology perfectly. The description itself was well-written and that’s what’s important in terms of finding/fixing the issue.
- rjmunro 5y agoThe bug reporter merely copies the incorrect error that gitlab gave him; The issue here is that someone working in login / security of gitlab doesn't know what a mime type is. That is extremely worrying - it's not a part of the code where you can afford to be sloppy. It's a very odd error. Apparently .nro is a file extension used by the Nintendo Switch video game console; .o is obviously the output of compilers, so I'm not sure why my username wasn't rejected. Maybe it would be if I tried to register now.
- em3rgent0rdr 5y agoReminds me of https://xkcd.com/327/ https://xkcd.com/327/ "Did you really name your son Robert'); DROP TABLE Students;-- ?" "Oh, yes. Little Bobby Tables, we call him."
- InsomniacL 5y agowhy would this get a single downvote? i suspect the underlying reason to the restriction is somewhat related to sanitising input.
- FroshKiller 5y agoI did not downvote, but I don't like reading the same xkcd references over and over, so I can imagine others feel the same way.
- Grollicus 5y agoI suspect because Isaac Asimov was named before any of these problems with his name were a thing and his name is not something artificially constructed to cause problems with buggy computer systems.
- RobertWorkbench 5y agocame here to downvote this
- deleted 5y ago[deleted]
- paxys 5y agoTL;DR for those wondering: - There was a yet-undisclosed security vulnerability in Gitlab usernames - Staff member made a change to disallow usernames ending with `Mime::EXTENSION_LOOKUP.keys`, which I assume is a set of recognized file extensions (hidden – https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/1450 https://gitlab.com/gitlab-org/security/gitlab/-/merge_reques...) - This was overly broad since it caught a lot of common names (like "asimov") (https://gitlab.com/gitlab-org/gitlab/-/issues/335278 https://gitlab.com/gitlab-org/gitlab/-/issues/335278) - The check was updated to additionally look for a "." before the extension (https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954 https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954)
- akie 5y agoIf anyone ever says that software engineering is completely unlike plumbing, I will point them towards your comment. All systems, however nicely architected, are full of duct-tape solutions like this.
- Grollicus 5y agoAre they though? What I can think of (broken upstream reverse proxies that do mime type inference by filename) would warrant a WE_USE_BROKEN_LEGACY_SHIT_UPSTREAM config flag so that it doesn't get in the way of normal users. So I'm probably missing something and I'm really curious for the underlying vulnerability.
- akie 5y agoI have no clue about the underlying issue, but I'm guessing it's occurring on a boundary or an interplay between two systems. Something like "the username can be part of the URL, and if the URL contains .mov, some browsers will misinterpret this and assume it's a movie file, leading to bad things™". Or: "the username is sometimes used as a folder name, and our syncing software contains rules to exclude certain file extensions, so these folders were never synced, which lead to issues on production servers" I'm guessing it's something along these lines. Something that you control, but not really, leading to these kind of haphazard workarounds.
- fregante 5y agoMy guess is that they were using /.mov$/ to check the username, which is missing an escape.
- ajkjk 5y agoNope: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954/diffs?commit_id=0b4e1788d14edaf16e3ce999fd0520b3d2467e01 https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954/... just a regular .ends_with()
- banana_giraffe 5y agoSo, .html is bad for a username, but .Html is ok? What is the reason for this?
- marwis 5y agoRails, which GitLab is based on, uses suffixes to select content renderer (such as .html, .json - apparently in case sensitive manner), also I think it will serve underlying files in some cases (e.g. the example of dashboard.html) given elsewhere. The proper fix is to disable this mechanism at least for the username segment of gitlab path but perhaps GitLab developers are too lazy or unaware or just in rush.
- ajkjk 5y agoProbably it's a spot fix for a security problem and not what anyone thinks is ideal.
- alfiedotwtf 5y agoSounds like a WAF is getting in the way. Wikileaks had the same issue years ago when you couldn't search for programming language binaries e.g "Ruby" or "Perl" etc
- kzrdude 5y agoWhat's waf?
- bigiain 5y agoWeb Application Firewall. https://en.wikipedia.org/wiki/Web_application_firewall https://en.wikipedia.org/wiki/Web_application_firewall
- Grollicus 5y agohttps://en.wikipedia.org/wiki/Web_application_firewall https://en.wikipedia.org/wiki/Web_application_firewall Basically something to extract boatloads of money from enterprise customers by annoying THEIR customers so they can't write "<script>" in texts in their application. Or, less tongue-in-cheek, a way to harden web applications against known attack patterns like sql injections or xss-attacks. As they work on pattern recognition and don't know anything about your application they sometimes get in the way. But they'll probably check some box for some security audit so they're used. Cloudflare for example offers one at https://www.cloudflare.com/waf/ https://www.cloudflare.com/waf/
- a-dub 5y agooh weird. i thought they also did stuff like parse and reconstruct requests to try and catch any funny business and centralize/add ease for things like ratelimiting and fail2ban for webapps. looking at this one, it appears not.
- saint_abroad 5y ago> But they'll probably check some box for some security audit so they're used. A WAF is useful for when a zero-day is found for that legacy application you just can't get patches for anymore because the team has "moved on".
- pechay 5y agoOnce had my Australian production system go down because a js plugin we were using had added .au to its list of media types. Integration had a different TLD. Response I got from the author was 'LOL!'. :)
- CodesInChaos 5y agoWhy is the application treating a TLD like a file extension?
- detaro 5y agoprobably because some broken code matched on the end of the URL, without checking if it was just the naked domain?
- cmpb 5y agoI'm going to have to find a way to work "naked domain" into my lexicon today, thanks!
- detaro 5y agoOn second thought, I actually used it wrong above: usually, naked domain specifically refers to the registered domain, i.e. without any subdomains.
- hinkley 5y agoEvery programming language has a library to parse urls. Please, for the love of your own mothers, people: stop pretending like you know how to parse urls like they're strings. You don't. And even if you can, you won't do it right every time. It's been demonstrated over and over again. Just use the already tested url facilities to give you the host/path/query parameters
- quesera 5y agoYet another reason Australia should have kept TLD .oz!
- Pxtl 5y agoThe fact that usernames can break anything speaks to something profoundly wrong in general. Why should usernames be leaking into URLs instead of a surrogate key? I mean, barring some tricks that could break layouts during rendering if people got really creative with them, why isn't any valid non-blank Unicode string a valid username? Reminds me of this xkcd: https://xkcd.com/1700/ https://xkcd.com/1700/
- SSLy 5y ago>why isn't any valid non-blank Unicode string a valid username? it's because HTTP is stringly-typed
- EE84M3i 5y agoSomething that hasn't been mentioned, this could also be related to defenses against web cache deception attacks.
- id5j1ynz 5y agoThis seems on par with the general GitLab-style. Is anybody else getting a bit frustrated with them? They keep on having high-severity security bugs being fixed every month (e.g. auth checks not being done everywhere). Then there's all these odd edge case bugs everywhere. As an outsider, it just seems to me that GitLab isn't being engineered in a principled way: on sound abstractions and with separation of concerns (e.g. auth should be some universal middleware, not ad-hoc per call). Just really basic stuff.
- sam0x17 5y agoFurther proof that gitlab is just super jank. What on earth are they doing inspecting usernames to see if they end in a file extension? What horrendous vulnerability is this band-aiding? That they don't seem to understand what a MIME type is just adds to this perception.
- lol768 5y agoYeah, this didn't leave me with a good impression of their security and development practices (not that I really had one anyway after they accidentally deleted their production PostgreSQL data directory and did a fairly poor job of responding to the situation). This change absolutely seems like the wrong place to fix any real security vulnerability and the fact that it affected a bunch of legitimate usernames is the icing on the cake.
- city41 5y agoI've never used gitlab but this issue has definitely caused me to take pause. It sort of feels like "not allowing 'OR' in strings to prevent SQL injection."
- dnsmichi 5y agoHi, the error message is a bit confusing - certain file type extensions may cause the user profile page not to load. Disallowing specific extensions in the username helps prevent that problem - it is not to prevent an injection attack or similar. This comment https://news.ycombinator.com/item?id=28540665 https://news.ycombinator.com/item?id=28540665 helps with more content and issue URLs including the problem discussion.
- malinens 5y agogitlab naming restrictions are also ridiculous for repositories. For example our files repository needed to be named filez. There is another repo which we needed to rename while migrating to gitlab but I dont remember it's name now. This has made many issues for us because repo name is no more related to project name...
- john_cogs 5y agoGitLab team member here. I'd like to add some additional context to my previous comments [1][2]. Due to a security concern in which a profile containing a file extension would not load [3], we do not allow usernames that end with file extensions (ex: .mov). As noted by many folks here, these are associated with a MIME type but are not MIME types themselves. It is not related to preventing an injection or any such attack vector. The error message for this check incorrectly included MIME type rather than file extension. This has been updated [4]. Additionally, there was an issue with how the actual check as it did not include the leading dot. The leading dot was added to the check in a subsequent MR [5]. Thanks for all the feedback. 1 - https://news.ycombinator.com/item?id=28535739 https://news.ycombinator.com/item?id=28535739 2 - https://news.ycombinator.com/item?id=28538166 https://news.ycombinator.com/item?id=28538166 3 - https://gitlab.com/gitlab-org/gitlab/-/issues/26295 https://gitlab.com/gitlab-org/gitlab/-/issues/26295 4 - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70374/diffs https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70374/... 5 - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954 https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65954
- Arnavion 5y ago>Due to a security concern in which a profile containing a file extension would not load [3], we do not allow usernames that end with file extensions (ex: .mov). Why did you not fix your routing engine to not consider file extensions where the username / group name should go?
- bogwog 5y agogitlab profile URLs are `gitlab.com/<username>`, so a user with the name "dashboard.html" would have the URL `gitlab.com/dashboard.html`, which obviously conflicts with the existing dashboard.html file. Besides blacklisting certain usernames or breaking a bunch links to profiles, how would you fix that? EDIT: IIRC, github has the same issue, but they have profiles as lower priority. So if your username conflicts with an existing URL, your profile page doesn't work.
- smallbizdev420 5y ago
- bob1029 5y agoWhy should the username matter? In my systems, I could have an insane URL like... https://myservice.com/my.super.duper.crazy-ass.user.name.pdf.exe and still have it return a proper HTML document that covers that user's profile page. Hell, the username could be some insane zalgo-tier shit and still function properly. I see some comments defending arbitrary "bandaid" architecture and I think that this is not defensible for something the scale of GitLab. This is basic HTTP stuff.
- dnsmichi 5y agoHi, how would you address this in GitLab's code? Maybe you'd like to create a merge request with suggested fixes :) https://news.ycombinator.com/item?id=28540665 https://news.ycombinator.com/item?id=28540665 has all URLs and issues available to get started in the code.
- bob1029 5y agoI would begin by familiarizing myself with the Content-Type HTTP header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Co... As for actually performing this change myself, it would take me quite some time to grok the codebase. Maybe I get bored tonight and see how hard this would be to resolve.
- vultour 5y agoIt’s not his job to fix a paid product.
- boleary-gl 5y agoOf note, GitHub doesn't allow periods in usernames either. I'm not a Ruby expert but I wonder if file extensions give some specific Ruby gotchas that means both GitLab and GitHub operate this way.
- e12e 5y agoRuby? No. Rails? Perhaps. (Both github and gitlab are built with ruby on rails). For an old post on a somewhat related topic, see: https://ryanbigg.com/2009/04/how-rails-works-2-mime-types-respond_to https://ryanbigg.com/2009/04/how-rails-works-2-mime-types-re... I could imagine the mix of rails #respond_to and "file extensions" at the end of urls might make a mess (think /users/profile/smith.html vs /smith vs /smith.json vs /smith.txt - essentially what might have been /smith?format=json etc). Ed: current documentation: https://apidock.com/rails/v6.1.3.1/ActionController/MimeResponds/respond_to https://apidock.com/rails/v6.1.3.1/ActionController/MimeResp...
- jrochkind1 5y ago> @brabant.benjamin Sorry, I cannot go into details right now. I will link the issue here once it goes public, is it ok? — (July 14) https://gitlab.com/gitlab-org/gitlab/-/issues/335278#note_625967627 https://gitlab.com/gitlab-org/gitlab/-/issues/335278#note_62... Has anyone traced down an explanation of the (presumably security-vuln-related) thing that was going on, motivating the original restriction? I would think it would be public now, with the restriction removed, but maybe not? It does seem like an unfortunate UX to exclude the lastnames of millions of people as usernames.
- john_cogs 5y agoPlease see https://news.ycombinator.com/item?id=28540665 https://news.ycombinator.com/item?id=28540665 for more detail and links to related issues and MRs.