10 ms·
GitHub code search is generally available
- thih9 4y agoI initially thought “generally available” refers to removing the login wall. I remember when you didn’t have to sign in to search code on GitHub; I miss that.
- marginalia_nu 4y agoProbably due to bot abuse. Can't have nice things on the internet. Requiring an account makes rate limiting vs botnets easier.
- throklrhjflk 4y ago[flagged]
- h1fra 4y agoI have been using since the beta, truly the most impressive product released in the last 5 years (along with chatgpt). The amount of indexed code, the quickness and the precision of this search is simply stunning.
- tex0 4y agoYes, it comes pretty close. Well done!
- jjeaff 4y agoCould you elaborate more on why this is a significant feature? I can see how it would be handy to search a codebase online, especially one you don't have cloned locally, but for my own codebases, I can search the entire thing just fine in VS Code with ctrl-shft-f.
- doodlesdev 4y agoIt's not only about searching your own repository, it allows you to search through every single public repository on GitHub. I personally use it a lot to learn more obscure APIs which are badly documented or which I'm just not used to, simply search for the method I'm trying to use and find infinite examples of real world usage, along with the code license right next to it. It's also great if you drank the GitHub kool-aid as you can do a single search and find related code snippets, issues, pull requests and discussions that could possibly help. I'm personally not to big into the ecosystem, in fact I'm considering moving to Fossil so I can have everything inside the repo, but for those who are it's a great feature.
- JansjoFromIkea 4y agoFor me it's been very useful at finding codebases with work done on extremely specific niche things that would've been near impossible to find otherwise (e.g. tools for obscure protocols hidden away on obtusely named repos)
- simonw 4y agoEver wanted to use an API and found the documentation to be lacking? GitHub code search pretty much solves that. For any API you can find an example of someone else using it. I've been using it for this for a year now and I wouldn't want to live without it.
- rco8786 4y ago> For any API you can find an example of someone else using it. Amazing. This was the light bulb moment for me. I work at [big tech co] and we have an internal code search tool, and anytime I need to use a new API I pull it up to find examples of how it's used. Now I can do this for the entire world of OSS, amazing.
- h1fra 4y agoI think others have pretty much summarised it already: Searching for non-documented or badly documented API, find implementation of an algorithm or specific pattern, find how people are using a niche tool, etc. I have even used it to find my own API in the wild to look for potential breaking changes and improvements to do.
- radicality 4y agoIs the difference that now the basic search functionality actually works? The previous standard GitHub search I found to be remarkably bad. I would be looking at some small public repo, search for an exact string match I know exists in the code, scope the search to that repo only, and still see zero results. Even copy-pasting a line of code from a file in the repo often resulted in zero matches.
- AtNightWeCode 4y agoI don't get it too. Maybe one needs to enable something. The search is still useless. Locally I use git-grep.
- fallat 4y agoSame experience here. Are there advertisement accounts posting here or something? Legitimately weird.
- colin353 4y agoAh, you need to log in to get access to the new code search!
- Trasmatta 4y agoYou might want to consider replacing git-grep with ripgrep.
- Arnavion 4y agoIt seems you still need to enable the feature. Click your user icon in the top right -> choose "Feature preview" in the dropdown -> enable the "New Code Search and Code View" feature.
- bagels 4y agoHow that is not standard for the last 10 years is baffling.
- csnover 4y agoAbsolutely. GitHub Code Search is by far the most valuable online development tool I have used the past year. It is so much more useful than Copilot or any of the AI LLMs in my experience. With Code Search, I have: * Rewritten a CMake build system, which would have been practically impossible without access to real-world examples because of how poorly designed and documented it is; * Validated machine-generated translations by looking up language strings from projects that used human translators; * Tracked down bugs in unfamiliar codebases, using symbol-based navigation, without the downtime of fetching a bunch of files and waiting for a language server to process them locally; * Reviewed how projects were using the APIs of a library I work on to determine whether high-maintenance features were actually used, and whether tricky features were being used correctly or needed redesigning to reduce programmer error Kudos to the team at GitHub. Genuinely stellar work.
- neuronexmachina 4y agoThat's really interesting, do you have any examples handy of search queries you used?
- Kuinox 4y agoI'm not the one you asked to, but here my usages: Searching how an API is used. Searching how people configured something. Searching accidental uses of an attribute: https://github.com/dotnet/csharplang/discussions/5657#discussioncomment-4415454 https://github.com/dotnet/csharplang/discussions/5657#discus...
- MaxLeiter 4y agoIt's amazing for searching examples of rarely used / non-public APIs
- csnover 4y agoFor CMake stuff I would search like `<function name> path:/CMakeLists\.txt$|cmake/`. For translations I would search a phrase using `<phrase> lang:"Gettext Catalog" path:/<language iso code>/`. For reviewing library consumers I would just search for `<my library name> (<function name> OR <other function name> OR <third function name>) NOT path:<my library name>/**`. There may be better queries for some of these, but I really didn’t have to do any magic or weird tricks. Everything more or less just works.
- Arnavion 4y agoI'm pretty sure sourcegraph.com has been doing all that (search all GH repos, exact search in quotes, case-sensitive search, regex search, limit files using filename regex) for longer than 5 years.
- chx 4y agoDon't get me wrong but how is this any better than say Scintilla searching locally? I feel like the previous one was very bad and this is baseline but please tell me where I am wrong. From a brief test this looks like something you'd have built with Sphinxsearch a decade plus ago?
- perpil 4y agoThe time to pull request with a good code search, paired with GitHub.dev and Copilot is really a force multiplier. What a time to be alive.
- abathur 4y agoI generally like the new code search, but I've got one big gripe: there's no way to sort code results by any kind of proxy for recency. The old code search had the ability to sort by indexed date. This wasn't perfect, but it was something. I like keeping up with who's using my code and whether they're leaving comments or commit chains that outline trouble they're having with it. Sometimes old code pops up in the recently-indexed sort, but if I regularly search and look at the top page, I can see most new uses. Without it, code search is basically useless for this purpose :/
- 100k 4y ago(I work on code search.) Yeah, sorry about that. We've heard this feedback a lot. There's two reasons why we haven't implemented this. First, content is shared between repositories which makes this harder than before, when it wasn't. Second, we rebuild the index weekly or even more frequently, so the proxy of "when was this added" that was used doesn't work any more. What we would like to use is "when was this blob added to this branch" but that's extremely expensive to retrieve from Git because Git trees don't record it.
- colin353 4y agoI'm Colin from GitHub's code search team, happy to answer any questions. For more info on how we built this, you can check out our technical blog post from a few months ago https://github.blog/2023-02-06-the-technology-behind-githubs-new-code-search/ https://github.blog/2023-02-06-the-technology-behind-githubs...
- davidrjenni 4y agoHow does it compare to Sourcegraph? What is the main differentiator?
- panic 4y agoAre you planning to release code search as open source? I can’t find a link to the source code anywhere.
- jhgg 4y agoIt would be really awesome if code search could one day consume LSIF for precise results in its index similar to source graph. The symbol search is good now, but approximate. Having more precise code search by allowing devs to upload LSIF data in their CI pipelines would allow for precise symbol search (go to definition / find usages actually being accurate) and remove irrelevant result.
- colin353 4y agoGreat point. Yes, we initially focused on zero-config approximate code navigation. But we do intend to support build-based code navigation in the future, since the approximate code navigation experience can be pretty poor for some languages (e.g. C/C++).
- lexh 4y agoWill this make it to GH enterprise eventually?
- colin353 4y agoYes, we're working on bringing it to GitHub enterprise right now.
- aetherane 4y agoI don't get the praise over GitHub code search. I find it very inaccurate and often missing references etc. Maybe it depends on the language you are using it with? (Go here)
- bdcravens 4y agoAre you referring to what they just released? Github code search has always been notoriously bad. This is a new search product: "today, our new code search and code view are generally available to all users on GitHub.com"
- jacobr1 4y agoHave you tried the new search that was just launched? It seems to have significantly improved search accuracy. I agree the old version wasn't that great (though still was one of the better options for finding usage of things in the wild like rarely used OSS dependencies I needed to debug).
- rav 4y agoI have missed Google Code Search, which launched in 2006 and was discontinued in 2013. Similar to GitHub's code search, it supported searching by regex and filtering by language etc. - but obviously the amount of code to search through is orders of magnitude larger than it was 10-15 years ago. Still I wonder what took GitHub so long to build this - it's hardly a novel idea, and it seems like such an obvious power tool for programmers to have.
- esprehn 4y agoYou could try https://sourcegraph.com/search https://sourcegraph.com/search Though GitHub will probably have all the same features eventually and cheaper too.
- SoKamil 4y ago> Still I wonder what took GitHub so long to build this - it's hardly a novel idea, and it seems like such an obvious power tool for programmers to have. Scale. You can read more about their engineering at https://github.blog/2023-02-06-the-technology-behind-githubs-new-code-search/ https://github.blog/2023-02-06-the-technology-behind-githubs...
- rightbyte 4y agoSeems like grep-aaS? Or am I missing something. It could not e.g. get the definition of a C++ function for me. Really useful still, but should have been there years ago.
- ren_engineer 4y agoSurprised more people aren't talking about how Microsoft has a near monopoly on the developer ecosystem. They've got GitHub, OpenAI, and VS Code all working together and collecting data that strengthen each other's products while also using their embrace, extend, extinguish strategy with WSL and all of these steer people towards Azure services whenever possible. Seems like something that verges on an anti-trust situation when you think about the flywheel effect data has for AI credit to Microsoft for rehabbing their reputation with developers but it seems like a massive trojan horse
- chillel 4y agoI guess Steve Ballmer was right all along...
- misterprime 4y agoAbout developers?
- jansan 4y agoNo, about Playday: https://www.youtube.com/watch?v=V7PYQCXdX3A https://www.youtube.com/watch?v=V7PYQCXdX3A
- misterprime 4y agoHighly amusing.
- wongarsu 4y agodevelopers, developers, developers, developers https://www.youtube.com/watch?v=XxbJw8PrIkc https://www.youtube.com/watch?v=XxbJw8PrIkc
- zzzzzzzza 4y agogitea
- AtNightWeCode 4y agoHow do one access the new search?
- linhns 4y agoShould be available to all users now, previously was on beta
- AtNightWeCode 4y agoI cannot detect any difference.
- hddqsb 4y agoI think you have to visit https://github.com/search https://github.com/search, type a query (use /.../ for a regex, and repo:... to filter by repo), then click "Code" in the left navigation bar (and sign in). The search box on project pages appears to use the old search.
- spyremeown 4y ago>Without code search, you might have to clone a bunch of repositories and grep through them Why is this an issue? I do have a clone of all my company repos, doesn't everyone? Memory is cheap and ripgrep/ag are fast.
- edflsafoiewq 4y agoBandwidth is not cheap for many of us.
- henryfjordan 4y agohow big are the repos at your company? Bitbucket has a 2GB max size on each repo, I'd suspect other providers have similar restrictions. Would be a problem on mobile but repos that hit that limit are rare.
- deleted 4y ago[deleted]
- sk0g 4y agoNo, every microservice or new project/ tool get its own repository. There's new ones being created quite frequently, so I have ~10 repositories cloned that most relate to the work I do, but nothing beyond that.
- j1elo 4y agoI'm happy that at last, my Stack Overflow question and answer have been fully solved with technology improvements! https://stackoverflow.com/questions/43891605/search-partial-words-in-github-organizations-code/43911827 https://stackoverflow.com/questions/43891605/search-partial-... It's been almost 6 years, though... for a search scenario that would be trivial to implement with grep (at scale that's another thing...) Still, a nice example of perfect being the enemy of good, I guess.
- 100k 4y agoThanks for your patience! It has been a long road with some dead ends (we've wanted to add this since 2012 at least). We actually wrote about why we didn't just use grep in our last blog post: https://github.blog/2023-02-06-the-technology-behind-githubs-new-code-search/ https://github.blog/2023-02-06-the-technology-behind-githubs...
- RulerOf 4y agoI really enjoyed that blog post. Especially the comparisons that lay out how feasible it can actually be to do the stupid, simple thing of "grep the whole data set every time" up to a surprising point.
- synergy20 4y agohold on, I paid for copilot, what does github-code-search buy for me? not to mention I can kind of search its code already in the past.
- kazinator 4y agoIt isn't generally available; it requires login.
- unicornmama 4y agoRIP Sourcegraph
- midoBB 4y agoIn my experience Sourcegraph offers a better integration with Gitlab and Github both. And their code search is far superior.
- totalhack 4y agoThis is a huge upgrade...but one major annoyance with their new interface is some keyboard shortcuts conflict with browser shortcuts. I'm looking at my repo and I ctrl+f to search the current file and start typing my search query assuming it's working...but it's actually focused on the file browser and started trying to edit the file instead?!?
- xvilka 4y agoThe feature isn't working well yet on C and C++. If I recall correctly it's based on Tree-Sitter[1] parsing, and there are still too many bugs in corresponding grammars - tree-sitter-c[2] and tree-sitter-cpp[3]. Hopefully, it will be greatly improved in the future as the share of the existing and newly written code in C and C++ is quite significant. [1] https://tree-sitter.github.io/tree-sitter/ https://tree-sitter.github.io/tree-sitter/ [2] https://github.com/tree-sitter/tree-sitter-c/issues https://github.com/tree-sitter/tree-sitter-c/issues [3] https://github.com/tree-sitter/tree-sitter-cpp/issues https://github.com/tree-sitter/tree-sitter-cpp/issues
- slondr 4y agoAs an Emacs user, the tree-sitter-based Elixir and HEEx modes are vastly superior to the built-in HTML+ and Elixir modes.
- solarkraft 4y agoIt's good. it's mind bogglingly insanely good compared to the garbage that was there before for years and (I think) still is in Azure DevOps.
- naveen99 4y agoNice, regular expressions.
- usr1106 4y agoMy *first* test is not very successful. > Whoa there! > You have exceeded a secondary rate limit. > Please wait a few minutes before you try again; in some cases this may take up to an hour.
- pvman 4y agoNo way to sort by recency makes the new code search completely unusable. Having to hover the file title to see the updated date is horrible. For some searches, I want to see how developers are solving problems nowadays, not 10 years ago. No more number of matching files per language. Not possible to filter by multiple languages anymore. Such a downgrade.