25 ms·

Sourcegraph is now open source

594 points by aranw 8y ago

boyter 8y ago
Very nice. Poking though the source now, cmdfrontend/graphqlbackend/textsearch.go is one of the more interesting files I have found so far. It's very neat to look under the hood.
- sqs 8y ago
  Hey, Ben! Yes, that is a neat source file that shows how Sourcegraph does really fast live code search. (Sourcegraph has hybrid search so you get fast search over any repository and any commit, without needing to wait for indexing. It'll hit an index on the default branch, but for any historical commit/branch that isn't indexed, it has a super-optimized live searcher. We sponsored some contributions to the Go stdlib to improve this, in fact.) BTW, you can pull up that file quickly on Sourcegraph: 1. Search for repo:ph/sourcegraph textsearch.go — https://sourcegraph.com/search?q=repo%3Aph%2Fsourcegraph+textsearch.go https://sourcegraph.com/search?q=repo%3Aph%2Fsourcegraph+tex... 2. Go to the first file — https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/cmd/frontend/graphqlbackend/textsearch.go#L148 https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-... 3. Now click around! You get code intelligence: go-to-definition, find-references, hover tooltips, etc. It works on any open-source repository, and on your internal repositories if you spin up your own Sourcegraph instance. As for licensing, we went with open-source using the Apache License because that's what makes it as easy as possible to get user #1 and user #20 inside of companies. See https://twitter.com/sqs/status/1046913901688807424 https://twitter.com/sqs/status/1046913901688807424 for a bit more info there.
sqs 8y ago
Sourcegraph CEO here. We're back to work on testing release candidates for Sourcegraph 2.12, coming out this week. I'm especially excited about a few big new features that fit really nicely into the code search and browsing workflows people use Sourcegraph for: - https://github.com/sourcegraph/about/blob/master/projects/extension-authoring.md https://github.com/sourcegraph/about/blob/master/projects/ex... - https://github.com/sourcegraph/about/blob/master/projects/sourcegraph-for-gitlab.md https://github.com/sourcegraph/about/blob/master/projects/so... - https://about.sourcegraph.com/blog/discuss-code-and-docs-in-repositories/ https://about.sourcegraph.com/blog/discuss-code-and-docs-in-... along with a whole host of improvements to code search, code intelligence, self-hosted deployment on a single node and k8s, etc. Happy to answer any questions folks have about our open sourcing. I summarized the business case at https://twitter.com/sqs/status/1046913901688807424 https://twitter.com/sqs/status/1046913901688807424. And we went with the Apache License (2.0) and an open core model because that's about as standard and as open as possible for this kind of thing.
- neves 8y ago
  Is it easy to add new languages? I work at a large company with some very old codebases.
  - beliu 8y ago
    @chrismwendt is a Sourcegrapher working on this. Posting this on his behalf: Here is how I added GraphQL support last week: - Follow the hello world tutorial https://github.com/sourcegraph/sourcegraph-extension-docs/blob/master/tutorials/hello-world.md https://github.com/sourcegraph/sourcegraph-extension-docs/bl... - Add logic to handle hovers and definitions https://github.com/sourcegraph/sourcegraph-graphql https://github.com/sourcegraph/sourcegraph-graphql - Write the server part https://github.com/chrismwendt/graphql-ws-langserver https://github.com/chrismwendt/graphql-ws-langserver (once a browser-based GraphQL analysis library exists, this would be unnecessary because the analysis could be done entirely in the browser) It's available at https://sourcegraph.com/extensions/chris/graphql https://sourcegraph.com/extensions/chris/graphql and once you enable it you can test it out on a GraphQL file such as https://sourcegraph.com/github.com/chrismwendt/graphql-ws-langserver@master/-/blob/schema.gql?diff=5a6f2dc350ec67e8aa8c8b606fd3a8fb0c66d578 https://sourcegraph.com/github.com/chrismwendt/graphql-ws-la... (it's pretty slow/janky at the moment and could be improved, but works). If you're interested in interfacing with other tools too, you might be interested in LSP (Language Server Protocol): - https://langserver.org/ https://langserver.org/ - https://microsoft.github.io/language-server-protocol/specification https://microsoft.github.io/language-server-protocol/specifi...
- ntonozzi 8y ago
  Congrats Quinn and company! I hope there's not too much of my code lurking in there =)
  - slimsag 8y ago
    Oh! Hey Nico :) Hope all is going well! I miss having ya around!
- brylie 8y ago
  Cool! Are you working on IDE plygins, e.g. for Atom?
asdkhadsj 8y ago
Holy cow, does this mean I can use source graph as a single dev wanting the features, but not able to upload private source to a cloud service? Fwiw, I'd love to pay as a private user for this type of service. It's just that for many, we can't use a tool that leaks code. I look forward to that day, if it's not today :)
- deleted 8y ago
  [deleted]
- sqs 8y ago
  Yes! There is a `docker run` command that will spin up a Sourcegraph instance for you at https://about.sourcegraph.com/docs/ https://about.sourcegraph.com/docs/. The self-hosted Docker (or, for big clusters, Kubernetes) deployment method has been there for about 10 months now, and devs/companies really like it. Making it so easy to self-host Sourcegraph was a big win, and going open-source is another step in that direction. What is new is that you can now also build Sourcegraph from source and see/hack on/etc. our source code.
  - smivan 8y ago
    Awesome!! So the Go code intelligence is now free for single devs who are self hosting?
    - sqs 8y ago
      Yes!
emmanueloga_ 8y ago
The kind of cross-reference SourceGraph provides is a must have for me after a few years of using a tool like it to understand big projects (I don't use SourceGraph, I use Grok/CodeSearch). Google open sourced a version of Grok some time ago (https://kythe.io/ https://kythe.io/). [1] A quick search also found this `OpenGrok` thing which I assume must be yet another generic indexer [2]. I wonder if there are other popular tools out there providing generic cross-referencing of code bases? <SourceGraph> I was trying to find the equivalent of this Kythe page (how to write an indexer)[6] on SourceGraph docs, after 20 minutes I was still confused. I wish there was more clarity on the different pieces of the system. At first I thought SourceGraph could create a cross reference index from an LSP server, but I guess makes more sense it is the order way around? (SG can talk to text editors via LSP). It must be a lot of work to make integrate any single language with both SG or Kythe and LSP, perhaps there's some tools or methodologies to make this easier? <Tangent> I feel like language designers don't always think about building their implementations with ease of integration as a goal. I remember wanting to play with doing some AST transformations with both TypeScript and Dart (independently) and finding both languages made the task so hard and tedious. Mandatory remark: this is so easy with Lisp derivatives :-) Things that a good language+ecosystem should have, IMO: * Easy to retrieve and manipulate CST (for things like writing refactoring and querying tools, writing code formatters/pretty printers, etc). Many languages provide the syntax tree say, without comments, so it is hard to write a pretty printer. * Documentation generator with cross-reference navigation. * Play along nicely with generic build tools like Ninja [3]. * Good debugger and value-pretty-printer support (so I can just drop a breakpoint and print anything easily). And some sort of API to use the debugger programmatically or connect to a remote process. * An LSP implementation [4], although it seems pretty much every popular language is getting one these days! 1: https://en.wikipedia.org/wiki/Google_Kythe https://en.wikipedia.org/wiki/Google_Kythe 2: http://oracle.github.io/opengrok/ http://oracle.github.io/opengrok/ 3: https://ninja-build.org/ https://ninja-build.org/ 4: https://langserver.org/ https://langserver.org/ 5: https://about.sourcegraph.com/docs/code-intelligence/ https://about.sourcegraph.com/docs/code-intelligence/ 6: https://kythe.io/docs/schema/writing-an-indexer.html https://kythe.io/docs/schema/writing-an-indexer.html <edit: line breaks>
- tmerr 8y ago
jaytaylor 8y ago
I was lucky to connect with Quinn and get the opportunity to volunteer as a guinea pig for open sourcing this code :). Sourcegraph is an interesting product and company. I see the potential for it to have a bigger positive impact on developer productivity than even Jetbrains IntelliJ has, because the reach can go beyond any particular editor or frontend. It could be a ubiquitous solution to efficient code search and discovery. Will definitely be watching how things unfold for Sourcegraph. If you haven't already read their Master Plan, it's a compelling vision. https://about.sourcegraph.com/plan https://about.sourcegraph.com/plan
- sqs 8y ago
  Thanks for running through the README and helping us prep the open-source release last week!
mikekchar 8y ago
I'm especially interested in open source/free software businesses and business models. Are you planning to do an open core kind of thing, or maintain full on open source while charging for hosting and/or other services? I realise that might be a difficult question to answer as you might want to keep your options open, so I'm not asking for a commitment :-) I'm just curious about your plans. Also do you ever think you might add (for want of a better phrase) custom development services if you aren't already? For example, allowing people to pay a fee to get specific features added to Sourcegraph. What are the kinds of benefits and risks you might see from going in that kind of direction? I've noticed that a lot of open core businesses tend to avoid offering customer development services -- and if they do it's often perceived as a cost centre rather than a profit centre. It's something that is provided out of necessity of having to provide custom solutions for a few big customers, but the services team often appears to be at odds with the core development team. As you have obviously gone through the thought process of how to make money when your software is out in the open, I wonder what kinds of thoughts you have on that topic. It's really nice to see this kinds of stuff happening and I wish you the best luck possible in your endeavours!
- pietroglyph 8y ago
  It looks like they're going with open core: https://about.sourcegraph.com/pricing https://about.sourcegraph.com/pricing
  - jordigh 8y ago
    > Open. For business. You know, it's funny that they are proudly using this slogan. OpenBSD has mocked it in the past, referring to Intel's firmware blobs for wireless NICs: > Some asshole said he was "open" > but he was only open for business. https://www.openbsd.org/lyrics.html#40b https://www.openbsd.org/lyrics.html#40b Open core is no big innovation. It's plain ol' proprietary software. It's just like macOS. Most software nowadays has significant free components. The only company that I know that does truly free commercial software is Red Hat. They don't sell you a drop of non-free software. Some people will argue that the degree of free software in your proprietary software makes a difference. I argue that it only matters if you need any of the proprietary bits or not. A non-free but desired bit of software in a package can be very restrictive and have the same undesirable impacts as having the whole thing being non-free.
mholt 8y ago
Awesome! You guys did what I could not. Congratulations! So excited to see where this takes Sourcegraph, and the community around it. In my experience, even just using Sourcegraph to browse source code in open source repos is way faster than the native git host browsing (BitBucket, GitHub, Gitlab, etc). We use it daily at my lab, especially as I help others learn the code base. Thanks for Sourcegraph.
mickael-kerjean 8y ago
Will definitely use to navigate code. It's way better than github/gitlab/bitbucket.
chubot 8y ago
Very cool! Does it have C or C++ support? Last time I looked it didn't. Something that understands Python calls into C and vice versa would be interesting to me as well.
- sqs 8y ago
  Sourcegraph has experimental C/C++ support. Emphasis on experimental. It uses the best C/C++ language server (based on the Language Server Protocol) we could find, but that still struggles with some codebases. You can try it on any open-source C/C++ file on Sourcegraph.com before you spin up a self-hosted instance for your own code, such as: https://sourcegraph.com/github.com/antirez/redis/-/blob/src/expire.c#L60:9 https://sourcegraph.com/github.com/antirez/redis/-/blob/src/... Here is more info on using C/C++ on Sourcegraph (and an animated GIF): https://github.com/sourcegraph/lsp-adapter/tree/master/dockerfiles/cpp https://github.com/sourcegraph/lsp-adapter/tree/master/docke...
  - romed 8y ago
    That seems to have a lot of rough edges. For me, when browsing C++, there is exactly 1 killer feature: find callers. I recently deployed Kythe just to get this and it took many hours of my time. I would love to have something a little easier to maintain (that works, though).
- robinshen 8y ago
  OneDev developer here. OneDev is a self-hosted git server and it provides symbol cross referencing for main stream languages including Java, JavaScript, CSharp, C, C++, PHP, Go, Python, R, CSS, LESS, SCSS. Taking linux project for example: https://go.onedev.io/projects/linux/blob/master/tools/lguest/lguest.c?mark=2967.2-2967.15 https://go.onedev.io/projects/linux/blob/master/tools/lguest...
StavrosK 8y ago
Is this the thing that caused furore a while ago because it was installed on some editors and it sent all your code to a third party server, or am I thinking of something completely different?
- sqs 8y ago
  No, that is https://kite.com/ https://kite.com/ (absolutely no relation to Sourcegraph).
- posnet 8y ago
  No that was Kite, they paid editor plugin developers to take over the plugin and then added their code complete which requires sending all source code to their servers. https://qz.com/1043614/this-startup-learned-the-hard-way-that-you-do-not-piss-off-open-source-programmers/ https://qz.com/1043614/this-startup-learned-the-hard-way-tha... https://kite.com https://kite.com
  - StavrosK 8y ago
    That's the one, thanks. I can't wait to try Sourcegraph in that case, it sounds very useful!
- vanesa- 8y ago
  Nope, not Sourcegraph!!!
mmirate 8y ago
> Together, these companies succeeded in bringing computing to billions of people. But these billions of people are still using software applications built by just 0.2% of the world's population (those who can code). > The next step is to make it so billions of people, not just 0.2% of the world population, can build software (not just use it). I must admit, this sounds indubitably noble. But if you think that your software-engineering career is safe from (partial) automation, then these Sourcegraph folks have an iridium-plated bridge to sell you. In not-so-tongue-and-cheek terms: if Sourcegraph succeeds, then tomorrow's software engineers will be - at most - as valuable as today's factory-robot operators. * * * (Of course, anyone worth their salt in this industry ought to be able to find the LSP's inherent limitations, and structure their codebases accordingly.) EDIT: counterarguments are more persuasive than downvotes.
korantu 8y ago
"Master plan" link [0] is 404 in the linked article. (Not sure if this is right way to report it) [0] https://about.sourcegraph.com/plan https://about.sourcegraph.com/plan
- sqs 8y ago
  Thanks, just fixed (should be live soon).
lhorie 8y ago
We've been using Sourcegraph at Uber and it has been working pretty well. We saw a performance regression last week with `max` queries but it got fixed fairly quickly (less than a week from reporting to having the fix deployed) Some wishlist things: - would be nice if the query language was better documented (not the GraphQL console, the help section for that is comprehensive) - a coworker had expressed a desire for multi-line searches (which is supposedly in the pipeline)
- slimsag 8y ago
  (Sourcegrapher here) Just curious, have you discovered this page before from the in-app help area? https://about.sourcegraph.com/docs/search/query-syntax/ https://about.sourcegraph.com/docs/search/query-syntax/ (We will improve our docs here & make them stand out more in the app regardless, I think that is a great suggestion.) I also very much want multi-line searches, I can't wait for them :)
  - lhorie 8y ago
    Yeah, this help page is fairly easy to find (two clicks: ? icon -> "How to search"). I figured I'd ask because for example `max` is not documented there, even though it's pretty important for the sort of stuff I use Sourcegraph for (i.e. finding out all the repos that use X).
- sqs 8y ago
  Sourcegraph CEO here. Thanks for the feedback and sorry for the regression! I filed an issue to improve our search docs at https://github.com/sourcegraph/sourcegraph/issues/190 https://github.com/sourcegraph/sourcegraph/issues/190 (will fix in the next day or so) and to add multi-line searches at https://github.com/sourcegraph/sourcegraph/issues/191 https://github.com/sourcegraph/sourcegraph/issues/191 (this is one of 3 significant search improvements coming in the next few months).
  - lhorie 8y ago
    Awesome! Rock on!
kemitchell 8y ago
Was Fair Source a detour or a stepping stone?
- slimsag 8y ago
  Sourcegrapher here (speaking from my own viewpoint): Stepping stone. A fully open-source codebase[1] with Apache license is simply a better match as we aim to have a completely open and transparent product and company, where even e.g. planning is done in the open[2]. [1] https://github.com/sourcegraph/sourcegraph https://github.com/sourcegraph/sourcegraph [2] https://github.com/sourcegraph/about/blob/master/projects/about-repository.md#open-product-open-company https://github.com/sourcegraph/about/blob/master/projects/ab...
stephenr 8y ago
So before anything else - congratulations and thank you for releasing something as open source, under a business-friendly licence. Regardless of the project itself, that should be commended. However. I've never paid much attention before, so perhaps I'm missing something.. but this "just" seems to be the code intelligence you'd expect to find in a good IDE. Is there more to it? Or is this the 'solution' for the lack of such code intel in the plethora of text editors people use these days?
- slimsag 8y ago
  Sourcegrapher here -- two main things: 1. Sourcegraph offers the code intelligence your used to in your editor, but instead of being in your editor it is when you are browsing code on your code host (i.e. on GitHub, BitBucket, etc.) Even when reviewing pull requests / diffs! 2. Sourcegraph provides fast, advanced code search across multiple repositories -- akin to what Google and Facebook offer their devs internally. Regular expression support, extremely up-to-date results, etc. You can read more about this here: https://about.sourcegraph.com/docs/search https://about.sourcegraph.com/docs/search
k4ch0w 8y ago
This is amazing for secure code reviews. I have already spun up and instance and am demoing it for my team. Is it possible to get Jira integration? Can you test bitbucket.org connection with credentials some place I am not seeing? I can't tell what is causing it to fail. Is there a way to add markers in the code and then a queue to clear it? I.E I mark hotspots in the code and want a developer to review them or edit them. I think a current pain point would be no flat file upload. I understand you only need the .git directory and this was designed with a developer in mind. A lot of times when doing a secure code review we don't get credentials, we just get a current clone of the code. I just tried to do it using the instructions and had some trouble ended up just cloning it on the host machine. I usually get a zip/tar of the code. Another feature that would be awesome is export a search to a CSV file. I could script together something for Jira integration with that pretty quickly. Something to get the line affected #, branch and filepath. The subscription to searches was a brilliant idea. I wish I thought of it. SSO already integrated? Nice man! I'd make the discussions default too, I discovered that after playing with it for a bit. It's an awesome feature. I look forward to seeing more of it.
- slimsag 8y ago
  Heya, I'm the Sourcegrapher working on the discussions feature[1]. It's great to hear your are enjoying it. We're looking for users to help drive the direction of the feature as it matures and really influence what we work on next with it, so if you're up for it I'd really love to get your feedback on it (as much as you'd be willing to provide). :) Jira integration -> We're working on it! Mind commenting on https://github.com/sourcegraph/sourcegraph/issues/141 https://github.com/sourcegraph/sourcegraph/issues/141 ? Markers in the code -> If you could archive or dismiss the inline discussion threads (so that they no longer show up in the inline view) would that work well? Flat file upload -> Could you file a feature request for this at https://github.com/sourcegraph/sourcegraph/issues/new/choose https://github.com/sourcegraph/sourcegraph/issues/new/choose with some more details on how e.g. you would wish to give Sourcegraph access to the code? Exporting search to CSV -> It might be pretty easy to write something like this with our `src` CLI tool: https://github.com/sourcegraph/src-cli https://github.com/sourcegraph/src-cli You can get JSON output via e.g. `src search -json 'repogroup:sample error'` Hope to hear more from you! :) [1] https://about.sourcegraph.com/blog/discuss-code-and-docs-in-repositories https://about.sourcegraph.com/blog/discuss-code-and-docs-in-...
frou_dh 8y ago
I remember Sourcegraph as being a sort of sidecar to your text editor. Based on your caret position, it would automatically show you crawled examples of how other people use the functions/classes you are dealing with. Has that been de-emphasised or gone away?
- slimsag 8y ago
  I think you might be confusing Sourcegraph with Kite[1] (zero relation with Sourcegraph). Kite tries to do something like what you are describing. With Sourcegraph, we have editor plugins for various editors[2] but they only perform "Open current file on Sourcegraph" and "Search selection on Sourcegraph" actions, nothing like what you are describing yet. [1] https://kite.com/ https://kite.com/ [2] https://about.sourcegraph.com/docs/integrations/editor-plugins https://about.sourcegraph.com/docs/integrations/editor-plugi...
  - wholien 8y ago
    It might have been what I worked on at my internship at SG; demo here: https://vimeo.com/164809163 https://vimeo.com/164809163 It makes sense not to make this a big part of SG though, since latency is a huge issue and detracts from the rest of the product which is amazing!
JoshuaAshton 8y ago
Hard pass. I dont want an "intelligent" search I just want a search. Will be sticking with Agent Ransack.
- slimsag 8y ago
  To be clear, this isn't "intelligent" search. There is no magic here. It's just fast, advanced code search across multiple repositories (like what Google and Facebook offer their devs internally). This means support for regular expressions, always getting extremely up-to-date results, etc. You can read more about it here: https://about.sourcegraph.com/docs/search https://about.sourcegraph.com/docs/search There is "code intelligence", which is a phrase we use to describe IDE-like features such as jump-to-definition, find-references, etc.
Dowwie 8y ago
Know those cool features in your editor that allow you to hover over source and get more detail on what you are looking at? Also, know that feature in an editor that lets you go to definition? Imagine this and not within your editor but rather your browser, whatever host you choose, such as GitHub or GitLab. Sourcegraph enables this feature with browser extensions.
- sqs 8y ago
  Sourcegraph CEO here. This is a great way to describe how Sourcegraph integrates with your code host. I love it. :) Links to save you from searching: https://chrome.google.com/webstore/detail/sourcegraph/dgjhfomjieaadpoljlnidmbgkdffpack?hl=en https://chrome.google.com/webstore/detail/sourcegraph/dgjhfo... https://addons.mozilla.org/en-US/firefox/addon/sourcegraph/ https://addons.mozilla.org/en-US/firefox/addon/sourcegraph/
mariusmg 8y ago
So what's the use case for this ? If somebody wants "Find all references" , what's preventing them to open the code in Visual Studi/Intellij/Eclipse etc.
mysteriosaga 8y ago
Open source is the way to go. Onwards and upwards.
root3 8y ago
is it working only with git vcs?

594 points by aranw 8y ago

Very nice. Poking though the source now, cmdfrontend/graphqlbackend/textsearch.go is one of the more interesting files I have found so far. It's very neat to look under the hood.

sqs 8y ago
Hey, Ben! Yes, that is a neat source file that shows how Sourcegraph does really fast live code search. (Sourcegraph has hybrid search so you get fast search over any repository and any commit, without needing to wait for indexing. It'll hit an index on the default branch, but for any historical commit/branch that isn't indexed, it has a super-optimized live searcher. We sponsored some contributions to the Go stdlib to improve this, in fact.) BTW, you can pull up that file quickly on Sourcegraph: 1. Search for repo:ph/sourcegraph textsearch.go — https://sourcegraph.com/search?q=repo%3Aph%2Fsourcegraph+textsearch.go https://sourcegraph.com/search?q=repo%3Aph%2Fsourcegraph+tex... 2. Go to the first file — https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/cmd/frontend/graphqlbackend/textsearch.go#L148 https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-... 3. Now click around! You get code intelligence: go-to-definition, find-references, hover tooltips, etc. It works on any open-source repository, and on your internal repositories if you spin up your own Sourcegraph instance. As for licensing, we went with open-source using the Apache License because that's what makes it as easy as possible to get user #1 and user #20 inside of companies. See https://twitter.com/sqs/status/1046913901688807424 https://twitter.com/sqs/status/1046913901688807424 for a bit more info there.

sqs 8y ago

Sourcegraph CEO here. We're back to work on testing release candidates for Sourcegraph 2.12, coming out this week. I'm especially excited about a few big new features that fit really nicely into the code search and browsing workflows people use Sourcegraph for: - https://github.com/sourcegraph/about/blob/master/projects/extension-authoring.md https://github.com/sourcegraph/about/blob/master/projects/ex... - https://github.com/sourcegraph/about/blob/master/projects/sourcegraph-for-gitlab.md https://github.com/sourcegraph/about/blob/master/projects/so... - https://about.sourcegraph.com/blog/discuss-code-and-docs-in-repositories/ https://about.sourcegraph.com/blog/discuss-code-and-docs-in-... along with a whole host of improvements to code search, code intelligence, self-hosted deployment on a single node and k8s, etc. Happy to answer any questions folks have about our open sourcing. I summarized the business case at https://twitter.com/sqs/status/1046913901688807424 https://twitter.com/sqs/status/1046913901688807424. And we went with the Apache License (2.0) and an open core model because that's about as standard and as open as possible for this kind of thing.

neves 8y ago
Is it easy to add new languages? I work at a large company with some very old codebases.
- beliu 8y ago
  @chrismwendt is a Sourcegrapher working on this. Posting this on his behalf: Here is how I added GraphQL support last week: - Follow the hello world tutorial https://github.com/sourcegraph/sourcegraph-extension-docs/blob/master/tutorials/hello-world.md https://github.com/sourcegraph/sourcegraph-extension-docs/bl... - Add logic to handle hovers and definitions https://github.com/sourcegraph/sourcegraph-graphql https://github.com/sourcegraph/sourcegraph-graphql - Write the server part https://github.com/chrismwendt/graphql-ws-langserver https://github.com/chrismwendt/graphql-ws-langserver (once a browser-based GraphQL analysis library exists, this would be unnecessary because the analysis could be done entirely in the browser) It's available at https://sourcegraph.com/extensions/chris/graphql https://sourcegraph.com/extensions/chris/graphql and once you enable it you can test it out on a GraphQL file such as https://sourcegraph.com/github.com/chrismwendt/graphql-ws-langserver@master/-/blob/schema.gql?diff=5a6f2dc350ec67e8aa8c8b606fd3a8fb0c66d578 https://sourcegraph.com/github.com/chrismwendt/graphql-ws-la... (it's pretty slow/janky at the moment and could be improved, but works). If you're interested in interfacing with other tools too, you might be interested in LSP (Language Server Protocol): - https://langserver.org/ https://langserver.org/ - https://microsoft.github.io/language-server-protocol/specification https://microsoft.github.io/language-server-protocol/specifi...
ntonozzi 8y ago
Congrats Quinn and company! I hope there's not too much of my code lurking in there =)
- slimsag 8y ago
  Oh! Hey Nico :) Hope all is going well! I miss having ya around!
brylie 8y ago
Cool! Are you working on IDE plygins, e.g. for Atom?

asdkhadsj 8y ago

Holy cow, does this mean I can use source graph as a single dev wanting the features, but not able to upload private source to a cloud service? Fwiw, I'd love to pay as a private user for this type of service. It's just that for many, we can't use a tool that leaks code. I look forward to that day, if it's not today :)

deleted 8y ago
[deleted]
sqs 8y ago
Yes! There is a `docker run` command that will spin up a Sourcegraph instance for you at https://about.sourcegraph.com/docs/ https://about.sourcegraph.com/docs/. The self-hosted Docker (or, for big clusters, Kubernetes) deployment method has been there for about 10 months now, and devs/companies really like it. Making it so easy to self-host Sourcegraph was a big win, and going open-source is another step in that direction. What is new is that you can now also build Sourcegraph from source and see/hack on/etc. our source code.
- smivan 8y ago
  Awesome!! So the Go code intelligence is now free for single devs who are self hosting?
  - sqs 8y ago
    Yes!

emmanueloga_ 8y ago

The kind of cross-reference SourceGraph provides is a must have for me after a few years of using a tool like it to understand big projects (I don't use SourceGraph, I use Grok/CodeSearch). Google open sourced a version of Grok some time ago (https://kythe.io/ https://kythe.io/). [1] A quick search also found this `OpenGrok` thing which I assume must be yet another generic indexer [2]. I wonder if there are other popular tools out there providing generic cross-referencing of code bases? <SourceGraph> I was trying to find the equivalent of this Kythe page (how to write an indexer)[6] on SourceGraph docs, after 20 minutes I was still confused. I wish there was more clarity on the different pieces of the system. At first I thought SourceGraph could create a cross reference index from an LSP server, but I guess makes more sense it is the order way around? (SG can talk to text editors via LSP). It must be a lot of work to make integrate any single language with both SG or Kythe and LSP, perhaps there's some tools or methodologies to make this easier? <Tangent> I feel like language designers don't always think about building their implementations with ease of integration as a goal. I remember wanting to play with doing some AST transformations with both TypeScript and Dart (independently) and finding both languages made the task so hard and tedious. Mandatory remark: this is so easy with Lisp derivatives :-) Things that a good language+ecosystem should have, IMO: * Easy to retrieve and manipulate CST (for things like writing refactoring and querying tools, writing code formatters/pretty printers, etc). Many languages provide the syntax tree say, without comments, so it is hard to write a pretty printer. * Documentation generator with cross-reference navigation. * Play along nicely with generic build tools like Ninja [3]. * Good debugger and value-pretty-printer support (so I can just drop a breakpoint and print anything easily). And some sort of API to use the debugger programmatically or connect to a remote process. * An LSP implementation [4], although it seems pretty much every popular language is getting one these days! 1: https://en.wikipedia.org/wiki/Google_Kythe https://en.wikipedia.org/wiki/Google_Kythe 2: http://oracle.github.io/opengrok/ http://oracle.github.io/opengrok/ 3: https://ninja-build.org/ https://ninja-build.org/ 4: https://langserver.org/ https://langserver.org/ 5: https://about.sourcegraph.com/docs/code-intelligence/ https://about.sourcegraph.com/docs/code-intelligence/ 6: https://kythe.io/docs/schema/writing-an-indexer.html https://kythe.io/docs/schema/writing-an-indexer.html <edit: line breaks>

tmerr 8y ago

jaytaylor 8y ago

I was lucky to connect with Quinn and get the opportunity to volunteer as a guinea pig for open sourcing this code :). Sourcegraph is an interesting product and company. I see the potential for it to have a bigger positive impact on developer productivity than even Jetbrains IntelliJ has, because the reach can go beyond any particular editor or frontend. It could be a ubiquitous solution to efficient code search and discovery. Will definitely be watching how things unfold for Sourcegraph. If you haven't already read their Master Plan, it's a compelling vision. https://about.sourcegraph.com/plan https://about.sourcegraph.com/plan

sqs 8y ago
Thanks for running through the README and helping us prep the open-source release last week!

mikekchar 8y ago

I'm especially interested in open source/free software businesses and business models. Are you planning to do an open core kind of thing, or maintain full on open source while charging for hosting and/or other services? I realise that might be a difficult question to answer as you might want to keep your options open, so I'm not asking for a commitment :-) I'm just curious about your plans. Also do you ever think you might add (for want of a better phrase) custom development services if you aren't already? For example, allowing people to pay a fee to get specific features added to Sourcegraph. What are the kinds of benefits and risks you might see from going in that kind of direction? I've noticed that a lot of open core businesses tend to avoid offering customer development services -- and if they do it's often perceived as a cost centre rather than a profit centre. It's something that is provided out of necessity of having to provide custom solutions for a few big customers, but the services team often appears to be at odds with the core development team. As you have obviously gone through the thought process of how to make money when your software is out in the open, I wonder what kinds of thoughts you have on that topic. It's really nice to see this kinds of stuff happening and I wish you the best luck possible in your endeavours!

pietroglyph 8y ago
It looks like they're going with open core: https://about.sourcegraph.com/pricing https://about.sourcegraph.com/pricing
- jordigh 8y ago
  > Open. For business. You know, it's funny that they are proudly using this slogan. OpenBSD has mocked it in the past, referring to Intel's firmware blobs for wireless NICs: > Some asshole said he was "open" > but he was only open for business. https://www.openbsd.org/lyrics.html#40b https://www.openbsd.org/lyrics.html#40b Open core is no big innovation. It's plain ol' proprietary software. It's just like macOS. Most software nowadays has significant free components. The only company that I know that does truly free commercial software is Red Hat. They don't sell you a drop of non-free software. Some people will argue that the degree of free software in your proprietary software makes a difference. I argue that it only matters if you need any of the proprietary bits or not. A non-free but desired bit of software in a package can be very restrictive and have the same undesirable impacts as having the whole thing being non-free.

mholt 8y ago

Awesome! You guys did what I could not. Congratulations! So excited to see where this takes Sourcegraph, and the community around it. In my experience, even just using Sourcegraph to browse source code in open source repos is way faster than the native git host browsing (BitBucket, GitHub, Gitlab, etc). We use it daily at my lab, especially as I help others learn the code base. Thanks for Sourcegraph.

mickael-kerjean 8y ago

Will definitely use to navigate code. It's way better than github/gitlab/bitbucket.

chubot 8y ago

Very cool! Does it have C or C++ support? Last time I looked it didn't. Something that understands Python calls into C and vice versa would be interesting to me as well.

sqs 8y ago
Sourcegraph has experimental C/C++ support. Emphasis on experimental. It uses the best C/C++ language server (based on the Language Server Protocol) we could find, but that still struggles with some codebases. You can try it on any open-source C/C++ file on Sourcegraph.com before you spin up a self-hosted instance for your own code, such as: https://sourcegraph.com/github.com/antirez/redis/-/blob/src/expire.c#L60:9 https://sourcegraph.com/github.com/antirez/redis/-/blob/src/... Here is more info on using C/C++ on Sourcegraph (and an animated GIF): https://github.com/sourcegraph/lsp-adapter/tree/master/dockerfiles/cpp https://github.com/sourcegraph/lsp-adapter/tree/master/docke...
- romed 8y ago
  That seems to have a lot of rough edges. For me, when browsing C++, there is exactly 1 killer feature: find callers. I recently deployed Kythe just to get this and it took many hours of my time. I would love to have something a little easier to maintain (that works, though).
robinshen 8y ago
OneDev developer here. OneDev is a self-hosted git server and it provides symbol cross referencing for main stream languages including Java, JavaScript, CSharp, C, C++, PHP, Go, Python, R, CSS, LESS, SCSS. Taking linux project for example: https://go.onedev.io/projects/linux/blob/master/tools/lguest/lguest.c?mark=2967.2-2967.15 https://go.onedev.io/projects/linux/blob/master/tools/lguest...

StavrosK 8y ago

Is this the thing that caused furore a while ago because it was installed on some editors and it sent all your code to a third party server, or am I thinking of something completely different?

sqs 8y ago
No, that is https://kite.com/ https://kite.com/ (absolutely no relation to Sourcegraph).
posnet 8y ago
No that was Kite, they paid editor plugin developers to take over the plugin and then added their code complete which requires sending all source code to their servers. https://qz.com/1043614/this-startup-learned-the-hard-way-that-you-do-not-piss-off-open-source-programmers/ https://qz.com/1043614/this-startup-learned-the-hard-way-tha... https://kite.com https://kite.com
- StavrosK 8y ago
  That's the one, thanks. I can't wait to try Sourcegraph in that case, it sounds very useful!
vanesa- 8y ago
Nope, not Sourcegraph!!!

mmirate 8y ago

> Together, these companies succeeded in bringing computing to billions of people. But these billions of people are still using software applications built by just 0.2% of the world's population (those who can code). > The next step is to make it so billions of people, not just 0.2% of the world population, can build software (not just use it). I must admit, this sounds indubitably noble. But if you think that your software-engineering career is safe from (partial) automation, then these Sourcegraph folks have an iridium-plated bridge to sell you. In not-so-tongue-and-cheek terms: if Sourcegraph succeeds, then tomorrow's software engineers will be - at most - as valuable as today's factory-robot operators. * * * (Of course, anyone worth their salt in this industry ought to be able to find the LSP's inherent limitations, and structure their codebases accordingly.) EDIT: counterarguments are more persuasive than downvotes.

korantu 8y ago

"Master plan" link [0] is 404 in the linked article. (Not sure if this is right way to report it) [0] https://about.sourcegraph.com/plan https://about.sourcegraph.com/plan

sqs 8y ago
Thanks, just fixed (should be live soon).

lhorie 8y ago

We've been using Sourcegraph at Uber and it has been working pretty well. We saw a performance regression last week with `max` queries but it got fixed fairly quickly (less than a week from reporting to having the fix deployed) Some wishlist things: - would be nice if the query language was better documented (not the GraphQL console, the help section for that is comprehensive) - a coworker had expressed a desire for multi-line searches (which is supposedly in the pipeline)

slimsag 8y ago
(Sourcegrapher here) Just curious, have you discovered this page before from the in-app help area? https://about.sourcegraph.com/docs/search/query-syntax/ https://about.sourcegraph.com/docs/search/query-syntax/ (We will improve our docs here & make them stand out more in the app regardless, I think that is a great suggestion.) I also very much want multi-line searches, I can't wait for them :)
- lhorie 8y ago
  Yeah, this help page is fairly easy to find (two clicks: ? icon -> "How to search"). I figured I'd ask because for example `max` is not documented there, even though it's pretty important for the sort of stuff I use Sourcegraph for (i.e. finding out all the repos that use X).
sqs 8y ago
Sourcegraph CEO here. Thanks for the feedback and sorry for the regression! I filed an issue to improve our search docs at https://github.com/sourcegraph/sourcegraph/issues/190 https://github.com/sourcegraph/sourcegraph/issues/190 (will fix in the next day or so) and to add multi-line searches at https://github.com/sourcegraph/sourcegraph/issues/191 https://github.com/sourcegraph/sourcegraph/issues/191 (this is one of 3 significant search improvements coming in the next few months).
- lhorie 8y ago
  Awesome! Rock on!

kemitchell 8y ago

Was Fair Source a detour or a stepping stone?

slimsag 8y ago
Sourcegrapher here (speaking from my own viewpoint): Stepping stone. A fully open-source codebase[1] with Apache license is simply a better match as we aim to have a completely open and transparent product and company, where even e.g. planning is done in the open[2]. [1] https://github.com/sourcegraph/sourcegraph https://github.com/sourcegraph/sourcegraph [2] https://github.com/sourcegraph/about/blob/master/projects/about-repository.md#open-product-open-company https://github.com/sourcegraph/about/blob/master/projects/ab...

stephenr 8y ago

So before anything else - congratulations and thank you for releasing something as open source, under a business-friendly licence. Regardless of the project itself, that should be commended. However. I've never paid much attention before, so perhaps I'm missing something.. but this "just" seems to be the code intelligence you'd expect to find in a good IDE. Is there more to it? Or is this the 'solution' for the lack of such code intel in the plethora of text editors people use these days?

slimsag 8y ago
Sourcegrapher here -- two main things: 1. Sourcegraph offers the code intelligence your used to in your editor, but instead of being in your editor it is when you are browsing code on your code host (i.e. on GitHub, BitBucket, etc.) Even when reviewing pull requests / diffs! 2. Sourcegraph provides fast, advanced code search across multiple repositories -- akin to what Google and Facebook offer their devs internally. Regular expression support, extremely up-to-date results, etc. You can read more about this here: https://about.sourcegraph.com/docs/search https://about.sourcegraph.com/docs/search

k4ch0w 8y ago

This is amazing for secure code reviews. I have already spun up and instance and am demoing it for my team. Is it possible to get Jira integration? Can you test bitbucket.org connection with credentials some place I am not seeing? I can't tell what is causing it to fail. Is there a way to add markers in the code and then a queue to clear it? I.E I mark hotspots in the code and want a developer to review them or edit them. I think a current pain point would be no flat file upload. I understand you only need the .git directory and this was designed with a developer in mind. A lot of times when doing a secure code review we don't get credentials, we just get a current clone of the code. I just tried to do it using the instructions and had some trouble ended up just cloning it on the host machine. I usually get a zip/tar of the code. Another feature that would be awesome is export a search to a CSV file. I could script together something for Jira integration with that pretty quickly. Something to get the line affected #, branch and filepath. The subscription to searches was a brilliant idea. I wish I thought of it. SSO already integrated? Nice man! I'd make the discussions default too, I discovered that after playing with it for a bit. It's an awesome feature. I look forward to seeing more of it.

slimsag 8y ago
Heya, I'm the Sourcegrapher working on the discussions feature[1]. It's great to hear your are enjoying it. We're looking for users to help drive the direction of the feature as it matures and really influence what we work on next with it, so if you're up for it I'd really love to get your feedback on it (as much as you'd be willing to provide). :) Jira integration -> We're working on it! Mind commenting on https://github.com/sourcegraph/sourcegraph/issues/141 https://github.com/sourcegraph/sourcegraph/issues/141 ? Markers in the code -> If you could archive or dismiss the inline discussion threads (so that they no longer show up in the inline view) would that work well? Flat file upload -> Could you file a feature request for this at https://github.com/sourcegraph/sourcegraph/issues/new/choose https://github.com/sourcegraph/sourcegraph/issues/new/choose with some more details on how e.g. you would wish to give Sourcegraph access to the code? Exporting search to CSV -> It might be pretty easy to write something like this with our `src` CLI tool: https://github.com/sourcegraph/src-cli https://github.com/sourcegraph/src-cli You can get JSON output via e.g. `src search -json 'repogroup:sample error'` Hope to hear more from you! :) [1] https://about.sourcegraph.com/blog/discuss-code-and-docs-in-repositories https://about.sourcegraph.com/blog/discuss-code-and-docs-in-...

frou_dh 8y ago

I remember Sourcegraph as being a sort of sidecar to your text editor. Based on your caret position, it would automatically show you crawled examples of how other people use the functions/classes you are dealing with. Has that been de-emphasised or gone away?

slimsag 8y ago
I think you might be confusing Sourcegraph with Kite[1] (zero relation with Sourcegraph). Kite tries to do something like what you are describing. With Sourcegraph, we have editor plugins for various editors[2] but they only perform "Open current file on Sourcegraph" and "Search selection on Sourcegraph" actions, nothing like what you are describing yet. [1] https://kite.com/ https://kite.com/ [2] https://about.sourcegraph.com/docs/integrations/editor-plugins https://about.sourcegraph.com/docs/integrations/editor-plugi...
- wholien 8y ago
  It might have been what I worked on at my internship at SG; demo here: https://vimeo.com/164809163 https://vimeo.com/164809163 It makes sense not to make this a big part of SG though, since latency is a huge issue and detracts from the rest of the product which is amazing!

JoshuaAshton 8y ago

Hard pass. I dont want an "intelligent" search I just want a search. Will be sticking with Agent Ransack.

slimsag 8y ago
To be clear, this isn't "intelligent" search. There is no magic here. It's just fast, advanced code search across multiple repositories (like what Google and Facebook offer their devs internally). This means support for regular expressions, always getting extremely up-to-date results, etc. You can read more about it here: https://about.sourcegraph.com/docs/search https://about.sourcegraph.com/docs/search There is "code intelligence", which is a phrase we use to describe IDE-like features such as jump-to-definition, find-references, etc.

Dowwie 8y ago

Know those cool features in your editor that allow you to hover over source and get more detail on what you are looking at? Also, know that feature in an editor that lets you go to definition? Imagine this and not within your editor but rather your browser, whatever host you choose, such as GitHub or GitLab. Sourcegraph enables this feature with browser extensions.

sqs 8y ago
Sourcegraph CEO here. This is a great way to describe how Sourcegraph integrates with your code host. I love it. :) Links to save you from searching: https://chrome.google.com/webstore/detail/sourcegraph/dgjhfomjieaadpoljlnidmbgkdffpack?hl=en https://chrome.google.com/webstore/detail/sourcegraph/dgjhfo... https://addons.mozilla.org/en-US/firefox/addon/sourcegraph/ https://addons.mozilla.org/en-US/firefox/addon/sourcegraph/

mariusmg 8y ago

So what's the use case for this ? If somebody wants "Find all references" , what's preventing them to open the code in Visual Studi/Intellij/Eclipse etc.

mysteriosaga 8y ago

Open source is the way to go. Onwards and upwards.

root3 8y ago

is it working only with git vcs?