4 ms·

Whoa, wasn't expecting to see this on HN! I've only done a week of work on this, super early stages - here are our plans for it: * 100% OSS tool, run locally o

Whoa, wasn't expecting to see this on HN! I've only done a week of work on this, super early stages - here are our plans for it: * 100% OSS tool, run locally on your machine (static Go binary) or use https://doctree.org https://doctree.org (not online yet, plug in a repository name, get docs) - really want this to be a proper, useful FOSS tool. * Work with any language, based on tree-sitter. * Provide symbol-level search functionality. * Surface real-world usage examples automatically, probably based on some statistical analysis of how functions are commonly used in open source code via Sourcegraph API, similar to what https://codestat.dev https://codestat.dev is doing. Tech details (again, just a week in): * Go for backend, Elm for frontend * Indexers will be written in Go, use tree-sitter queries to produce a standard index schema which then gets served to Elm frontend for rendering. https://github.com/sourcegraph/doctree/blob/main/doctree/schema/schema.go https://github.com/sourcegraph/doctree/blob/main/doctree/sch... Probably not worth trying out right now, but if you're interested in it we set up a Discord server for collaboration, etc. https://discord.com/invite/vqsBW8m5Y8 https://discord.com/invite/vqsBW8m5Y8 Happy to answer any questions!

learndeeply 4y ago
Why is this open source? Sourcegraph could make this a successful paid, enterprise product.
- slimsag 4y ago
  Fair question! I want to say really, the goal is just to find a way to make an actually useful library documentation tool like this that people enjoy using. Myself, Beyang (CTO), and Quinn (CEO) have all wanted something like this for a while and think there's potential for something cool here. It'll require _a lot_ of iteration to make this work really well, though, and make it something that everyone feels good about using *for their projects*. Don't want there to be barriers to using it, if it was enterprise/paid it'd be tough to do that. There will be features/functionality doctree _can_ gain if you connect it to a Sourcegraph instance, but largely because they'd be impossible to do otherwise: * Usage examples - we need statistical analysis of a large corpus of open source code to find good real-world usage examples, so we'll leverage Sourcegraph for that (it already has that data.) * Respecting repository permissions, OAuth integration, etc.; very important in enterprise environments, super complex/annoying to do. Sourcegraph already has all this data about your github/gitlab/bitbucket repos, user accounts, etc. and so maybe you can one-click connect doctree to a Sourcegraph instance to gain this functionality if you're some large enterprise that needs it. I think there are some great synergistic ways doctree will work with Sourcegraph if you use that (or are OK with it contacting Sourcegraph.com for public code, but very important to make that respectful / opt in.) I want to be clear, though, doctree is 100% open source, it'll be a proper OSS project - just want to make a useful tool for everyone first and foremost.
- learndeeply 4y ago
  Can someone explain why I was downvoted?
  - slimsag 4y ago
    I interpreted your question as "How does it make sense for Sourcegraph to build this as an open source thing, and not as a paid product?" which I thought was a super reasonable question. I think "Why is this open source?", though, is what got you down-voted because it implies it should be closed source, when folks obviously prefer open source. Hope that helps!
bradrn 4y ago
After a week of work, how much of this have you managed to accomplish so far? (Very little, I assume, but I could be wrong.)
- slimsag 4y ago
  Quite a lot, actually! It's definitely not ready for prime-time usage or anything, but I feel good about the week of time on it: * There are some Docker commands you can use to try it out on some Go code right now[0] (still working on getting binary releases for each OS so Docker is not necessary) * There's a not-too-bad frontend (written in Elm), screenshots in the README are real & it all functions! * There's an indexer implemented for the Go language[1] that runs tree-sitter queries & emits a basic schema[2], the idea is each language would emit to a common schema like this and then the frontend can serve it, we can index it for search, etc. So, I mean, yeah - just a week into it, but like - you can already view documentation for Go functions in it so moving quickly! [0] https://github.com/sourcegraph/doctree#try-it-out-extremely-early-stages https://github.com/sourcegraph/doctree#try-it-out-extremely-... [1] https://github.com/sourcegraph/doctree/tree/main/doctree/indexer/golang https://github.com/sourcegraph/doctree/tree/main/doctree/ind... [2] https://github.com/sourcegraph/doctree/blob/main/doctree/schema/schema.go https://github.com/sourcegraph/doctree/blob/main/doctree/sch...
  - bradrn 4y ago
    Impressive, thanks for clarifying!

learndeeply 4y ago

Why is this open source? Sourcegraph could make this a successful paid, enterprise product.

slimsag 4y ago
Fair question! I want to say really, the goal is just to find a way to make an actually useful library documentation tool like this that people enjoy using. Myself, Beyang (CTO), and Quinn (CEO) have all wanted something like this for a while and think there's potential for something cool here. It'll require _a lot_ of iteration to make this work really well, though, and make it something that everyone feels good about using *for their projects*. Don't want there to be barriers to using it, if it was enterprise/paid it'd be tough to do that. There will be features/functionality doctree _can_ gain if you connect it to a Sourcegraph instance, but largely because they'd be impossible to do otherwise: * Usage examples - we need statistical analysis of a large corpus of open source code to find good real-world usage examples, so we'll leverage Sourcegraph for that (it already has that data.) * Respecting repository permissions, OAuth integration, etc.; very important in enterprise environments, super complex/annoying to do. Sourcegraph already has all this data about your github/gitlab/bitbucket repos, user accounts, etc. and so maybe you can one-click connect doctree to a Sourcegraph instance to gain this functionality if you're some large enterprise that needs it. I think there are some great synergistic ways doctree will work with Sourcegraph if you use that (or are OK with it contacting Sourcegraph.com for public code, but very important to make that respectful / opt in.) I want to be clear, though, doctree is 100% open source, it'll be a proper OSS project - just want to make a useful tool for everyone first and foremost.
learndeeply 4y ago
Can someone explain why I was downvoted?
- slimsag 4y ago
  I interpreted your question as "How does it make sense for Sourcegraph to build this as an open source thing, and not as a paid product?" which I thought was a super reasonable question. I think "Why is this open source?", though, is what got you down-voted because it implies it should be closed source, when folks obviously prefer open source. Hope that helps!

bradrn 4y ago

After a week of work, how much of this have you managed to accomplish so far? (Very little, I assume, but I could be wrong.)

slimsag 4y ago
Quite a lot, actually! It's definitely not ready for prime-time usage or anything, but I feel good about the week of time on it: * There are some Docker commands you can use to try it out on some Go code right now[0] (still working on getting binary releases for each OS so Docker is not necessary) * There's a not-too-bad frontend (written in Elm), screenshots in the README are real & it all functions! * There's an indexer implemented for the Go language[1] that runs tree-sitter queries & emits a basic schema[2], the idea is each language would emit to a common schema like this and then the frontend can serve it, we can index it for search, etc. So, I mean, yeah - just a week into it, but like - you can already view documentation for Go functions in it so moving quickly! [0] https://github.com/sourcegraph/doctree#try-it-out-extremely-early-stages https://github.com/sourcegraph/doctree#try-it-out-extremely-... [1] https://github.com/sourcegraph/doctree/tree/main/doctree/indexer/golang https://github.com/sourcegraph/doctree/tree/main/doctree/ind... [2] https://github.com/sourcegraph/doctree/blob/main/doctree/schema/schema.go https://github.com/sourcegraph/doctree/blob/main/doctree/sch...
- bradrn 4y ago
  Impressive, thanks for clarifying!