4 ms·

I was recently looking into CRDTs for text-editing, and was surprised to see that Github devs were working on an experimental text editor (Xray) [1] along with

by bhl 7y ago

I was recently looking into CRDTs for text-editing, and was surprised to see that Github devs were working on an experimental text editor (Xray) [1] along with a finer-grained version control system for real-time collaborative editing (Memo) [2]. That project, however, seems to have been lost (and archived) in the Microsoft reshuffling of things. [1] https://news.ycombinator.com/item?id=16525735 https://news.ycombinator.com/item?id=16525735 [2] https://github.com/atom-archive/xray/tree/master/memo_core https://github.com/atom-archive/xray/tree/master/memo_core

anchpop 7y ago
Memo is really cool, I'm sad it seems to not be worked on anymore.
vmchale 7y ago
Wonderful! Was that the fruit of your searches? It seems like an interesting topic.
- bhl 7y ago
  There are other libraries that I've found and used, Yjs [1] for an example because it supports the text editor (ProseMirror) I'm building upon. However, a good version control system for CRDTs, like git for flat text, does not still exist. [1] http://yjs.dev/ http://yjs.dev/
touisteur 7y ago
Yeah github seemed to also have some interesting innovations working with in the (programming) language classification domain. There has been some work done by people from the ML@B over at Berkeley (?) to greatly improve github-linguist, the tool GitHub uses (I think) to identify programming languages in GitHub repositories. The project is named lexicon (used to be hosted at https://lexicon.github.io/ https://lexicon.github.io/ but now it 404s...) and the results seemed very interesting. Tried pinging github support about open-sourcing this work, but it seems it's not going to happen. I had very friendly and helpful people answering me, and I totally understand not wanting to open-source something. I just feel a bit sad for the lost opportunity.
- bhl 7y ago
  I found this link [1], but nothing related to ML@B with it. [1] https://github.com/github/linguist https://github.com/github/linguist
  - touisteur 7y ago
    Best I could locate right now is https://medium.com/@ml.at.berkeley/demo-day-september-2016-252bcd8b395 https://medium.com/@ml.at.berkeley/demo-day-september-2016-2... "Every day, millions of files are uploaded onto Github, a coding repository where users share code and collaborate on projects. But how do you tell what language those files are written in? Identifying programming languages are surprisingly hard. Symbols used in one language often have different meanings in another. For example, ‘#’ in python indicates a comment, while in C it indicates a preprocessor command. Even worse, code in one language can actually contain code in different languages, such as HTML, which can contain CSS and/or Javascript. At the moment, Github uses a giant checklist to identify unique quirks in the language. For example, if the code contains “:- module”, then it’s probably"); background-size: 1px 1px; background-position: 0px calc(1em + 1px);"> Mercury. However, most languages simply don’t have enough unique quirks for this method to be accurate enough. The current solution the Github team at ML@B came up with is to use a machine learning algorithm called a Naïve Bayes Classifier. To optimize the program, they used Github’s checklist as a guideline for choosing the correct language rather than a hard-and-fast rule. Currently, the team is working on scraping Rosetta Code, a large repository of code in hundred of different programming languages, for data that they can use for training the program and testing its accuracy. Eventually, they will want to see if implementing other models, such as neural networks, can improve accuracy." I'm certain I saw more advanced claims of success on this project and that github knows about it, since last time I asked they were 'still talking about it internally', but every link I had is dead... WELL now that I googled seriously, I think I was too pessimistic, there seems to be some movement on this front at github : https://github.blog/2019-07-02-c-or-java-typescript-or-javascript-machine-learning-based-classification-of-programming-languages/ https://github.blog/2019-07-02-c-or-java-typescript-or-javas... Excited to see what's coming out of this new endeavour... Just sent an email to github support, fingers crossed.

by bhl 7y ago

anchpop 7y ago

Memo is really cool, I'm sad it seems to not be worked on anymore.

vmchale 7y ago

Wonderful! Was that the fruit of your searches? It seems like an interesting topic.

bhl 7y ago
There are other libraries that I've found and used, Yjs [1] for an example because it supports the text editor (ProseMirror) I'm building upon. However, a good version control system for CRDTs, like git for flat text, does not still exist. [1] http://yjs.dev/ http://yjs.dev/

touisteur 7y ago

Yeah github seemed to also have some interesting innovations working with in the (programming) language classification domain. There has been some work done by people from the ML@B over at Berkeley (?) to greatly improve github-linguist, the tool GitHub uses (I think) to identify programming languages in GitHub repositories. The project is named lexicon (used to be hosted at https://lexicon.github.io/ https://lexicon.github.io/ but now it 404s...) and the results seemed very interesting. Tried pinging github support about open-sourcing this work, but it seems it's not going to happen. I had very friendly and helpful people answering me, and I totally understand not wanting to open-source something. I just feel a bit sad for the lost opportunity.

bhl 7y ago
I found this link [1], but nothing related to ML@B with it. [1] https://github.com/github/linguist https://github.com/github/linguist
- touisteur 7y ago
  Best I could locate right now is https://medium.com/@ml.at.berkeley/demo-day-september-2016-252bcd8b395 https://medium.com/@ml.at.berkeley/demo-day-september-2016-2... "Every day, millions of files are uploaded onto Github, a coding repository where users share code and collaborate on projects. But how do you tell what language those files are written in? Identifying programming languages are surprisingly hard. Symbols used in one language often have different meanings in another. For example, ‘#’ in python indicates a comment, while in C it indicates a preprocessor command. Even worse, code in one language can actually contain code in different languages, such as HTML, which can contain CSS and/or Javascript. At the moment, Github uses a giant checklist to identify unique quirks in the language. For example, if the code contains “:- module”, then it’s probably"); background-size: 1px 1px; background-position: 0px calc(1em + 1px);"> Mercury. However, most languages simply don’t have enough unique quirks for this method to be accurate enough. The current solution the Github team at ML@B came up with is to use a machine learning algorithm called a Naïve Bayes Classifier. To optimize the program, they used Github’s checklist as a guideline for choosing the correct language rather than a hard-and-fast rule. Currently, the team is working on scraping Rosetta Code, a large repository of code in hundred of different programming languages, for data that they can use for training the program and testing its accuracy. Eventually, they will want to see if implementing other models, such as neural networks, can improve accuracy." I'm certain I saw more advanced claims of success on this project and that github knows about it, since last time I asked they were 'still talking about it internally', but every link I had is dead... WELL now that I googled seriously, I think I was too pessimistic, there seems to be some movement on this front at github : https://github.blog/2019-07-02-c-or-java-typescript-or-javascript-machine-learning-based-classification-of-programming-languages/ https://github.blog/2019-07-02-c-or-java-typescript-or-javas... Excited to see what's coming out of this new endeavour... Just sent an email to github support, fingers crossed.