7 ms·

Show HN: Cyborg Writer - In-Browser Text Editor with Neural Autocomplete

71 points by antimatter15 9y ago

rborn 9y ago
"Today I went on a hike with Benny, while [tab]" "Soldier: I will not see your son to your house." errr.. not sure how that follows..
ttul 9y ago
Mobile Safari just sits there...
giosch 9y ago
I was expecting it to at least generate valid words, but that doesn't seem to be the case...
- ballenf 9y ago
  Safari on latest MacOS gives me unintelligible words pretty often. My input: "I went to the grocery store to pick up some milk for" Output on wikipedia setting: "I went to the grocery story to pick up some milk for ... proposes of such to take a amplifier and the exporteruation of the player of former" None of the options seem to key off of the text of my sentence except for vaguely the last word (in my case a generic preposition). Am I doing it wrong? Here's another example, with the ellipses where I pressed tab: > As I walked along the forest trail, I came to a most amazing [...] the history of the town himself, the complete material was made to be a traditional
  - yorwba 9y ago
    You're starting the text like a story (in the first person even!). Wikipedia articles aren't usually written like that, so the network is in unfamiliar territory. Its only option is to quickly revert back to something closer to the style of an informative article.
- bijection 9y ago
  What is it generating for you? (and what OS/browser are you using?)
stmw 9y ago
This is actually pretty neat, although requires more digging
cd0x23 9y ago
I like to lick [my gun, huh, I got some things since I couldn't see.] - Tupac Shakur
antimatter15 9y ago
Hey HN! This is our latest demo for TensorFire (https://tenso.rs/ https://tenso.rs/), a Javascript library for running neural networks in the browser GPU-accelerated by WebGL. Each of these models is a simple two-layer char-rnn (the kind explored in Karpathy's "Unreasonable Effectiveness of RNNs" post http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/). When you hit "Tab" the editor feeds in the contents of your document letter-by-letter into a network which is trying to predict the most likely next letter. It samples from that probability distribution and feeds it back into the network to generate an arbitrarily long string. We've published code and instructions for training new models on Github (https://github.com/tensorfire/cyborg-writer/tree/master/training https://github.com/tensorfire/cyborg-writer/tree/master/trai...). With about a megabyte of any sort of text, you can train a model and easily embed it into any web application. We (@bijection and @antimatter15) will be answering questions in the comments.
- projectant 9y ago
  No offence, but with a megabyte of text per speaker I feel I could do better with word n-gram Markov models (even bigrams), and random path choice. I think deep learning is great and all ( and I'm meaning to learn it ) but shouldn't it be able to do far better than Markov models, or other simple things? Image captioning? Incredible. Deep Mind winning video games? Incredible. Style transfer? Incredible. With one exception ( I saw on HN ages ago, sorry I have no link -- it basically generates novel text using deep learning, across all sorts of genres, such as "academic paper", "math paper", "novel", "film script" and I found the results remarkable and interesting ) I question if many text applications are doing better than Markov. I think the issue is there is something fundamental and sophisticated about human language which our current deep learning models, with all their omniscient benevolence ( or whatever ), are missing. There's something deep about the structure of language that we are not modelling yet in deep learning as far as I've seen. When we do .... boom ... computers that learn from the internet and amaze us all. Then we'll have something to shine, smile about or fear. Sorry for the digression and what may be inapplicable comparisons. I can get impassioned about this topic.
  - 9y ago
_grep_ 9y ago
Hitting tab crashes Chrome for me.
- bijection 9y ago
  Thanks for the tip! What OS and chrome version are you using?
  - snerbles 9y ago
    Not OP, same issue here. Whole browser just locks up, pretty impressive. Windows 7, Version 61.0.3163.100 (Official Build) (64-bit)
    - _grep_ 9y ago
      Same specs here.
    - mipmap04 9y ago
      Reporting the same issue, same build.
nsomaru 9y ago
Fun with Shakespeare: It seemed, somehow mock'd; Unbashed as it was my father. My lady shall have him, perchance and reasons will abide it. Upon his unspoken thing, the devil shall see. MISTRESS QUICKLY: Hide amongst you the gods? Wherefore comes here, sir?
westoncb 9y ago
Pretty hilarious. I've found Tupac to work the best, e.g. (my starting text in quotes): "I was reading Hacker News when" he gladed out Somebody excuse I went down today I wonder if I give a fuck, when I creep For some reason Donald Trump always seems to produce garbage...
- sajattack 9y ago
  "For some reason Donald Trump always seems to produce garbage..." I think that means it's working correctly.
jaster 9y ago
Ha very nice! I made a vary similar (but much rougher around the edges) project for fun a few months ago using a smaller french corpus of press releases from La Quadrature du Net (a non-profit lobbying for net neutrality in France and Europe). There is a live demo at http://jorquera.net/quadra_neural/ http://jorquera.net/quadra_neural/. You can press tab to auto-complete word-by-word, or just start typing and the suggestions update on the fly. Instead of "Weirdness" I called my cursor "Drunkenness" :) I used keras-js (https://github.com/transcranial/keras-js https://github.com/transcranial/keras-js) to run my model in the browser. If you are interested in this kind of experiments, do go for it! It's quite accessible, and projects of this scale do not require heavy hardware (I generated my models using a GTX 770). If you need a kickstart, in addition to the repo from the OP all the code for my project is Free Software (https://github.com/tomjorquera/quadra_neural https://github.com/tomjorquera/quadra_neural), and I tried to document the generation process in a jupyter notepad https://github.com/tomjorquera/quadra_neural/blob/master/generation/model_gen.ipynb https://github.com/tomjorquera/quadra_neural/blob/master/gen.... While I did it on a french corpus, it should work with any language. I used two main resources for inspiration : - http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py https://github.com/fchollet/keras/blob/master/examples/lstm_...
dchuk 9y ago
We have to be pretty close to someone having a system to generate mountains of convincing/unique "content" using Neural Nets (or something comparable) for the purposes of web spam/seo right? Back in the day, SEOs would use spun content to generate lots of unique variants. But at this point, surely a ML algorithm can crack this nut?
- PeterisP 9y ago
  Sure, at this point building RNNs that generate unique text on a particular topic isn't a big thing to do, it's something that can be given as a home exercise for undergrads.
c3534l 9y ago
Eh, it's scarcely better than a Markov chain. I was hoping for something a little more useful, like the ability to scroll through many similar words or concepts.
- gwern 9y ago
  You might like this, then: a word2vec tool which suggests words of the same rough meaning starting with a particular letter or spelling, allowing you to write Genesis solely in words starting with 'a' https://llamasandmystegosaurus.blogspot.com/2017/05/alpha.html https://llamasandmystegosaurus.blogspot.com/2017/05/alpha.ht... or generate large numbers of rhyming definitions (see comments), some examples of which: Cat: purr fur Dolphin: bark shark Puppy: yelp welp Shetland pony: shag nag Mosquito: mighty bitey Shrimp: chill krill
novium 9y ago
It's pretty neat, even got a t.co URL. Sadly, it didn't lead anywhere.
sterex 9y ago
Is there a technical reason why it generates text with lot of spelling mistakes? Ayn Rand seems very sloppy with her spelling!

71 points by antimatter15 9y ago

"Today I went on a hike with Benny, while [tab]" "Soldier: I will not see your son to your house." errr.. not sure how that follows..

ttul 9y ago

Mobile Safari just sits there...

giosch 9y ago

I was expecting it to at least generate valid words, but that doesn't seem to be the case...

ballenf 9y ago
Safari on latest MacOS gives me unintelligible words pretty often. My input: "I went to the grocery store to pick up some milk for" Output on wikipedia setting: "I went to the grocery story to pick up some milk for ... proposes of such to take a amplifier and the exporteruation of the player of former" None of the options seem to key off of the text of my sentence except for vaguely the last word (in my case a generic preposition). Am I doing it wrong? Here's another example, with the ellipses where I pressed tab: > As I walked along the forest trail, I came to a most amazing [...] the history of the town himself, the complete material was made to be a traditional
- yorwba 9y ago
  You're starting the text like a story (in the first person even!). Wikipedia articles aren't usually written like that, so the network is in unfamiliar territory. Its only option is to quickly revert back to something closer to the style of an informative article.
bijection 9y ago
What is it generating for you? (and what OS/browser are you using?)

stmw 9y ago

This is actually pretty neat, although requires more digging

cd0x23 9y ago

I like to lick [my gun, huh, I got some things since I couldn't see.] - Tupac Shakur

antimatter15 9y ago

Hey HN! This is our latest demo for TensorFire (https://tenso.rs/ https://tenso.rs/), a Javascript library for running neural networks in the browser GPU-accelerated by WebGL. Each of these models is a simple two-layer char-rnn (the kind explored in Karpathy's "Unreasonable Effectiveness of RNNs" post http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/). When you hit "Tab" the editor feeds in the contents of your document letter-by-letter into a network which is trying to predict the most likely next letter. It samples from that probability distribution and feeds it back into the network to generate an arbitrarily long string. We've published code and instructions for training new models on Github (https://github.com/tensorfire/cyborg-writer/tree/master/training https://github.com/tensorfire/cyborg-writer/tree/master/trai...). With about a megabyte of any sort of text, you can train a model and easily embed it into any web application. We (@bijection and @antimatter15) will be answering questions in the comments.

projectant 9y ago
No offence, but with a megabyte of text per speaker I feel I could do better with word n-gram Markov models (even bigrams), and random path choice. I think deep learning is great and all ( and I'm meaning to learn it ) but shouldn't it be able to do far better than Markov models, or other simple things? Image captioning? Incredible. Deep Mind winning video games? Incredible. Style transfer? Incredible. With one exception ( I saw on HN ages ago, sorry I have no link -- it basically generates novel text using deep learning, across all sorts of genres, such as "academic paper", "math paper", "novel", "film script" and I found the results remarkable and interesting ) I question if many text applications are doing better than Markov. I think the issue is there is something fundamental and sophisticated about human language which our current deep learning models, with all their omniscient benevolence ( or whatever ), are missing. There's something deep about the structure of language that we are not modelling yet in deep learning as far as I've seen. When we do .... boom ... computers that learn from the internet and amaze us all. Then we'll have something to shine, smile about or fear. Sorry for the digression and what may be inapplicable comparisons. I can get impassioned about this topic.
- 9y ago

_grep_ 9y ago

Hitting tab crashes Chrome for me.

bijection 9y ago
Thanks for the tip! What OS and chrome version are you using?
- snerbles 9y ago
  Not OP, same issue here. Whole browser just locks up, pretty impressive. Windows 7, Version 61.0.3163.100 (Official Build) (64-bit)
  - _grep_ 9y ago
    Same specs here.
  - mipmap04 9y ago
    Reporting the same issue, same build.

nsomaru 9y ago

Fun with Shakespeare: It seemed, somehow mock'd; Unbashed as it was my father. My lady shall have him, perchance and reasons will abide it. Upon his unspoken thing, the devil shall see. MISTRESS QUICKLY: Hide amongst you the gods? Wherefore comes here, sir?

westoncb 9y ago

Pretty hilarious. I've found Tupac to work the best, e.g. (my starting text in quotes): "I was reading Hacker News when" he gladed out Somebody excuse I went down today I wonder if I give a fuck, when I creep For some reason Donald Trump always seems to produce garbage...

sajattack 9y ago
"For some reason Donald Trump always seems to produce garbage..." I think that means it's working correctly.

jaster 9y ago

Ha very nice! I made a vary similar (but much rougher around the edges) project for fun a few months ago using a smaller french corpus of press releases from La Quadrature du Net (a non-profit lobbying for net neutrality in France and Europe). There is a live demo at http://jorquera.net/quadra_neural/ http://jorquera.net/quadra_neural/. You can press tab to auto-complete word-by-word, or just start typing and the suggestions update on the fly. Instead of "Weirdness" I called my cursor "Drunkenness" :) I used keras-js (https://github.com/transcranial/keras-js https://github.com/transcranial/keras-js) to run my model in the browser. If you are interested in this kind of experiments, do go for it! It's quite accessible, and projects of this scale do not require heavy hardware (I generated my models using a GTX 770). If you need a kickstart, in addition to the repo from the OP all the code for my project is Free Software (https://github.com/tomjorquera/quadra_neural https://github.com/tomjorquera/quadra_neural), and I tried to document the generation process in a jupyter notepad https://github.com/tomjorquera/quadra_neural/blob/master/generation/model_gen.ipynb https://github.com/tomjorquera/quadra_neural/blob/master/gen.... While I did it on a french corpus, it should work with any language. I used two main resources for inspiration : - http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py https://github.com/fchollet/keras/blob/master/examples/lstm_...

dchuk 9y ago

We have to be pretty close to someone having a system to generate mountains of convincing/unique "content" using Neural Nets (or something comparable) for the purposes of web spam/seo right? Back in the day, SEOs would use spun content to generate lots of unique variants. But at this point, surely a ML algorithm can crack this nut?

PeterisP 9y ago
Sure, at this point building RNNs that generate unique text on a particular topic isn't a big thing to do, it's something that can be given as a home exercise for undergrads.

c3534l 9y ago

Eh, it's scarcely better than a Markov chain. I was hoping for something a little more useful, like the ability to scroll through many similar words or concepts.

gwern 9y ago
You might like this, then: a word2vec tool which suggests words of the same rough meaning starting with a particular letter or spelling, allowing you to write Genesis solely in words starting with 'a' https://llamasandmystegosaurus.blogspot.com/2017/05/alpha.html https://llamasandmystegosaurus.blogspot.com/2017/05/alpha.ht... or generate large numbers of rhyming definitions (see comments), some examples of which: Cat: purr fur Dolphin: bark shark Puppy: yelp welp Shetland pony: shag nag Mosquito: mighty bitey Shrimp: chill krill

novium 9y ago

It's pretty neat, even got a t.co URL. Sadly, it didn't lead anywhere.

sterex 9y ago

Is there a technical reason why it generates text with lot of spelling mistakes? Ayn Rand seems very sloppy with her spelling!