4 ms·

As with most of the API services for text analytics, sending out confidential data to third parties is simply not acceptable. I run into this almost daily where

by emidln 14y ago

As with most of the API services for text analytics, sending out confidential data to third parties is simply not acceptable. I run into this almost daily where I work. Contractually, we are neither allowed to send our clients' data to a fly-by-night startup nor a an established 800 pound gorilla. The other part is that often in text analytics, the training corpus matters significantly (also this seems less so with summaries at first glance).

drakaal 14y ago
Then you license it for onsite. The training corpus doesn't matter one bit. The type of document does. (Report, op-ed, News, thesis, Fiction, etc) We even do summary based on query. Summarize a 1000 page report only looking at things related to surgeries, or IT. We have hundreds of traits you can mix and match. Want to read the Snows of Kilimanjaro with only the parts that are directly about winter, or foreshadow winter themes? We can do that. Lord of the Rings only when encountering evil. We can do that.
- emidln 14y ago
  Licensing it runs into another hurdle. How much value does random feature give you? Is the feature valuable enough to justify custom licensing costs? Would 100 lines of Python/NLTK give you enough value for your target market? I'm researching this right now for theme clustering. There are some products available for purchase/license that provide reasonable results. The question is if the difference between these reasonable results and 100 lines of Python/Clojure sitting ontop of Storm is enough to justify the cost of the license. That question relates to how much value our 100 line knockoff get our clients vs the extras that a full-feature solution would bring. In the few times I've done the comparison before, it has not been worth buying. Edit: I want to make it clear I'm not saying build it yourself 100% or that building it yourself will give you a reasonable facsimile in all (maybe any instances) with NLTK/OpenNLP/etc. If summarizing is a core feature of your product, by all means, license away. What I'm specifically referencing are situations where maybe I'd like to add a feature, but it's highly unlikely that the single feature along is going to drive a sale. In these circumstances, build it yourself with NLTK or the like is extremely attractive.
  - drakaal 14y ago
    The better question is if your clients care so much about privacy, is the risk that you shorten 3 sentences in such a way that "My client, Mr. Smith would only kill in self-defense. On the other hand Mr. Jones is a cold blooded killer. He has killed dozens of people" Becomes "My client, Mr. Jones is a cold blooded Killer" Because of sentence parsing. Or "Smith would only kill in self-defense. He has killed dozens of people." Because you used keyword density.

by emidln 14y ago

drakaal 14y ago

Then you license it for onsite. The training corpus doesn't matter one bit. The type of document does. (Report, op-ed, News, thesis, Fiction, etc) We even do summary based on query. Summarize a 1000 page report only looking at things related to surgeries, or IT. We have hundreds of traits you can mix and match. Want to read the Snows of Kilimanjaro with only the parts that are directly about winter, or foreshadow winter themes? We can do that. Lord of the Rings only when encountering evil. We can do that.

emidln 14y ago
Licensing it runs into another hurdle. How much value does random feature give you? Is the feature valuable enough to justify custom licensing costs? Would 100 lines of Python/NLTK give you enough value for your target market? I'm researching this right now for theme clustering. There are some products available for purchase/license that provide reasonable results. The question is if the difference between these reasonable results and 100 lines of Python/Clojure sitting ontop of Storm is enough to justify the cost of the license. That question relates to how much value our 100 line knockoff get our clients vs the extras that a full-feature solution would bring. In the few times I've done the comparison before, it has not been worth buying. Edit: I want to make it clear I'm not saying build it yourself 100% or that building it yourself will give you a reasonable facsimile in all (maybe any instances) with NLTK/OpenNLP/etc. If summarizing is a core feature of your product, by all means, license away. What I'm specifically referencing are situations where maybe I'd like to add a feature, but it's highly unlikely that the single feature along is going to drive a sale. In these circumstances, build it yourself with NLTK or the like is extremely attractive.
- drakaal 14y ago
  The better question is if your clients care so much about privacy, is the risk that you shorten 3 sentences in such a way that "My client, Mr. Smith would only kill in self-defense. On the other hand Mr. Jones is a cold blooded killer. He has killed dozens of people" Becomes "My client, Mr. Jones is a cold blooded Killer" Because of sentence parsing. Or "Smith would only kill in self-defense. He has killed dozens of people." Because you used keyword density.