4 ms·

Since so long, I have been waiting for Indian universities especially IITs to invest and publish in building such corpora. Being a founder of AI/ML startup, I a

Since so long, I have been waiting for Indian universities especially IITs to invest and publish in building such corpora. Being a founder of AI/ML startup, I am surprised at the appalling lack of datasets available to work on Indian problems. Contrast this with Chinese universities where they have built some world class datasets to build NLP solutions in Mandarin. Our sentiment analysis works in 8 different languages but none of it is in Indian languages despite we being in India!

ravivooda 9y ago
I guess it’s not the most demanding requirement at the moment. But happy to see progress. Btw, where do you work? Can we see your work?
woodson 9y ago
The data set is released as CC-BY-NC and, thus, cannot be used in commercial applications.
- rspeer 9y ago
  And also cannot be used in open-source projects. CC-BY-NC amounts to saying: you can play around with this in demos and academic projects that no lawyer would ever go after anyway, but you can't use it for real.
  - dylz 9y ago
    Is that entirely a bad thing? If a commercial, for profit wants to enter this field then they can pay for it or licence it?
    - rspeer 9y ago
      And what if an open source project wants to enter this field?
      - snugghash 9y ago
        For open source commercial, it's the same as for-profit. They'll probably get a discount. For open source non-profit, it'd still be NC and therefore legal?
        rspeer 9y ago
        "Non-profit" is a whole different kettle of fish than "non-commercial". It doesn't mean you're not selling anything. It doesn't mean you're morally good. It just means you registered for a particular business status with particular restrictions. And it's not what CC is talking about. A key part of the Open Source Definition is that you do not lose your permission to use and copy the code based on what you do with it. A project that you aren't allowed to use anymore if you start making money from it is not Open Source. "Non-commercial" code restrictions are more like the thing where you're allowed to look at the Windows source code, if you're an academic and you ask nicely and you won't ever do anything with it.
- muktabh 9y ago
  A lot of universities are open to give a separate commercial license when contacted. They charge for their efforts, which is fair. Whether public universities should charge given we already pay them from our taxes is a different issue. source: I am also cofounder at an AI startup, we often buy licenses to use academic datasets for commercial usage.

ravivooda 9y ago

I guess it’s not the most demanding requirement at the moment. But happy to see progress. Btw, where do you work? Can we see your work?

woodson 9y ago

The data set is released as CC-BY-NC and, thus, cannot be used in commercial applications.

rspeer 9y ago
And also cannot be used in open-source projects. CC-BY-NC amounts to saying: you can play around with this in demos and academic projects that no lawyer would ever go after anyway, but you can't use it for real.
- dylz 9y ago
  Is that entirely a bad thing? If a commercial, for profit wants to enter this field then they can pay for it or licence it?
  - rspeer 9y ago
    And what if an open source project wants to enter this field?
    - snugghash 9y ago
      For open source commercial, it's the same as for-profit. They'll probably get a discount. For open source non-profit, it'd still be NC and therefore legal?
      - rspeer 9y ago
        "Non-profit" is a whole different kettle of fish than "non-commercial". It doesn't mean you're not selling anything. It doesn't mean you're morally good. It just means you registered for a particular business status with particular restrictions. And it's not what CC is talking about. A key part of the Open Source Definition is that you do not lose your permission to use and copy the code based on what you do with it. A project that you aren't allowed to use anymore if you start making money from it is not Open Source. "Non-commercial" code restrictions are more like the thing where you're allowed to look at the Windows source code, if you're an academic and you ask nicely and you won't ever do anything with it.
muktabh 9y ago
A lot of universities are open to give a separate commercial license when contacted. They charge for their efforts, which is fair. Whether public universities should charge given we already pay them from our taxes is a different issue. source: I am also cofounder at an AI startup, we often buy licenses to use academic datasets for commercial usage.