3 ms·

Looks like they offer a full data dump: https://dl.opensubtitles.org/addons/export/ https://dl.opensubtitles.org/addons/export/ I haven't actually tried to use

Looks like they offer a full data dump: https://dl.opensubtitles.org/addons/export/ https://dl.opensubtitles.org/addons/export/ I haven't actually tried to use it. So long as they offer a dump you could use I think it's entirely reasonable to require payment for the API. I think it's reasonable to consider them morally open even if the web UI is closed source if the subtitles are open. It feels to me like a project selling a shiny hosted app around an open source project. I think in this sort of project the core "source" is the subtitles, not the web UI. If OpenAI let you download their models but didn't provide the tooling you'd need to run them I think their name would be much more reasonable.

red_trumpet 3y ago
That's not a full dump of subtitles, but rather of list of which subtitles they have, with download links. Here are the first lines of the `subtitles_all.txt` file: IDSubtitle MovieName MovieYear LanguageName ISO639 SubAddDate ImdbID SubFormat SubSumCD MovieReleaseName MovieFPS SeriesSeason SeriesEpisode SeriesIMDBParent MovieKind URL 1 Alien3 1992 English en 2004-10-31 23:54:23 103644 sub 2 Alien.3 11.000 0 0 0 movie http://www.opensubtitles.org/subtitles/1/alien3-en 2 Identity 2003 Slovenian sl 2004-10-31 23:54:23 309698 sub 1 Identity.DVDRiP.XViD 0.000 0 0 0 movie http://www.opensubtitles.org/subtitles/2/identity-sl 3 Ghost in the Shell 2: Innocence 2004 English en 2004-10-31 23:54:23 347246 srt 1 Innocence.2004.DVDRip.XviD 23.980 0 0 0 movie http://www.opensubtitles.org/subtitles/3/ghost-in-the-shell-2-innocence-en
- fundatus 3y ago
  Yeah, I was wondering if that could be the full dump. I know how far compression algorithms have come, but 318MB for all the subtitles of the world? Would be interesting to know though how large a full dump would be.
  - jck 3y ago
    My guess is not by much. Text doesn't require a lot of storage. After looking at a few random files my estimate is that the size of the full compressed dump would be about 30x (~10 GB)
  - adt 3y ago
    EleutherAI cleaned an OpenSubtitles dataset that was 12.98GiB (13.9GB). https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027 https://lifearchitect.ai/whats-in-my-ai/ https://lifearchitect.ai/whats-in-my-ai/

red_trumpet 3y ago

That's not a full dump of subtitles, but rather of list of which subtitles they have, with download links. Here are the first lines of the `subtitles_all.txt` file: IDSubtitle MovieName MovieYear LanguageName ISO639 SubAddDate ImdbID SubFormat SubSumCD MovieReleaseName MovieFPS SeriesSeason SeriesEpisode SeriesIMDBParent MovieKind URL 1 Alien3 1992 English en 2004-10-31 23:54:23 103644 sub 2 Alien.3 11.000 0 0 0 movie http://www.opensubtitles.org/subtitles/1/alien3-en 2 Identity 2003 Slovenian sl 2004-10-31 23:54:23 309698 sub 1 Identity.DVDRiP.XViD 0.000 0 0 0 movie http://www.opensubtitles.org/subtitles/2/identity-sl 3 Ghost in the Shell 2: Innocence 2004 English en 2004-10-31 23:54:23 347246 srt 1 Innocence.2004.DVDRip.XviD 23.980 0 0 0 movie http://www.opensubtitles.org/subtitles/3/ghost-in-the-shell-2-innocence-en

fundatus 3y ago
Yeah, I was wondering if that could be the full dump. I know how far compression algorithms have come, but 318MB for all the subtitles of the world? Would be interesting to know though how large a full dump would be.
- jck 3y ago
  My guess is not by much. Text doesn't require a lot of storage. After looking at a few random files my estimate is that the size of the full compressed dump would be about 30x (~10 GB)
- adt 3y ago
  EleutherAI cleaned an OpenSubtitles dataset that was 12.98GiB (13.9GB). https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027 https://lifearchitect.ai/whats-in-my-ai/ https://lifearchitect.ai/whats-in-my-ai/