3 ms·

Tone: Cross platform audio tagger and metadata editor

82 points by ingve 3y ago

leetrout 3y ago
Ooo I will try this on my next podcast episode edit. I ended up using mp3tag on windows and didnt see anything cross platform for free in a quick search last year
- majkinetor 3y ago
  Picard.
AlphaWeaver 3y ago
I've been looking for a tool that can passively transcribe audio files to make them easier to search - this looks like it could almost solve that use case - maybe with a scripted tagger.
- calciphus 3y ago
  Check out Whisper, it is surprisingly good and several of the output formats include time codes, works with multiple languages. https://github.com/openai/whisper https://github.com/openai/whisper
  - yorwba 3y ago
    Whisper is now outclassed by Facebook's MMS https://github.com/facebookresearch/fairseq/tree/main/examples/mms https://github.com/facebookresearch/fairseq/tree/main/exampl... though the integration work to make it a turnkey solution for a wider audience hasn't happened yet. (E.g. https://github.com/ggerganov/whisper.cpp/issues/950 https://github.com/ggerganov/whisper.cpp/issues/950 is still open.)
    - angrais 3y ago
      Have you tested MMS with real-world data? It's perhaps outclassed on evaluation metrics but on real-world data it is not as good as whisper.
      - ck_one 3y ago
        Have you tried MMS on real world data or is it just assumption?
        angrais 3y ago
        Yes, of course. Real world data being: one on one interviews (no background noise), small groups of people chatting (lots of background noise), and specific audio recordings ( with varying British regional accents. In all three instances whisper produced a more accurate transcription. This is for personal use. The license of MMS is also restrictive so cannot he used for commercial uses while whisper can. Another key consideration when wondering what to choose. On the other hand, one can train MMS (so using own custom dataset) so for some projects it may be more suitable.
- sandreas 3y ago
  That is funny. For audio books I'm currently working on an `epub` command for `tone` which will be able to extract text from `epub` files, e.g.: tone epub --format="markdown" --extract-sentences --one-file-per-chapter output-path/ As a result, you can use https://github.com/readbeyond/aeneas https://github.com/readbeyond/aeneas with the generated text / markdown files to create a json mapping file looking like this: { "fragments": [ { "begin": "0.000", "children": [], "end": "7.920", "id": "f000001", "language": "eng", "lines": [ "This is the first sentence of the audio book." ] } } Since aeneas is a bit inaccurate, I'm also working on an improvement with silence detection for these mapping files. If you are looking for something that is "ready to use", you could check out https://github.com/r4victor/syncabook https://github.com/r4victor/syncabook or the according library https://github.com/r4victor/afaligner https://github.com/r4victor/afaligner If you have audio files, that are NOT audio books, the epub approach will not help you and the other comments are more helpful.
- thangalin 3y ago
  See my comment showing how to transcribe using Whisper: https://news.ycombinator.com/item?id=35366778 https://news.ycombinator.com/item?id=35366778
k2enemy 3y ago
Apologies for a slightly off-topic question, but people discussing tone are probably the ones with an answer! I have a series of "mixtape" mp3 files and I'd like to generate a list of the songs within the one long file. I'm aware of musicbrainz, but is there anything that works on a single file with multiple songs? Perhaps my best bet is to generate 10 second long files sampled every two minutes, then run the fingerprinting on those individual files?
- sandreas 3y ago
  You could try to perform a silence detection, if there is a silence after every song. https://stackoverflow.com/questions/42507879/how-to-detect-the-silence-at-the-end-of-an-audio-file https://stackoverflow.com/questions/42507879/how-to-detect-t... ffmpeg -i audio.mp3 -af silencedetect=n=-50dB:d=0.5 -f null - 2>&1 # -50dB is the threshold for detecting audio as a silence # 0.5 means the silences must at least have a duration of 0.5 seconds In your case I would try: ffmpeg -i audio.mp3 -af silencedetect=n=-30dB:d=2.5 -f null - 2>&1 That should give you a pretty accurate list of where the songs begin and end... that way, you can extract the songs and run fingerprinting. After this, you could use the ability of `tone` to add `chapters` to audio files (for mp3 it is id3v2 chapter addendum, see https://mutagen-specs.readthedocs.io/en/latest/id3/id3v2-chapters-1.0.html https://mutagen-specs.readthedocs.io/en/latest/id3/id3v2-cha...) to have at least marks for each track as title. MP3 chapters will not be recognized by most players, but at least it is within the specs.
  - thedaly 3y ago
    Typically a mixtape is a continuous mix with no song breaks. If that is the case, track lists are using listed where the mix is posted, mixcloud, etc.
- majkinetor 3y ago
  That may be impossible because in the mix, songs may be played with different speed so no detection app that I know will find them. Otherwise, you could play a mixtape and put shazam in the background to see if it will find the songs (it can do that continiously). If it can, then you can easily automate file splitting using ffmpeg and powershell and grabbing a list using GUI/CLI automation via shazam desktop or somethig like https://audd.io https://audd.io.
slmjkdbtl 3y ago
If you want to do this with ffmpeg: https://gist.github.com/eyecatchup/0757b3d8b989fe433979db2ea7d95a01 https://gist.github.com/eyecatchup/0757b3d8b989fe433979db2ea...
- sandreas 3y ago
  `ffmpeg` is great, but it needs improvements for some tags / formats (that is why I developed tone in the first place). Examples: - Duration detection is inaccurate - Tags `MovementName` and `MovementIndex` (for series) are not supported for some formats - etc. Another disadvantage is, that (afaik) `ffmpeg` needs to reprocess the file (audio wise) to write tags instead of just changing the metadata inplace.
sandreas 3y ago
Author here, thanks for mentioning my project on hackernews. Let me know if I can help you with anything or answer questions :-)
lynx23 3y ago
Bless you lltag, you are all I ever needed.

82 points by ingve 3y ago

Ooo I will try this on my next podcast episode edit. I ended up using mp3tag on windows and didnt see anything cross platform for free in a quick search last year

majkinetor 3y ago
Picard.

AlphaWeaver 3y ago

I've been looking for a tool that can passively transcribe audio files to make them easier to search - this looks like it could almost solve that use case - maybe with a scripted tagger.

calciphus 3y ago
Check out Whisper, it is surprisingly good and several of the output formats include time codes, works with multiple languages. https://github.com/openai/whisper https://github.com/openai/whisper
- yorwba 3y ago
  Whisper is now outclassed by Facebook's MMS https://github.com/facebookresearch/fairseq/tree/main/examples/mms https://github.com/facebookresearch/fairseq/tree/main/exampl... though the integration work to make it a turnkey solution for a wider audience hasn't happened yet. (E.g. https://github.com/ggerganov/whisper.cpp/issues/950 https://github.com/ggerganov/whisper.cpp/issues/950 is still open.)
  - angrais 3y ago
    Have you tested MMS with real-world data? It's perhaps outclassed on evaluation metrics but on real-world data it is not as good as whisper.
    - ck_one 3y ago
      Have you tried MMS on real world data or is it just assumption?
      - angrais 3y ago
        Yes, of course. Real world data being: one on one interviews (no background noise), small groups of people chatting (lots of background noise), and specific audio recordings ( with varying British regional accents. In all three instances whisper produced a more accurate transcription. This is for personal use. The license of MMS is also restrictive so cannot he used for commercial uses while whisper can. Another key consideration when wondering what to choose. On the other hand, one can train MMS (so using own custom dataset) so for some projects it may be more suitable.
sandreas 3y ago
That is funny. For audio books I'm currently working on an `epub` command for `tone` which will be able to extract text from `epub` files, e.g.: tone epub --format="markdown" --extract-sentences --one-file-per-chapter output-path/ As a result, you can use https://github.com/readbeyond/aeneas https://github.com/readbeyond/aeneas with the generated text / markdown files to create a json mapping file looking like this: { "fragments": [ { "begin": "0.000", "children": [], "end": "7.920", "id": "f000001", "language": "eng", "lines": [ "This is the first sentence of the audio book." ] } } Since aeneas is a bit inaccurate, I'm also working on an improvement with silence detection for these mapping files. If you are looking for something that is "ready to use", you could check out https://github.com/r4victor/syncabook https://github.com/r4victor/syncabook or the according library https://github.com/r4victor/afaligner https://github.com/r4victor/afaligner If you have audio files, that are NOT audio books, the epub approach will not help you and the other comments are more helpful.
thangalin 3y ago
See my comment showing how to transcribe using Whisper: https://news.ycombinator.com/item?id=35366778 https://news.ycombinator.com/item?id=35366778

k2enemy 3y ago

Apologies for a slightly off-topic question, but people discussing tone are probably the ones with an answer! I have a series of "mixtape" mp3 files and I'd like to generate a list of the songs within the one long file. I'm aware of musicbrainz, but is there anything that works on a single file with multiple songs? Perhaps my best bet is to generate 10 second long files sampled every two minutes, then run the fingerprinting on those individual files?

sandreas 3y ago
You could try to perform a silence detection, if there is a silence after every song. https://stackoverflow.com/questions/42507879/how-to-detect-the-silence-at-the-end-of-an-audio-file https://stackoverflow.com/questions/42507879/how-to-detect-t... ffmpeg -i audio.mp3 -af silencedetect=n=-50dB:d=0.5 -f null - 2>&1 # -50dB is the threshold for detecting audio as a silence # 0.5 means the silences must at least have a duration of 0.5 seconds In your case I would try: ffmpeg -i audio.mp3 -af silencedetect=n=-30dB:d=2.5 -f null - 2>&1 That should give you a pretty accurate list of where the songs begin and end... that way, you can extract the songs and run fingerprinting. After this, you could use the ability of `tone` to add `chapters` to audio files (for mp3 it is id3v2 chapter addendum, see https://mutagen-specs.readthedocs.io/en/latest/id3/id3v2-chapters-1.0.html https://mutagen-specs.readthedocs.io/en/latest/id3/id3v2-cha...) to have at least marks for each track as title. MP3 chapters will not be recognized by most players, but at least it is within the specs.
- thedaly 3y ago
  Typically a mixtape is a continuous mix with no song breaks. If that is the case, track lists are using listed where the mix is posted, mixcloud, etc.
majkinetor 3y ago
That may be impossible because in the mix, songs may be played with different speed so no detection app that I know will find them. Otherwise, you could play a mixtape and put shazam in the background to see if it will find the songs (it can do that continiously). If it can, then you can easily automate file splitting using ffmpeg and powershell and grabbing a list using GUI/CLI automation via shazam desktop or somethig like https://audd.io https://audd.io.

slmjkdbtl 3y ago

If you want to do this with ffmpeg: https://gist.github.com/eyecatchup/0757b3d8b989fe433979db2ea7d95a01 https://gist.github.com/eyecatchup/0757b3d8b989fe433979db2ea...

sandreas 3y ago
`ffmpeg` is great, but it needs improvements for some tags / formats (that is why I developed tone in the first place). Examples: - Duration detection is inaccurate - Tags `MovementName` and `MovementIndex` (for series) are not supported for some formats - etc. Another disadvantage is, that (afaik) `ffmpeg` needs to reprocess the file (audio wise) to write tags instead of just changing the metadata inplace.

sandreas 3y ago

Author here, thanks for mentioning my project on hackernews. Let me know if I can help you with anything or answer questions :-)

lynx23 3y ago

Bless you lltag, you are all I ever needed.