FLAC Metadata loop tags no longer being observed?

Forum rules
Please don't bump threads here if you have a problem - it will often be forgotten about if you do. Instead, make a new thread here.

Post a reply

Smilies
:D :) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :wink: :geek: :ugeek: :!: :?: :idea: :arrow: :| :mrgreen: :3: :wub: >:( :blergh:
View more smilies

BBCode is OFF
Smilies are ON

Topic review
   

Expand view Topic review: FLAC Metadata loop tags no longer being observed?

Re: FLAC Metadata loop tags no longer being observed?

by Chris » Sun Feb 11, 2018 5:14 pm

Ah. Made a new fresh fork, then. Here's the PR.

Re: FLAC Metadata loop tags no longer being observed?

by Graf Zahl » Sun Feb 11, 2018 8:25 am

Your repo has rheit/zdoom as its parent, but I had that connection severed for maintenance reasons so that GZDoom is not handled as a fork anymore. So your repo is not being considered to be related with mine and you cannot do pull requests. You will have to create a new fork and move over to that.

Re: FLAC Metadata loop tags no longer being observed?

by Chris » Sun Feb 11, 2018 8:03 am

I've made a fix that seems to correctly parse FLAC and Ogg Vorbis files for the comment metadata. I'm having trouble making an actual pull request though; I can't seem to specify Graf's repo when making a PR from mine (I know I could before), and I can't seem to specify mine from Graf's. I'm freshly updated and rebased with it, so I don't know what's wrong.

Re: FLAC Metadata loop tags no longer being observed?

by Graf Zahl » Sat Feb 10, 2018 3:49 am

Chris wrote: (RIFF-)WAVE is horrible. It's limited to 4GB files, and the structure can get pretty nuts with lists. It also doesn't play nice with streaming (in the internet/icecast sense) and is difficult to parse efficiently. Any standardized documentation there is for the FOURCC chunk labels (and the format of the data they contain) is not easy to come by, leaving many questions about what to expect or use for various functionality.

At least it DOES have marked chunks. No such luck with Ogg and Flac. Both formats have 'use our standard library to read it so you don't need a good specification' written all over it. Too bad if you only need that to extract a tiny piece of data and it all becomes very unwieldy.
With WAV you could embed the metadata in a separate chunk - let's call it META - and have a well defined way to deal with it. (Also, who does actually need 4GB uncompressed sound files... (that'd be roughly 288 days of CD quality recordings.) :P)

Re: FLAC Metadata loop tags no longer being observed?

by Chris » Sat Feb 10, 2018 3:18 am

Turns out, the reason libFLAC wasn't recognizing vorbis comment metadata was "my" fault. Really, the fault of whoever decided the metadata callback function would only be invoked on StreamInfo metadata by default, and you need to make extra calls to explicitly enable other metadata types to also invoke the callback. So it doesn't appear there's anything wrong with the file, just a wonky libFLAC API.
Gez wrote:Would this and that help? I remember writing this based on xyph.org documentation.
That looks fairly adequate. It's rather unfortunate that libsndfile doesn't seem to have a way to pass through these comments; it does actually read and parse them, but it only lets you get specific ones like "TITLE", "COPYRIGHT", etc, and not custom ones like "LOOP_START" and "LOOP_END".
Graf Zahl wrote:I still wonder why people invented such insane container formats. Even ancient file types like WAV, AIFF and MID got it right, why not these supposedly 'better' formats? :?
(RIFF-)WAVE is horrible. It's limited to 4GB files, and the structure can get pretty nuts with lists. It also doesn't play nice with streaming (in the internet/icecast sense) and is difficult to parse efficiently. Any standardized documentation there is for the FOURCC chunk labels (and the format of the data they contain) is not easy to come by, leaving many questions about what to expect or use for various functionality.

Re: FLAC Metadata loop tags no longer being observed?

by Graf Zahl » Sat Feb 10, 2018 1:42 am

The code in GZDoom was written solely with some Oggs as reference as I simply had neither any usable documentation for the file format, let alone for how it embeds the tag list.
If in FLACs the list can be further inside the file it needs to have a larger threshold for scanning, unless I find some explanation how to do it right.

I still wonder why people invented such insane container formats. Even ancient file types like WAV, AIFF and MID got it right, why not these supposedly 'better' formats? :?

Re: FLAC Metadata loop tags no longer being observed?

by Gez » Fri Feb 09, 2018 8:51 pm

Chris wrote:Well, the reason GZDoom isn't picking it up is because it does a raw search for a '=' character in the first 256 bytes of a file to find the first vorbis comment, then works backwards to find the metadata block header, then goes through the tags from there. However, the first = (0x3D) character appears in a part of the file that's not the comment metadata block, meaning it fails to parse it and fails to find the vorbis comments there. The next 0x3D byte, which is part of a vorbis comment, doesn't appear until the 299th byte, outside of the search range.

I'd need to get a better understanding of the FLAC bitstream format, and how these tags are inserted, to figure out how to correctly fix this. Given how the official libFLAC doesn't recognize it (it doesn't even let on that such metadata comments exist), the fact that anything else does makes me feel like this is a non-standard extension, which means correctly handling it will need further investigation. This whole current approach is a big kludge; it's simply trying to recognize a list structure in some arbitrarily-sized beginning chunk, only verifying that it hasn't got anything egregiously bad, and which isn't guaranteed to hold true even in a properly formatted file.
Would this and that help? I remember writing this based on xyph.org documentation.

Re: FLAC Metadata loop tags no longer being observed?

by wildweasel » Fri Feb 09, 2018 8:13 pm

MusicallyInspired wrote:Strange. Even Winamp and VLC both recognize it as metadata. Is that because they probably use FMod? Why was FMod removed anyway? (I'm sure it was necessary or favourable, I'm just curious)
FMod kept updating in ways that were not compatible with previous implementations of the API, removing the download links to older versions, and the license was one of many things holding GZDoom back from going fully, properly open-source.

Re: FLAC Metadata loop tags no longer being observed?

by MusicallyInspired » Fri Feb 09, 2018 7:32 pm

Strange. Even Winamp and VLC both recognize it as metadata. Is that because they probably use FMod? Why was FMod removed anyway? (I'm sure it was necessary or favourable, I'm just curious)

Re: FLAC Metadata loop tags no longer being observed?

by Chris » Fri Feb 09, 2018 5:51 pm

Well, the reason GZDoom isn't picking it up is because it does a raw search for a '=' character in the first 256 bytes of a file to find the first vorbis comment, then works backwards to find the metadata block header, then goes through the tags from there. However, the first = (0x3D) character appears in a part of the file that's not the comment metadata block, meaning it fails to parse it and fails to find the vorbis comments there. The next 0x3D byte, which is part of a vorbis comment, doesn't appear until the 299th byte, outside of the search range.

I'd need to get a better understanding of the FLAC bitstream format, and how these tags are inserted, to figure out how to correctly fix this. Given how the official libFLAC doesn't recognize it (it doesn't even let on that such metadata comments exist), the fact that anything else does makes me feel like this is a non-standard extension, which means correctly handling it will need further investigation. This whole current approach is a big kludge; it's simply trying to recognize a list structure in some arbitrarily-sized beginning chunk, only verifying that it hasn't got anything egregiously bad, and which isn't guaranteed to hold true even in a properly formatted file.

Re: FLAC Metadata loop tags no longer being observed?

by MusicallyInspired » Fri Feb 09, 2018 6:59 am

I used a command line tool simply called Tag for batch processing. I used it for all my music packs. I also used it for the OGG packs, which work fine. Those FLAC files work fine with EDuke32, Duke3D Megaton Edition, Chocolate Doom, ECWolf (for Super Noah's Ark 3D), and, again, earlier versions of GZDoom/ZDoom.

Re: FLAC Metadata loop tags no longer being observed?

by Gez » Thu Feb 08, 2018 5:23 pm

Looks fine when seen from SLADE.

Image

Re: FLAC Metadata loop tags no longer being observed?

by Chris » Thu Feb 08, 2018 4:19 pm

MusicallyInspired wrote:Here's DE1M1 on its own, though, for a quick reference.
That doesn't seem to have proper metadata tags. At least, libFLAC itself only shows one StreamInfo metadata block which defines the stream format, and no VorbisComment metadata block that I'd expect for custom key/value pairs. I do see the LOOP_START and LOOP_END keys when looking at the file in a hex editor, but they don't seem to be part of a valid FLAC metadata block. How was the loop metadata inserted?

Re: FLAC Metadata loop tags no longer being observed?

by MusicallyInspired » Thu Feb 08, 2018 7:18 am

You can download the full pack at my website at http://sc55.duke4.net/games.php#doom. Here's DE1M1 on its own, though, for a quick reference.

Re: FLAC Metadata loop tags no longer being observed?

by Graf Zahl » Thu Feb 08, 2018 1:18 am

I need the FLAC file. The documentation for the tag format is rather poor and the detection may not be able to handle all cases if something unexpected is present in the tag data. The older versions used FMod to read the tags but that's no longer present.

Top