ALAC Encoder {implemented with a third-party plug-in by thinktink}

[thinktink]

Work In Progress.

CAF format seems to be working fine. I'm still working on the M4A version. There's no config dialog yet.

wip

[MrSinatra]

amazing! testing now... (m4a though i think is the mainstream format, its what everything i've used contains an ALAC in) thx!

(TT, sent a PM)

[thinktink]

Got your PM.

At the moment I'm looking at the file spec for m4a/mp4 and it's giving me a headache.

[MrSinatra]

ok, so the first thing i notice is that it lists CAF and m4a as 0.0.0.0 in the cd ripping and format convertor. i'm assuming thats as expected, as is that the plugin/encoder does not show up in the ctrl-p prefs, right?

i ripped a CD track to CAF. seemed to create a file that looked right, around 12 meg.

winamp won't play it. CAF seems to be under the nullsoft waveform decoder, do i need to adjust?

also, can't you just put the raw ALAC into a m4a container that is already handled by winamp? i admit i don't know how it works, but i thought m4a container handling (incl creating and deciphering) was already implemented by winamp? just trying to save you from having to learn the whole m4a file spec, and instead just figure how to get the raw audio inside what winamp already does, but again i'm likely just ignorant of how it actually works.

http://atomicparsley.sourceforge.net/mpeg-4files.html

[thinktink]

Originally Posted by MrSinatra

ok, so the first thing i notice is that it lists CAF and m4a as 0.0.0.0 in the cd ripping and format convertor. i'm assuming thats as expected, as is that the plugin/encoder does not show up in the ctrl-p prefs, right?

Correct but for the wrong reason. Encoder plugins don't get displayed in the Ctrl+P and I don't know of a Winamp version that ever did. The plugin is purely an encoder, no input plugin. It has it's own version spec. Since the first release is a WIP then the major and minor versions are zero.

Originally Posted by MrSinatra

i ripped a CD track to CAF. seemed to create a file that looked right, around 12 meg.

winamp won't play it. CAF seems to be under the nullsoft waveform decoder, do i need to adjust?

I don't know of any native Winamp plugin that can decode ALAC except in_mp4 through the alac.w5s provider. CAF is a container format, not a codec-specific file type. CAF files can store other formats than just ALAC. Since a number of PCM and other RAW (uncompressed) audio waveform data commonly get stored in the CAF file format, the waveform plugins get the call. But since it's ALAC and not a waveform format the waveform libraries can't access it.

Originally Posted by MrSinatra

also, can't you just put the raw ALAC into a m4a container that is already handled by winamp? i admit i don't know how it works, but i thought m4a container handling (incl creating and deciphering) was already implemented by winamp? just trying to save you from having to learn the whole m4a file spec, and instead just figure how to get the raw audio inside what winamp already does, but again i'm likely just ignorant of how it actually works.

http://atomicparsley.sourceforge.net/mpeg-4files.html

I don't know of any mechanism in the Winamp API for mp4/m4a enveloping. Yes, there are some encoder plugins that already encode and transport with mp4/m4a but it doesn't appear to be external to the plugin. I have to do it myself.

[MrSinatra]

ok, so something interesting, bit of a tangent but...

i took my CAF ALAC and tried to play it with VLC and it WORKED.

this then reminded me of a debate i had with DrO about images (that i think spilled over into audio formats), and my memory on this is fuzzy so i might be recounting it wrong, but winamp seems to fail as a deliberate design choice if things don't match, like if an image is png but has a jpg extension, winamp will show nothing purposely, even though a web browser will display it. i think that choice also extends to audio decoding.

i believe i said i think winamp should, to whatever degree possible, examine headers and so on in the file and try to use/decode the file (image or audio) in spite of whatever mistaken extension it may have. needless to say, i don't think i got very far with that argument.

in any case, it may not even apply to any of this, but just thought i'd mention it.

anyway, i don't know if my link was any use to you, but i am curious where you found links that explain the m4a file spec? i ask b/c i can't find anything that looked useful to me. where is the info showing you how to do this?

[thinktink]

From multiple sources. At the moment, some from "2835389_1302074539.pdf" that I found from a google search and an accumulation of notes from random sites that pop up from searching for specific atoms that I find. However, I keep having to look at an actual M4A file generated with (the real) ffmpeg with my hex editor for clarification and specific values that aren't being shown in the documentation.

[thinktink]

Just got m4a container encoding working.

Just have to do the configuration window and then I'm done.

[thinktink]

Version 1.0.0.0 now available.

Includes language support.

Download from here.




Localization files attached.

[thinktink]

Doesn't seem to be a lot of activity on this. Hope MrSinatra is happy.

[MrSinatra]

just seeing this now, its great. don't have time atm to fully test it but did install it.

one thing, windows put up that dialog saying "this program might not have installed correctly" and i could choose to install again or say it did install ok. i said it was ok, as often windows is wrong, but just a fyi for you.

what is fast mode? does it relate to compression? i like max compression.

i'm very happy btw!

[thinktink]

Originally Posted by MrSinatra

...

what is fast mode? does it relate to compression? i like max compression.

...

I have no idea, I never bothered to check the source code's description. I just saw it as an option (seems like the only option) and so I put it in as it's labeled. I imagine it's a mode that allows it to use less compression so that it will compress faster, but that's just a guess. I actually didn't notice any difference while testing, though I wasn't using a stopwatch or anything.

[Benski]

I can help you with using libmp4v2 to encode the ALAC file to mp4/m4a if you want. just e-mail me

[thinktink]

Originally Posted by Benski

I can help you with using libmp4v2 to encode the ALAC file to mp4/m4a if you want. just e-mail me

Well I'll be damned, I didn't even see libmp4v2.dll. Looking at the exports I imagine it's far better than what I'm doing. Does it support in-memory reading/writting?

[thinktink]

Version 1.0.1.13 now available.

Changes:

• Added support for very large files in the m4a encoder (Automatic switching between individual 32 and 64 bit boxes (atoms) depending on size.)

Download.

[MrSinatra]

Fantastic!

Is this with winamps library, or your own implementation?

Did you get in touch with Benski?

I posted this thread to the fb winamp group.

[thinktink]

Originally Posted by MrSinatra

Fantastic!

Is this with winamps library, or your own implementation?

Did you get in touch with Benski?

I don't actually have a full implementation of the full ISO standard, just a basic hierarchical class handler to help me organize and construct the Moov box piece by piece. Once everything is in place, I call a single function which then triggers callbacks that outputs the data in it's proper order. All I had to do was update the class handler to support 64 bit boxes. I never did contact Benski as my preliminary research into libmp4v2 revealed that it seems to exclusively deal with files already on the drive, which doesn't work very well for a standard Winamp encoder. If memory serves, DrO described one of the encoders that ships with Winamp that just outputs dummy data to Winamp and writing the actual file contents to a temporary location and then swapping the file in once the data is finalized, I'm guessing that encoder uses the libmp4v2 library.

Originally Posted by MrSinatra

I posted this thread to the fb winamp group.

Very nice.

[thinktink]

Version 1.1.2.21 now available.

Changes:

• Code optimizations for the m4a container.
• Memory utilization optimizations for the m4a container and encoder.
• Minor code optimizations for code shared between the caf and m4a containers.

Last night I bench-marked some of the code and found nothing really horrendous with my code, at least relative to the encoding call. The actual encoding function call was the speed bottleneck.

But, while I was at it I figured I'd get something in there to help things along so I optimized as much as I practically could anyways.

I'm not sure you'll actually see a difference in speed, especially with standard length music files. But maybe somebody somewhere will see an improvement.



Download.

[EDIT/]
I also tested with DiskWrite (don't know why I missed to check that) and everything seems fine with it.

[Benski]

Originally Posted by thinktink

I never did contact Benski as my preliminary research into libmp4v2 revealed that it seems to exclusively deal with files already on the drive, which doesn't work very well for a standard Winamp encoder. If memory serves, DrO described one of the encoders that ships with Winamp that just outputs dummy data to Winamp and writing the actual file contents to a temporary location and then swapping the file in once the data is finalized, I'm guessing that encoder uses the libmp4v2 library.

This is a true because of the limitations of the Winamp Encoder API (it was originally designed for real-time SHOUTcast encoding and not really meant for file encoding, hence all the little added functions and quirks along the way). The M4A encoder does indeed send back dummy data while at the same time making a temp file. It's not all that bad, though.

[thinktink]

Originally Posted by Benski

... It's not all that bad, though.

I agree especially since it's going to a file anyway, but since I had already put the time and effort into doing it in-memory I decided not to use it. Had I known about it before I started work on the m4a container I would have been on it like a duck on a June-bug simply to avoid dealing with the headache.

I am forced to update the file on FinishAudio3, and it's not like I'm against post-processing updates as a matter of principle or anything.

[MrSinatra]

so far working as advertised, more testing to come however...

[thinktink]

Glad to hear it's working correctly so far.

[MrSinatra]

question:

does CAF support tags? and if so, will your plugin convert a CAF ALAC to a m4a ALAC with tags intact?

edit:

second question:

how do you get winamp to PLAY ALACs in a CAF container? it seems odd to me that it can't, given that other multi format containers, like m4a, are handled properly.

[thinktink]

Originally Posted by MrSinatra

question:

does CAF support tags? and if so, will your plugin convert a CAF ALAC to a m4a ALAC with tags intact?

edit:

second question:

how do you get winamp to PLAY ALACs in a CAF container? it seems odd to me that it can't, given that other multi format containers, like m4a, are handled properly.

2nd point, same issue: http://forums.winamp.com/showthread....17#post3043817

[MrSinatra]

yeah, i did read that previously, and perhaps i'm just not getting the under the hood technicality, but my point is that winamp can decode successfully completely different codecs in the same container type, like aac or mp3 or ALAC in a m4a, so why can't it do the same type thing with CAF? like wav or ALAC in a CAF?

again, i'm probably just ignorant of the under hood mechanics, but it seems to me if it can for the one, it could (should be able to?) for the other...?

also curious as to my first question... i can't test at the moment but even if you just answered that one in theory it would be useful. thx again!

[thinktink]

Originally Posted by MrSinatra

yeah, i did read that previously, and perhaps i'm just not getting the under the hood technicality, but my point is that winamp can decode successfully completely different codecs in the same container type, like aac or mp3 or ALAC in a m4a, so why can't it do the same type thing with CAF? like wav or ALAC in a CAF?

again, i'm probably just ignorant of the under hood mechanics, but it seems to me if it can for the one, it could (should be able to?) for the other...?

...

AFAIK, the plugin in_mp4.dll uses several WASABI components for decoding different codecs from within the same mp4/m4a container format files which is dynamically mutable.

AFAIK, with the dll that handles CAF files, it seems to always assume that all CAF files are always uncompressed with hard-coded handlers for the various formats supported by the CAF container. To support CAF files with ALAC data in them the dll that handles the CAF files would have to be recoded to recognize the codec and then call a codec handler, which currently does not exist for CAF apparently.

It should be noted that the information I'm providing is completely inferred by me by the dll's that I've found in the Winamp install folder.

That's not something I can change.

Yes, you're asking questions that requires under-the-hood knowledge to understand the answer to.

Originally Posted by MrSinatra

...

also curious as to my first question... i can't test at the moment but even if you just answered that one in theory it would be useful. thx again!

The container format itself supports tagging so if Winamp could handle a CAF encoded ALAC file then tags would automatically work (theoretically.)

[MrSinatra]

Originally Posted by thinktink

Yes, you're asking questions that requires under-the-hood knowledge to understand the answer to.

ok, but i did basically understand the gist of what you were saying, (thx), so maybe Benski will see this post and make a note to adjust things so CAF playback is not hardwired to wav only.

Originally Posted by thinktink

The container format itself supports tagging so if Winamp could handle a CAF encoded ALAC file then tags would automatically work (theoretically.)

ok, cool, and i will assume with that answer that your plugin can convert an ALAC CAF to an ALAC m4a. (meaning basically, a conversion of container format)

thx again.

[MrSinatra]

just wanted to gently suggest that you make the item titles here:

http://bogproghome.host-ed.me/

be links to the related threads here at winamp forums, so folks who find that page can read the documentation.

[vasdrakken]

This might help.

AVI is the container, it has a encoder file stored in the header. The hex code will be either hashed with the stored encoder making it a MP4, MKV, M4P, M4A, AAC, NERO AAC, etc...

There are four parts.
+ One the header file. This is usually documented in a standard dot h file before being complied or encoded. This lists the number of tracks how much data is supposed to be in both and the stops and headers of the tracks below. Ie it means that if it was recorded as analog at 44.1 Hertz basically fourty four thousand sample per second verse forty eight thousand samples per eight of a second. the sampling of 48 has more lead in than 44 because even though is samples every 48 thousandth of a second it does this eight times. Those are then over laid over each other and the best avg is kept. To get a MKV or MP4 instead of AVI you toss the rest of the codecs and only leave the codec info for that encoding. So they are still AVI containers but they save space by not have to store the encoding data for all the formats they did not encode with. Which is why AVI will play on more systems if the codecs are missing because they can use the frame of reference for all the other codec's stored in that AVI container header. So if you are looking at AAC files they can have more than one codec stored in the header like nero would store H.264, H263, H261, and a bunch of variations of VC1, H.264 Part 10. So that older stuff would still be readable by their decoder. But it is still the AVI container file and has to follow the binary and plain text encoding rules. So did up those or read through the Matroska Vision format of the AVI container, it will help explain how the AVI container works. All of these are AVI: AVI, ASF, MOV, RM, MP4, MPG ES, but to protect their work claim they are new and improved containers but can be renamed and run through direct show to decode the avi data. Advance Vector Indices.
+ Two the video track data, I was told years ago when I was working on encoding that the audio tracks follow because the screen has to have something on it even if just a black box. The joke being that black boxes in object oriented coding are functions that the person maintaining the code does not need to understand as long as they take the expected info and return the expected results. The other joke was that light travels faster than sound so it went first. the last rumor I heard was that once the video is compressed if stuff needs to be lost it is better to have video meet a minimum threshold as the sound can be compressed into rougher digital vector data while still sounding the same if it is played and recorded at fast enough hertz. The video data is stored as frames or series of pictures. There is no actual way to store motion data unless you store it like 3D dimensional audio data then you get mocap data and no pictures, just point cloud data. Basically think about like this the images are a number of tracks specified in the header of pictures that the header tells the codec how often to change the picture how the bits can be compressed. Like with jpg the picture uses something like raid data it multiples the images against grey and when it can reproduce within a percentage of the same math that is what is save. Lazco is really cool. Then to get a zip file those are then treated as numbers and stored so that the numbers can be reproduced with no errors. Lossy verse lossless. The first is lossy jpg and the second zip is lossless. Since the first can have errors as long as you can not see them and the second is not allowed any errors.
+ The audio tracks.
AVI is the designed to store bits like a wave file does pre-encoded analog vector data. Think of it as a series of spline curves stored at wave height table and trough distance vector. IE a direction and speed. So the un compressed data is a series of arrows pointing left and from zero pointing directly down to mean a highlight sound or an eight note. If you look at how old school midi files are built you can see this.
Basically think of it as series of arrows with a number behind them. They all point to the right of the screen unless less than three and greater than one seventy five or so. When they point like they are eight notes holding the previous sound or raising or falling note. The notes falling or getting deeper in the trough the notes raising are rasing in pitch the thing is notes stored by the highest point that is set when they are encoded by bit depth. so when you are recording and the notes start getting deeper or higher the bars start scaling the previous recorded notes smaller. The vectors when you hit stop lock the recording to sample based on the highest or lowest note. It is always stored and equal distance from flat which is ninety. Ninety sounds like Alvin and the chipmunks... or really bad speed metal. lol. There is some that impressive for the sheer skill but still sounds like they are wasting their time playing... but there are some that impressive because they are hitting the notes long enough to be heard then the next without skipping any. Those are usually seventy or lower and one hundred and ten or higher.

>>>///>>\\\ etc...
+ The last part is the extra stuff... like file security and copyright data stuff you see in grace note and on the file itself.

MPG or MP1 did not have this it was built when the MP3 file format was created and was just the header and muxed data files. IE it was encoded it tracks or streams.The compressed as one encrypted file. Apples first version of M4A was like this with the last part stored in a separate file and when they brought it over to the PC side of things they had to make a file with all of encrypted streams as one file like a MP1 file. Which meant the only thing that could decrypt it was the apple software as the public hash or public key was in the software and the private key was encoded into the file when you bought it which is why they did not let people re download everything because their servers took a hit when some one bought a lot of music and if someone needed those new keys generated until the machines got fast enough that generating a hash was no big deal. That and people had to use the right version some times to get their music to work if the key structure changed too much.

Hopefully this helps.
Benjamin Solheim.
I had another account on here at some point I used winamp to play my protected files from apples store on my android phone. grin.

[thinktink]

Originally Posted by MrSinatra

just wanted to gently suggest that you make the item titles here:

http://bogproghome.host-ed.me/

be links to the related threads here at winamp forums, so folks who find that page can read the documentation.

At the moment, my hosting provider has suspended my website with the message that my bill is passed due...


...for $0.00

It's a free account so I don't know what gives.

I'm done waiting for them to respond to my support ticket and I'm currently in the process of moving my site to a different host. I don't know if I'll set it up the same way or not.

[MrSinatra]

Originally Posted by thinktink

Version 1.1.2.21 now available.

Changes:

• Code optimizations for the m4a container.
• Memory utilization optimizations for the m4a container and encoder.
• Minor code optimizations for code shared between the caf and m4a containers.

Last night I bench-marked some of the code and found nothing really horrendous with my code, at least relative to the encoding call. The actual encoding function call was the speed bottleneck.

But, while I was at it I figured I'd get something in there to help things along so I optimized as much as I practically could anyways.

I'm not sure you'll actually see a difference in speed, especially with standard length music files. But maybe somebody somewhere will see an improvement.



Download.

[EDIT/]
I also tested with DiskWrite (don't know why I missed to check that) and everything seems fine with it.

i just wanted to thank u again.

is this available anywhere? via dropbox perhaps?

[cacafuti8888]

Hi, i've got this file in my computer i've downloaded from here months ago, i hope this is what you're looking for.

https://www.dropbox.com/s/3kvx1w1lwb...Setup.zip?dl=0

[MrSinatra]

hi, thx, yes i already had it, but i just hoped he had it back up in case i install elsewhere remotely, so that helps, thx.

[thinktink]

Attached.

[MrSinatra]

odd Q for you...

i noticed an issue where windows explorer, for any m4a, could have data divergent from "normal" atom tags and i couldn't understand it. the answer seems to be that windows uses "normal" atoms, but ALSO creates a custom moov.udta.Xtra atom and sticks a data blob in there, with what i believe is a WMA style WM tag; odd & bizarre, but NOT out of m4a spec. among the fields win exp uses in this blob, is one for ratings, that win exp can see:

https://community.mp3tag.de/t/no-sup...-m4a-mp4/15671

http://www.mediamonkey.com/forum/vie...hp?f=1&t=76321

http://forums.winamp.com/showthread.php?t=376798

this i think is relevant to you (& me!) if say you have a rated FLAC, and you want to transcode it to ALAC m4a, b/c you want the tag info to also go with the transcode.

Media Monkey solved the issue years ago, (i reported it), by basically updating/saving both sets of tags simultaneously. yes, its duplicative, but it keeps info from being divergent and it also makes the info exposed in windows explorer:

http://www.ventismedia.com/mantis/view.php?id=12017

any chance you could fool around with this and get it to work with your plugin?

also, i think the media monkey solution needs to be adopted by winamp as well, esp for the ML, so that ratings and other tags don't become divergent.

[thinktink]

Originally Posted by MrSinatra

odd Q for you...

i noticed an issue where windows explorer, for any m4a, could have data divergent from "normal" atom tags and i couldn't understand it. the answer seems to be that windows uses "normal" atoms, but ALSO creates a custom moov.udta.Xtra atom and sticks a data blob in there, with what i believe is a WMA style WM tag; odd & bizarre, but NOT out of m4a spec. among the fields win exp uses in this blob, is one for ratings, that win exp can see:

https://community.mp3tag.de/t/no-sup...-m4a-mp4/15671

http://www.mediamonkey.com/forum/vie...hp?f=1&t=76321

http://forums.winamp.com/showthread.php?t=376798

this i think is relevant to you (& me!) if say you have a rated FLAC, and you want to transcode it to ALAC m4a, b/c you want the tag info to also go with the transcode.

Media Monkey solved the issue years ago, (i reported it), by basically updating/saving both sets of tags simultaneously. yes, its duplicative, but it keeps info from being divergent and it also makes the info exposed in windows explorer:

http://www.ventismedia.com/mantis/view.php?id=12017

any chance you could fool around with this and get it to work with your plugin?

also, i think the media monkey solution needs to be adopted by winamp as well, esp for the ML, so that ratings and other tags don't become divergent.

So I just took another look at the encoder plugin source code. Nowhere in there do I add or modify file metadata except to annotate the (I think required) encoder and codec version strings. Winamp writes the other tags after transcoding with the input plugin that's handling/playing it. Tag writing is done with input plugins, not an encoder plugin. This plugin is purely an encoder plugin. You'd have to ask the Winamp developers to do that.

[MrSinatra]

Originally Posted by Benski

I can help you with using libmp4v2 to encode the ALAC file to mp4/m4a if you want. just e-mail me

Originally Posted by thinktink

Well I'll be damned, I didn't even see libmp4v2.dll. Looking at the exports I imagine it's far better than what I'm doing. Does it support in-memory reading/writting?

Originally Posted by MrSinatra

Fantastic!

Is this with winamps library, or your own implementation?

Did you get in touch with Benski?

I posted this thread to the fb winamp group.

Originally Posted by thinktink

I don't actually have a full implementation of the full ISO standard, just a basic hierarchical class handler to help me organize and construct the Moov box piece by piece. Once everything is in place, I call a single function which then triggers callbacks that outputs the data in it's proper order. All I had to do was update the class handler to support 64 bit boxes. I never did contact Benski as my preliminary research into libmp4v2 revealed that it seems to exclusively deal with files already on the drive, which doesn't work very well for a standard Winamp encoder. If memory serves, DrO described one of the encoders that ships with Winamp that just outputs dummy data to Winamp and writing the actual file contents to a temporary location and then swapping the file in once the data is finalized, I'm guessing that encoder uses the libmp4v2 library.

i realize i probably misunderstand / don't know how things work under the hood, but you are saying that when transcoding, your encoder isn't creating the m4a container, but rather the winamp libmp4v2.dll is? i was under the impression your encoder was standalone in that regard.

i would like to see all the apps that make m4a tags include WM tags and windows file property attributes, so everything is congruent; but i doubt this niche issue will get attention from winamp anytime soon.

[thinktink]

Originally Posted by MrSinatra

... i was under the impression your encoder was standalone in that regard.

...

For the initial m4a container creation, yes. However, there is no facility in the encoder API to receive metadata that I am aware of. Metadata is exclusively handled by input plugins, which is actually a smart move since if there was more than one handler things could get messy. I never did use libmp4v2.

[MrSinatra]

interesting... so if i use winamp to transcode a FLAC to ALAC using your encoder, should i expect Vorbis tags to xfer (and become m4a tags), and if so, how is their input plugin invoked to do it?

[thinktink]

Originally Posted by MrSinatra

interesting... so if i use winamp to transcode a FLAC to ALAC using your encoder, should i expect Vorbis tags to xfer (and become m4a tags), and if so, how is their input plugin invoked to do it?

As far as the exact structure and program flow for how Winamp determines which input plugin to use to write metadata tags I don't know. I do know that Winamp finds the input plugin that is able to play the file and tells that plugin to write the metadata tags when it's done transcoding and finalizing the file. Winamp gathers the original file's metadata from the input plugin that handles the original file's playback and then sends it to the new file's input plugin. Encoders themselves do not "transcode" files, Winamp does. Winamp coordinates everything from using the original file's input plugin to convert it to the raw PCM data, then Winamp directs the encoder plugin to encode the raw PCM data to the new codec, then using the new file's input plugin to write the metadata after encoding is complete.