Embedded Album Art and WA RAM Usage

[Aminifu]

If WA's (or any 32-bit app) RAM usage attempts to exceed 2 GB for any reason, during normal use, it will crash because Windows limits 32-bit apps to that much RAM under normal circumstances. It is possible for a 32-bit app to exceed the limit if certain procedures, not normally used, are applied. 64-bit apps normally have a 4 GB RAM limit, which also can be exceeded under certain circumstances.

Due to the total about of RAM in a computer and the number of apps running (including the OS), an app may not have the maximum amount of allowable RAM available to it. If an app attempts to use more RAM than what is available for it (intrudes on another app's RAM) it will also crash, even if it's RAM usage is below it's allowable maximum.

Windows will normally prevent apps from exceeding their RAM limits (allowable or available) by swapping data from RAM to drive storage and back again as needed. Drive storage used in this way is called virtual memory. Virtual memory is a lot slower than RAM, so apps that use a lot of it perform slower. All apps use some virtual memory from time to time during their normal execution.

It is fun to speculate and reasonably valid theories can be drawn if enough applicable facts are used. A couple of issues pertaining to embedded album art and WA RAM usage have been intruding on other threads, at times going close to being OT. The thread will let us vent and speculate with no worries about OT or highjacking. All feedback in the general area of this thread's title is welcome.

One Issue:
In the case of WA's RAM usage exceeding the normal (allowable or available) limit during scans of very large numbers of files with embedded art, there are not enough known facts to reach a reasonably valid theory as to why this happens, imo.

Another Issue:
Due to WA's design decision to not release RAM used for displaying album art, even after the art is no longer being displayed, and since it appears WA does not let this image data be swapped to virtual memory, an obvious conclusion can be drawn. Namely, under normal use, coupled with the RAM being using for other things, once enough images needed to reach the allowable or available limit have been displayed, WA will crash if it attempts to display another image or use more RAM for anything else.

Using embedded album art, especially high resolution art, allows the allowable or available limit to be reached much faster than not using embedded album art. RAM usage will increase for each unique image, once displayed, and for each multiple of the same image that is actively being displayed. This cumulative ****** can allow the allowable or available limit to be reached relatively quickly (the higher the image resolution, the fewer images it will take). The chances of having more unique images when embedding is higher than when using folder album art. With folder art, one image is used for all the tracks in the folder (usually one folder per album). When embedding art, it is more likely that all the tracks from the same album will not have the same image in them, unless they were all embedded at the same time.

Currently, the only way to release the RAM associated with WA's display of album art is to shutdown WA.

Users of embedded art need to be aware of this WA design approach and take all practical steps to prevent a crash due to displaying too much art during a single WA session. Those who let WA run days at a time need to be especially careful. The following steps can be taken, as applicable. They are listed in no particular order of importance.

1. Limit the number of different images in tracks from the same album.

2. Limit the use of views showing multiple images at the same time.

3. Keep an image size/resolution down to the smallest suitable for how its usually viewed.

4. Monitor WA's RAM usage and available system RAM at regular intervals. Allow a few hundred MB margin for error between what's being used and what is available for use and shutdown and restart below you enter it.

[MrSinatra]

first, i thought 64bit OS/apps could address more than 4gb?

my comments are all speculative and based on observation mostly, but here's what i have to add:

it could just be coincidence that victims of crashes at scan in [those with a lot of files who also happen to embed] have this occur b/c of the embeds. but it seemed to correlate a lot. i wonder if this might be due to the tag itself being too large to hold in memory, depending on how the db is written out during scans?

also, as i said back in this thread:

http://forums.winamp.com/showthread.php?t=348560

we can not as yet take as a given that all embeds are loaded. also, imo it likely does not matter if they are "the same" art as other embeds on a given album.

so, imo, #1 of your list does not matter in any likely scenario. (i am speaking of multiple images per album, multiple images in a single file might matter)

#2 seems plausible, since i think its been established that ram increases in an obvious manner when scrolling artwork views.

#3 seems like very good advice. using EAC, i generally pick 500x500 these days, usually ~50kb.

#4 also good advice.

i would add:

#5 avoid using embedded artwork unless you have a specific want or need of it. Folder based art doesn't seem to have all these problems.

[Aminifu]

Originally Posted by MrSinatra

first, i thought 64bit OS/apps could address more than 4gb?

Some 64-bit apps can, just like some 32-bit apps are designed to exceed the normal limit for 32-bit apps. My point is that there is a limit to what Windows will allow an app to address. Exceed that limit and the app crashes (could affect other running apps if it was able to write into the others memory space).

Microsoft has also placed arbitrary limits on what their 64-bit OS can address, based on which version you are using.

Originally Posted by MrSinatra

my comments are all speculative and based on observation mostly, but here's what i have to add:

so, imo, #1 of your list does not matter in any likely scenario. (i am speaking of multiple images per album, multiple images in a single file might matter)

#2 seems plausible, since i think its been established that ram increases in an obvious manner when scrolling artwork views.

#3 seems like very good advice. using EAC, i generally pick 500x500 these days, usually ~50kb.

#4 also good advice.

i would add:

#5 avoid using embedded artwork unless you have a specific want or need of it. Folder based art doesn't seem to have all these problems.

My comments are also based on observation. Some were based on faulty observations (like the results I was seeing resulting from a trailing space in album name tags), some of my current observations may likewise be faulty. Therefore I invite others to try to duplicate my results.

I think my suggestion #1 still matters if displaying 1 image (or multiple views of 1 image) at a time, not so much if the images are different, but if the sizes/resolutions are different. I agree it does not matter when using an album view, since only 1 image will be randomly selected for the album.

MP3s allow for multiple images in a file. I once used a plug-in that would cycle through multiple images in a mp3 file with an adjustable delay between images. This allowed the front and back album covers and anything else you wanted to display, to be displayed while the track was playing.

I do not believe your suggestion #5 is relevant to this thread because of the reasons stated in the first part of post #20.

[MrSinatra]

Originally Posted by Aminifu

I think my suggestion #1 still matters if displaying 1 image (or multiple views of 1 image) at a time, not so much if the images are different, but if the sizes/resolutions are different. I agree it does not matter when using an album view, since only 1 image will be randomly selected for the album.

(multiple views of one image? meaning like the same source file being displayed in the playlist, notifier, and file info window?)

lets recall the context. in the first post of this thread, you were talking about the RAM issues/limits. you then listed #1:

"1. Limit the number of different images in tracks from the same album."

you said that b/c you believed erroneously that more than one image per album would be displayed if given certain criteria regarding the artwork alone.

and thats what i was saying didn't matter to the ram issue, b/c winamp shows only one artwork per album for display. fair enough.

but going deeper: so even if it loads all the embeds somehow, as i contend it might, even though it only displays one artwork per album, it still doesn't matter insofar as you saying "different images in tracks" b/c even in that scenario i posit it wouldn't matter if the user was using, "different images" in tracks or not. each track/art would be its own weight based on its own unique attributes, whether "different" or not.

having said that, if winamp does load all embeds somehow, then yes, different sizes/resolutions probably do matter since that directly affects RAM usage.

the bottom line of my response though was that you simply DON'T have to limit the different images in an album at all, b/c it doesn't spawn multiple images to be displayed for otherwise correctly tagged albums, which was your ORIGINAL point, which you have now correctly rescinded.

Originally Posted by Aminifu

MP3s allow for multiple images in a file. I once used a plug-in that would cycle through multiple images in a mp3 file with an adjustable delay between images. This allowed the front and back album covers and anything else you wanted to display, to be displayed while the track was playing.

i'm not sure what your point here is, but if anything that would only make things potentially worse, depending on the reality of the under the hood mechanics.

Originally Posted by Aminifu

I do not believe your suggestion #5 is relevant to this thread because of the reasons stated in the first part of post #20.

of course its relevant. if winamp DOES have problems with embeds given whatever criteria, its sage advice. the whole point of any experiments is to prove or disprove it one way or the other. if proven, #5 is the best course of action for the present moment.

until it is proven that embeds are not the cause of these various issues, then given what i have observed and read, i think its wise to avoid them whenever possible.

[Aminifu]

Originally Posted by MrSinatra

(multiple views of one image? meaning like the same source file being displayed in the playlist, notifier, and file info window?)

Yes (could be other places depending on skin).

Originally Posted by MrSinatra

i'm not sure what your point here is, but if anything that would only make things potentially worse, depending on the reality of the under the hood mechanics.

That is my point. Multiple images in 1 file (if they are displayed) just lets users reach the breaking point faster. I saying users need to be aware of this and use this capability of mp3s carefully, as long as WA treats image data in RAM the way it does.

Originally Posted by MrSinatra

of course its relevant. if winamp DOES have problems with embeds given whatever criteria, its sage advice. the whole point of any experiments is to prove or disprove it one way or the other. if proven, #5 is the best course of action for the present moment.

The stated goal of this tread is "to discuss ways to identify and work-a-round real and suspected issues when using embedded art". How can not using embedded art be relevant to identifying solutions (partial or otherwise) to problems (real and suspected) when using it?

The relevant issue when displaying album art is, if users display enough images in one WA session, WA will crash due to excessive RAM usage (due to the way WA stores image data in RAM). The best course of action for now, for embedded art users, is to monitor WA's RAM usage and available system RAM at regular intervals and shut WA down before a crash is imminent. Embedded art users may have a greater chance of reaching the breaking point faster than folder art users, but this is good advice for them too, imo. They may only need to monitor in greater intervals.

The role embedded art has, if any, in the problem of the initial scanning of very large collections causing excessive RAM usage is unknown. The partial solution for now, for embedded art users, is to do the scan in stages. Monitor WA's RAM usage and available system RAM during scans and shut WA down before a crash is imminent. Then restart WA and the scan (which picks up from where it was stopped). Repeat the monitor - stop - restart process as many times as necessary to complete the scan.

Originally Posted by MrSinatra

until it is proven that embeds are not the cause of these various issues, then given what i have observed and read, i think its wise to avoid them whenever possible.

This 'solution' is like the old joke:

Patient says "Doctor, It hurts when I do this!"
Doctor says "Then don't do that."

That does not help when the user has a need or desire to do a thing. And what else does a user use anything for, if not due to a need or desire?

[MrSinatra]

continuing from the other thread:

Originally Posted by Aminifu

Last post by me on this topic in this thread, if that's rude so be it.

In a 3 panel media library view, list of artists on the top left (and whatever other data is shown in that panel), album images on the top right (with album names under the images, some truncated), and list of tracks in the bottom panel (with associated data in the columns selected for that panel).

list of artists, not albumartists? are the artist tags the same for all? in other words, are you citing single artist albums here, or VA/multiple artist albums?

i assume you are doing "album icons", as opposed to "album details" view?

Originally Posted by Aminifu

If I do a directed 'album tag' search for a particular album name (exactly as written in an album tag), I get the album's artist name on the top left, the album's image(s) on the top right, and the list of the album's tracks in the bottom panel. If those listed tracks all have the same embedded image, I see 1 image. If those listed tracks have different embedded images, I see 1 of each image that is different.

can you make example files available please?

Originally Posted by Aminifu

Or in the same 3 panel view, if I select an artist in the top left panel, I get all the tracks by that artist listed in the bottom panel. 1 image for each of the different images embedded in those tracks is shown in the top right panel. In one case, I had 14 tracks listed for tracks from 3 albums and 4 images were displayed. 4 instead of 3 were displayed because the embedded image in one of the tracks was a different resolution of the same cover used in the other tracks from the associated album. After I replaced the embedded image that was a different resolution, I now get 3 images displayed (1 for each album).

so i ask again, is it your contention that if you had 20 different artworks for 20 tracks of the same single artist album, you would then expect to see 20 albums listed??? (or put another way, all 20 artworks displayed in 20 placeholders for one album?)

[Aminifu]

Hi,

Able to pop back in for a few minutes.

Originally Posted by MrSinatra

so i ask again, is it your contention that if you had 20 different artworks for 20 tracks of the same single artist album, you would then expect to see 20 albums listed??? (or put another way, all 20 artworks displayed in 20 placeholders for one album?)

Yes, that is what I'm seeing on my system.

The 14 files I attached came from 3 albums. 1 from 1, 2 from another, and 11 from a third. Look at them in a Windows Explorer folder with icon view and you will see the images from the embedded art in each file.

Look at them in WA's media library and hopefully you will see what I described. I will be extremely interested if you don't.

Anyway, play with them. Change the embedded art in any or all of them and the media library should display 1 image for each image in the group that is different in any way.

Putting them in a folder with folder art should not change anything since WA reads embedded art first. It would to interesting to see what would happen in that case, if one or more of the files had their embedded art deleted. Would the folder art be used for them or would the embedded art in another file with the same album tag be used? I think I'll try that myself when I get some time.

[Aminifu]

Hi MrSinatra,

In response to your request in post #19 in the other thread, the 14 files I wrote about in post #20 of that thread are provided below. Sorry for the delay, my DSL upload speed is kinda slow.


http://www.mediafire.com/?11l2wp83eu2np3t

[Aminifu]

Hi MrSinatra,

I have to leave now, got some chores to do.

I will comment on your points here and from the other thread soon. Either later tonight or early next week. Gotta earn some money, so for the next 3 days I have to work. 12 hour shifts (6pm to 6am) do not leave a lot of time for much else.

Have a great weekend.

[MrSinatra]

you need to more closely examine your tags. open them in mp3tag.

Originally Posted by Aminifu

Yes, that is what I'm seeing on my system.

no, it isn't.

they are clearly 3 different albums BY TAG. embedded art has NOTHING to do with it.

Originally Posted by Aminifu

The 14 files I attached came from 3 albums. 1 from 1, 2 from another, and 11 from a third. Look at them in a Windows Explorer folder with icon view and you will see the images from the embedded art in each file.

if you KNOW they come from three different albums, and have them tagged such, which you do, then why would you think different embedded art has anything to do with it? you could delete all the art and you'd still have 3 albums.

let me put this another way...

you have 3 different albums here based on textual tags, namely the album tag. each of the three albums or "groups" has its own unique art.

you could remove an artwork from say track 6 of the "A Love Story" group, and you'd still only have 3 albums art displayed. you could then embed a random image into track 6 and you'd still only have 3 albums displayed afaik.

change the album tag to all the same value and you'll see only one album.

btw, some of your tracks have albumartist tags, and some don't. it doesn't affect anything here though b/c in this case the ones with a value equal each other, and the artist values.

[Aminifu]

You are confusing my answer to your question with the files I provided. The files are from 3 different albums. But if I had 20 files with the same album tag and 20 different images in them, then I would see 20 different images for that 1 album tag.

Do you still want me to go back and respond to past points or would that confuse things going forward?

[Aminifu]

I'm totally confused by the cross posting and my quick reading of your posts. We are not talking about the same stuff.

Its hard for me to read your posts, of more than a few sentences, quickly and grasp what you're saying. I usually have to read them a few times to be relatively sure I understand what is being said. That is not a knock on you or your writing style. Your style is just out of the norm for me and I just have to slow down and read carefully, we are more than a few generations apart.

I'm not sure what to say at this point or what those files were supposed to show you.

"you could remove an artwork from say track 6 of the "A Love Story" group, and you'd still only have 3 albums art displayed. you could then embed a random image into track 6 and you'd still only have 3 albums displayed afaik."

The quote above shows you did not understand what I wrote in post #20 in the other thread. I said I fixed those files so that only 3 images would be displayed for the whole group. Delete an image and yes you still get 3 for the whole group. But go ahead and use MP3Tag to change one of the embedded images of the "A Love Story" group (leave all the other tags as they are) and you will see 2 images when you select that group instead of 1. Change another one in that group and you will see 3.

I'm surprised that some of those files have albumartist tags. I don't use the albumartist tag and normally zero it out during my vetting process. Since they had the same values, I must have skipped deleting them in those files to save some time that day. I have WA set to substitute artist for albumartist whenever it needs to.

I know you use that tag and advocate its use. I even understand how you use it. I just don't have a need for it. I manage to fit everything I need in the artist tag, with the liberal use of the "&" character from time to time.

[MrSinatra]

the problem here, as i see it, is you are not listening to me.

here's a quick, easy experiment, try it:

i loaded your files into winamp, i saw 3 album's art/placeholders, as i would expect, b/c there are three different album tags.

i then edited the oddballs to all share the same album name, refreshed the view, and i saw... wait for it... ONE ALBUM ART/PLACEHOLDER.

can we just establish that please?

[Aminifu]

Exactly what view in what skin (I want to see it for myself)? Did you use a view like I described? I don't use Big Bento, but I thought it had something similar to what I described. I'm not disputing what you say, but I know what I saw.

[MrSinatra]

Bento, but that shouldn't matter.

albumartist\albumart or artist\albumart

small icons altho any of the others incl details are the same.

[Aminifu]

WA never ceases to amaze me!

I tried your experiment and got the same result, in Bento and my skin. Only thing is the cover is not from one of the 11 original "A Love Story" tracks.

Will you humor me and try what I suggested, put things back the way they were and change 1 of the embeds for 1 of the "A Love Story" tracks? I know I saw 4 images in that group when 1 of the "A Love Story" album tracks had a different image.

Gotta Go * and Wish We * are on the "Vivian" album and Love For Sale is on the "De-Lovely" album.

[MrSinatra]

Originally Posted by Aminifu

WA never ceases to amaze me!

I tried your experiment and got the same result, in Bento and my skin.

because thats how winamp works. so i trust you now concede you were mistaken? if so, i would ask that you re-read both threads since you now know things to be as i have portrayed them, and can better understand what i wrote from my POV with that new understanding, and we can now better diagnose together what we have yet to comprehend.

we actually went over this in detail some time ago:

http://forums.winamp.com/showthread.php?t=278586

Originally Posted by Aminifu

Only thing is the cover is not from one of the 11 original "A Love Story" tracks.

why should it be? its really neither here nor there. the point is it picks one (based on whatever under the hood underlying mechanics it uses) and shows JUST ONE. thats whats pertinent.

Originally Posted by Aminifu

Will you humor me and try what I suggested, put things back the way they were and change 1 of the embeds for 1 of the "A Love Story" tracks? I know I saw 4 images in that group when 1 of the "A Love Story" album tracks had a different image.

why don't you try it? you will see NO DIFFERENCE. winamp has no knowledge of what you know, it only knows what it is presented with in the moment, thats it.

why do you think winamp would behave differently given those criteria, than vs my experiment??? it won't!

[Aminifu]

Originally Posted by MrSinatra

why don't you try it? you will see NO DIFFERENCE. winamp has no knowledge of what you know, it only knows what it is presented with in the moment, thats it.

why do you think winamp would behave differently given those criteria, than vs my experiment??? it won't!

Ok, I will. I guess the reason I saw 4 images could be (must be) tagging error. Namely the file with the different image also had an extra space in the album name tag. Only I don't remember changing that, only the embedded image.

[Aminifu]

Now, I'm getting pissed.

When I tried to return the 3 tracks to their original album names, one of them now has no tags at all. I can replace that track from the archive I sent you, but why the f*** did that happen.

WA is getting real flaky all of a sudden. Like that guy with the Flac genre issue in the tag editor. Now I'm getting scare to use it.

[Aminifu]

Ok, my bad.

I had noticed that the track showing no tags in the editor had an extra space in it's file name causing it to be in the wrong place alphanumerically in Win explorer. I renamed the file in explorer to delete the space and did not rescan the library, so the library still had the file with the old name and when I opened it for editing of course nothing was there.

Although I would think, the editor should have told me file not found rather that open a blank editor window. Curious. But, I learned something new that WA does that's counter-intuitive.

[Aminifu]

Hi MrSinatra,

Yes, it must have been tagging error in the album name tag of the track with the different image. Sorry to have wasted so much of your and my time. An extra space at the beginning or end of a phrase needs to be ignored, but a character is a character and they all have meaning to a computer.

Now I need to go back through my collection and weed out these extra spaces where I have more than 1 image displayed for the same album. I should be done in about a month.

I also now recall the discussion from last year about the arbitrariness of WA's image selection when multiple images for the same album are available.

I will carefully go back over all your other points and if I have something sane to say, I'll comment.

[MrSinatra]

Originally Posted by Aminifu

Hi MrSinatra,

Yes, it must have been tagging error in the album name tag of the track with the different image. Sorry to have wasted so much of your and my time. An extra space at the beginning or end of a phrase needs to be ignored, but a character is a character and they all have meaning to a computer.

http://forums.winamp.com/showthread.php?t=299342

Originally Posted by Aminifu

Now I need to go back through my collection and weed out these extra spaces where I have more than 1 image displayed for the same album. I should be done in about a month.

https://www.google.com/search?hl=en&....1.m9Bmj-3klGY

Originally Posted by Aminifu

I also now recall the discussion from last year about the arbitrariness of WA's image selection when multiple images for the same album are available.

I will carefully go back over all your other points and if I have something sane to say, I'll comment.

i would like to continue the discussion now that we have an agreed upon common understanding. i would suggest we finish hashing it out here and then close this thread and start a new one based on the conclusions we reach.

[Aminifu]

Hi MrSinatra,

I do not advocate not using a WA feature unless it is hopelessly broken. I'm not overly concerned with the burden the use of some features places on WA support. I'm also not an advocate of protecting new users from themselves when they attempt to use features they do not know how to use properly. We all were new users at some point and we learn from mistakes. If instructions on how to use something are not available, that's a different issue. After years of using WA, I'm still learning stuff I was not aware of that WA has been able to do, or doing, for apparently quite some time

I believe the ability to use embedded art is a great feature. I agree that it is inefficient to use embedded art to display the same image for each track in an album. But album art does not have to be cover art, any image you want can be associated with a track. Embedded art guarantees that the image you want associated with a track is displayed for that track and can not be highjacked like folder art may be. Folder art restricts the user to one type of file storage structure. Embedded art allows for the use of any type of file storage structure. My goal with this thread is to discuss ways to identify and work-a-round real and suspected issues when using embedded art.

Not enough facts are known as to the involvement of embedded art in the issue with the initial scan of very large collections, imo. The most recent lengthy discussion related to this problem, that I'm aware of, started last March in response to user damenace's thread (including a video, which was started October of last year).

While it is not necessary to render embedded art during tag scanning, it may be happening. What is needed is knowledge of the code and design decisions as to what is supposed to happen during initial file scans for metadata that is included in the media library database. While it is desirable to have multiple pairs of eyes looking at the code (and any design documentation), we have (or had) only DrO looking into this. Posts #'s 92, 94, 101, 110, 115, 119, 145, and 147 in damenace's thread (link below) contain DrO's relevant comments, imo. From what he had time to look at, he appears to rule out embedded art involvement, imo. Post #138 by user Uwe Fissenebert also presents some interesting info. Anyway, much more investigation is needed.

http://forums.winamp.com/showthread.php?t=335960

As to the effect displayed art (embedded or folder) has on WA's memory usage, much more is known. Due to the early design decision to retain data in RAM for all art once displayed (to avoid re-rendering) and the apparent inability of WA to have art related data swapped to virtual memory, WA will exceed it's available RAM limit (could be up to the normal 2 GB maximum for a 32 bit app) and crash, when a sufficient number of art images have been displayed.

The number of displayed images it will take to crash WA depends on the amount of available RAM and the size/resolution of the images. I believe it is your contention that some memory is used each time an image is displayed and re-displayed and/or displayed multiple times at the same time. My use of the Windows Task Manager to monitor WA's memory usage (Working Set and Peak Working Set) does not seem to bear this out. To me, it seems like random leakage when displaying art causes memory usage to eventually increase to the breaking point.

The first time a unique image is displayed, WA adds related data to 1 of 3 data files. The art*.dat file used depends on the size/resolution of the image. If a lot of unique images are displayed, these art data files can grow quite large. On my system only 1 file (art_120.dat) is used because all my art is medium to high resolution. If you are using a skin that is displaying a media library view when WA is started or if a a media library view is opened during the session, all the data from these art data files is loaded into RAM. This data remains in RAM even if the media library view is closed. If you start WA without a media library view open and do not open one during the session (must use a skin, other than an all-in-one type, that allows all media library views to be closed), data from these art data files is not loaded into RAM.

I use a skin that displays a small view of the art associated with the selected or playing track in the main player view and in the pop-up notification when a new track starts or is selected. It also has a view called "Now Playing" that displays a large view of the art associated with the selected or playing track. I made 2 small playlists (14 tracks each) and monitored the Working Set and Peak Working Set memory usage while using the next and previous controls to cycle forth and back through the playlists to display the images associated with the tracks. I did not play the tracks, just cycled through them to display the images. I used 1 list to display the same image for each track and the other to display different images for each track. There was no significant difference in the results when using either playlist.

I paused between each track change to give the memory usage values time to stabilize. The first time through a playlist, I saw the Working Set memory rise and fall, consistent with WA using memory for more than just displaying art. I also saw the Peak Working Set memory increase with each track change, consistent with some memory being used each time an image is displayed. However, on subsequent passes through the playlists, I saw the Working Set memory move up and down, but the Peak Working Set memory did not continue to increase with each track change. The Peak Working Set memory did increase at random intervals and the increases were larger than those associated with single image changes the first time through the playlist. These results can be viewed in different ways, but I believe they are consistent with some kind of random memory leakage.

[MrSinatra]

Originally Posted by Aminifu

Hi MrSinatra,

I do not advocate not using a WA feature unless it is hopelessly broken.

and i do? not sure if you are addressing this to me or in general, but you start by saying my name, so i just want to state plainly that if so, its a strawman.

winamp does not itself embed images. and i have never suggested it should not READ or not allow one to delete embedded images.

i have suggested that users should not embed images, and obviously they would need a different app to do so.

its one thing to say using embedded images might be problematic for winamp, the ML, scanning, etc, which indeed i do say; and entirely another to say winamp should not use embedded images, which i have NEVER said.

its also not problematic to suggest that winamp might have issues with embedded images, and by saying so, it does not mean that if it does, it shouldn't fix them. of course i think any bugs should be fixed.

but i am free to suggest what i think will give a user the least amount of problems, and my advice is geared to that, and you of course are free to disagree. HOWEVER, it doesn't mean i think winamp shouldn't have the feature, or winamp shouldn't fix bugs.

Originally Posted by Aminifu

I'm not overly concerned with the burden the use of some features places on WA support. I'm also not an advocate of protecting new users from themselves when they attempt to use features they do not know how to use properly. We all were new users at some point and we learn from mistakes. If instructions on how to use something are not available, that's a different issue. After years of using WA, I'm still learning stuff I was not aware of that WA has been able to do, or doing, for apparently quite some time

fine, has nothing to do with me.

Originally Posted by Aminifu

I believe the ability to use embedded art is a great feature.

and i don't?

Originally Posted by Aminifu

I agree that it is inefficient to use embedded art to display the same image for each track in an album. But album art does not have to be cover art, any image you want can be associated with a track. Embedded art guarantees that the image you want associated with a track is displayed for that track and can not be highjacked like folder art may be. Folder art restricts the user to one type of file storage structure. Embedded art allows for the use of any type of file storage structure.

of course, but most people use one album art for albums. the vast overwhelming majority. for those who have a specific NEED or DESIRE otherwise, embedding the art, which they do with a different app, solves that issue, and in fact is not inefficient b/c the art itself is unique per track. i have no beef there. but its an exceedingly rare scenario in the real world.

Originally Posted by Aminifu

My goal with this thread is to discuss ways to identify and work-a-round real and suspected issues when using embedded art.

Not enough facts are known as to the involvement of embedded art in the issue with the initial scan of very large collections, imo. The most recent lengthy discussion related to this problem, that I'm aware of, started last March in response to user damenace's thread (including a video, which was started October of last year).

While it is not necessary to render embedded art during tag scanning, it may be happening. What is needed is knowledge of the code and design decisions as to what is supposed to happen during initial file scans for metadata that is included in the media library database. While it is desirable to have multiple pairs of eyes looking at the code (and any design documentation), we have (or had) only DrO looking into this. Posts #'s 92, 94, 101, 110, 115, 119, 145, and 147 in damenace's thread (link below) contain DrO's relevant comments, imo. From what he had time to look at, he appears to rule out embedded art involvement, imo. Post #138 by user Uwe Fissenebert also presents some interesting info. Anyway, much more investigation is needed.

http://forums.winamp.com/showthread.php?t=335960

my takeaway from that thread is nothing definitive has been proven one way or the other.

i would like to see my experiment run, which is namely to take 500k songs or so, embed them all with HQ art, and see if they can be scanned in. (i think not based on other reports)

then take the same 500k songs, strip all the art from them, (embeds, folders, etc), and see if you still get a crash. i bet not. but if you do, then retry the experiment, but this time with 400k songs, etc... do it several times until you know when it will reliably crash vs not crash.

you could also try the above with folder art only, as a 3rd test case.

i could be all wet, i fully acknowledge that. it could just be coincidence. it could be audio formats or who knows what. but i'd like to see someone with a real big library experiment like that. it would be interesting either way.

Originally Posted by Aminifu

As to the effect displayed art (embedded or folder) has on WA's memory usage, much more is known. Due to the early design decision to retain data in RAM for all art once displayed (to avoid re-rendering) and the apparent inability of WA to have art related data swapped to virtual memory, WA will exceed it's available RAM limit (could be up to the normal 2 GB maximum for a 32 bit app) and crash, when a sufficient number of art images have been displayed.

i tend to agree that winamp likely does this, although i notice that if i scroll, get the art, then scroll more, and then scroll back, it seems at times, in its execution of it, like its getting the art again for the first time...? but even if so, or even if thats a video card/driver/cache thing, etc, it still seems true that RAM is permanently captured by scrolling the art.

so until we have better data, we basically agree on this.

Originally Posted by Aminifu

The number of displayed images it will take to crash WA depends on the amount of available RAM and the size/resolution of the images. I believe it is your contention that some memory is used each time an image is displayed and re-displayed and/or displayed multiple times at the same time. My use of the Windows Task Manager to monitor WA's memory usage (Working Set and Peak Working Set) does not seem to bear this out. To me, it seems like random leakage when displaying art causes memory usage to eventually increase to the breaking point.

thats why i suggested an experiment.

it would be interesting to see if winamp "ate up" ram at a different pace when using embedded art in every file, vs folder art for each album, vs no art at all.

i think you said you got to 800 meg or so on your 5k files with embeds of HQ art, right? well i have 60k files of folder art at basically LQ, (mostly 200x200, with some at 500x500, with most well under 50kb, and almost all under 150kb). so i have 12 times as many files as you, but i don't recall getting to 800megs RAM.

i can't explain anything under the hood, i'm not a dev. i'm just trying to make sense of what i see, and what i read.

Originally Posted by Aminifu

The first time a unique image is displayed, WA adds related data to 1 of 3 data files. The art*.dat file used depends on the size/resolution of the image. If a lot of unique images are displayed, these art data files can grow quite large. On my system only 1 file (art_120.dat) is used because all my art is medium to high resolution. If you are using a skin that is displaying a media library view when WA is started or if a a media library view is opened during the session, all the data from these art data files is loaded into RAM. This data remains in RAM even if the media library view is closed. If you start WA without a media library view open and do not open one during the session (must use a skin, other than an all-in-one type, that allows all media library views to be closed), data from these art data files is not loaded into RAM.

I use a skin that displays a small view of the art associated with the selected or playing track in the main player view and in the pop-up notification when a new track starts or is selected. It also has a view called "Now Playing" that displays a large view of the art associated with the selected or playing track. I made 2 small playlists (14 tracks each) and monitored the Working Set and Peak Working Set memory usage while using the next and previous controls to cycle forth and back through the playlists to display the images associated with the tracks. I did not play the tracks, just cycled through them to display the images. I used 1 list to display the same image for each track and the other to display different images for each track. There was no significant difference in the results when using either playlist.

I paused between each track change to give the memory usage values time to stabilize. The first time through a playlist, I saw the Working Set memory rise and fall, consistent with WA using memory for more than just displaying art. I also saw the Peak Working Set memory increase with each track change, consistent with some memory being used each time an image is displayed. However, on subsequent passes through the playlists, I saw the Working Set memory move up and down, but the Peak Working Set memory did not continue to increase with each track change. The Peak Working Set memory did increase at random intervals and the increases were larger than those associated with single image changes the first time through the playlist. These results can be viewed in different ways, but I believe they are consistent with some kind of random memory leakage.

thats all interesting but its just guessing at whats going on.

what would actually mean something to the devs is being able to say:

A. if you do this, then this happens reliably.
B. if you then do this other thing, this then happens reliably.
C. given those two things, we can say that something is not right.

almost all my bug reports that have gotten action have been done in this way. you have to give them the ability to recreate it themselves, and then give them your theory as to why you think its happening, for them to either prove or disprove. but thats ultimately not the important thing, the important thing is that they, the devs, have to then work through the issue to be able to explain and or fix something that is otherwise demonstrably incongruent.

[Aminifu]

Originally Posted by MrSinatra

what would actually mean something to the devs is being able to say:

A. if you do this, then this happens reliably.
B. if you then do this other thing, this then happens reliably.
C. given those two things, we can say that something is not right.

almost all my bug reports that have gotten action have been done in this way. you have to give them the ability to recreate it themselves, and then give them your theory as to why you think its happening, for them to either prove or disprove. but thats ultimately not the important thing, the important thing is that they, the devs, have to then work through the issue to be able to explain and or fix something that is otherwise demonstrably incongruent.

During my years of programming, bugs came in one of 3 general categories.

Easy To Fix:
easily repeatable
one cause

Hard To Fix:
easily repeatable
multiple causes

Extremely Hard To Fix:
not easily repeatable (random)
one or more causes

I would like to think all kinds of bugs are important to WA devs, as they are to most devs. But the desire to fix and the ability to fix are two separate things. Things that could fit in your list would fall in the Easy To Fix and Hard To Fix groups, so it is no surprise if most of those got fixed. Extremely Hard To Fix bugs take a long time to fix, many never are.

I also would like to think that users speculations based on facts (not wild guesses) would mean something to the WA devs, even though 'wags' are useful on occasion. Sometimes a dev is too close to a problem to see it and a 'wild hint' from the outside provides just the refocus needed to spy it out.

[Aminifu]

Mr Sinatra,

I think most of your points, not related to my faulty observations due to tagging error, in this and the other thread have been addressed in posts #20 and #21. If I missed something you want me to comment on in particular, please point it out.

[MrSinatra]

Originally Posted by Aminifu

Mr Sinatra,

I think most of your points, not related to my faulty observations due to tagging error, in this and the other thread have been addressed in posts #20 and #21. If I missed something you want me to comment on in particular, please point it out.

i am going to attempt to reply to everything in the next post. i might need to edit it several times. i will let you know when i plan no further edits. please wait to reply until i have said such.

[MrSinatra]

so thats my attempt to respond to what you were saying. given everything thats transpired, our posts are getting long winded. therefore i wanted to concisely put into this post things i'm still interested in regarding these issues:

1. experiments with scan ins. as said, a large collection with HQ embeds, with HQ folder art, and with no art. see if we can break scan ins based on art status alone, in a reliable, reproduceable way. that would be one worthwhile experiment.

2. experiments with RAM usage, based on the same 3 situations above. you said definitively in the other thread (post 4) embeds will NOT use more ram vs folder art, but you haven't proven that. you may be right, it seems intuitive i agree, but it may not be the case.

3. i think its definitely the case that if you have one album of files that has 8 tracks and all 8 tracks show in ML b/c the tags are set such that it displays as 8 albums, (say via Disc 1-8 values) that definitely uses more RAM, regardless of if its the same image or not. you said different in post 4 of that thread, but granted that was before you realized your error, so i hope you see that now.

thats important b/c one of my contentions is that every artwork that winamp "finds" is seen as "unique" by winamp, ergo its location [probably] matters. that would really become important if they were all being referenced in some bad design mechanics under the hood.

a folder art would only spawn ONE location per album, while embeds, whether the "same" or not per album, would spawn however many locations as there are tracks per album, (and possibly provoke multiple loads beyond locations?)

again, speculation.

4. given that winamp makes no artwork comparisons or contrasts, its safe to assume that each art is considered unique by winamp. it may load only one art per album when scrolling, but when playing a 20 track album of embeds, it will load 20 unique artworks even if all 20 are the same art, b/c each track/art has a unique location. this would prob inflate RAM regardless of art scrolling.

5. if you were to look at a VA album, you might see multiple arts esp if you don't use albumartist tags. in fact, you can switch between scrolling art in artist\artwork or albumartist\artwork views, and winamp will show more art placeholders in artist view instantly.

6. it would be interesting to see how winamp handles deleting artwork from alt+3 windows, esp from embeds. does the next art load/show immediately? what about when switching views and back again, or refreshing views, or refreshing artwork? this may provide clues as to whether "unseen" art is loaded or inflating ram somehow, (as #5 might suggest as well).

ok, i am done editing this post and previous posts.

[Aminifu]

Originally Posted by MrSinatra

1. experiments with scan ins. as said, a large collection with HQ embeds, with HQ folder art, and with no art. see if we can break scan ins based on art status alone, in a reliable, reproduceable way. that would be one worthwhile experiment.

2. experiments with RAM usage, based on the same 3 situations above. you said definitively in the other thread (post 4) embeds will NOT use more ram vs folder art, but you haven't proven that. you may be right, it seems intuitive i agree, but it may not be the case.

3. i think its definitely the case that if you have one album of files that has 8 tracks and all 8 tracks show in ML b/c the tags are set such that it displays as 8 albums, (say via Disc 1-8 values) that definitely uses more RAM, regardless of if its the same image or not. you said different in post 4 of that thread, but granted that was before you realized your error, so i hope you see that now.

thats important b/c one of my contentions is that every artwork that winamp "finds" is seen as "unique" by winamp, ergo its location [probably] matters. that would really become important if they were all being referenced in some bad design mechanics under the hood.

a folder art would only spawn ONE location per album, while embeds, whether the "same" or not per album, would spawn however many locations as there are tracks per album, (and possibly provoke multiple loads beyond locations?)

again, speculation.

4. given that winamp makes no artwork comparisons or contrasts, its safe to assume that each art is considered unique by winamp. it may load only one art per album when scrolling, but when playing a 20 track album of embeds, it will load 20 unique artworks even if all 20 are the same art, b/c each track/art has a unique location. this would prob inflate RAM regardless of art scrolling.

5. if you were to look at a VA album, you might see multiple arts esp if you don't use albumartist tags. in fact, you can switch between scrolling art in artist\artwork or albumartist\artwork views, and winamp will show more art placeholders in artist view instantly.

6. it would be interesting to see how winamp handles deleting artwork from alt+3 windows, esp from embeds. does the next art load/show immediately? what about when switching views and back again, or refreshing views, or refreshing artwork? this may provide clues as to whether "unseen" art is loaded or inflating ram somehow, (as #5 might suggest as well).

ok, i am done editing this post and previous posts.

Some of these things you could do yourself (like number 6 with the files I provided earlier). I'm pretty sure these excessive memory usage problems would end if WA changed the way it stores image data in RAM.

I'm content for now to wait on the devs to have a chance to study the issue, my collection is not large enough to cause me any problems scanning, and will not be for quite some time. The way I currently display album art does not use excessive RAM during any one session of WA use, so fortunately I'm safe on that score as well.

I've already provided some observations and will post more if I come across any. I'm willing to try to duplicate what others see, when possible. But, I'm not willing to disturb my collection and storage scheme.

[Aminifu]

Hi MrSinatra,

Post #20 was addressed to you, but was meant for you and anyone. This post is for you.

In post #20, I presented a few statements on my POV, offered a few facts and some observations, and a few speculations. I never said or meant to imply that you do or don't "advocate not using a WA feature unless it is hopelessly broken" or that you do or don't "believe the ability to use embedded art is a great feature". My POV statements should not be taken to imply anything about your POV, regardless of where they agree or differ.

In all 4 instances, my use of the word "you" was meant in general, not personal. I probably should have used the word "users" instead. The only personal reference was in paragraph 6, were I rephrased your statement concerning RAM usage for each image, beginning with the words "I believe it is your contention".

I don't believe that because WA natively provides a way to copy images to a folder and does not provide a way to embed them in file implies a bias for one method over the other. I don't believe that because WA looks for and will use embedded art before folder art implies a bias for one method over the other. I also don't believe that WA dropping the album art look-up feature implies a bias against album art, in general.

I don't believe using other apps to support what WA does or doesn't do natively is any big deal or a reason for or against anything. We both use other apps for ripping and tagging (which WA does natively). I use another app for volume leveling mp3s (which WA does natively). What's so different with using other apps to embed images and now, look-up images?

I accept others will have opposing views. The opposing views may be the majority. That does not matter to me. I believe the needs of the few or one are just as important as the needs of the many. The fact that the needs of the few or one are often sacrificed or overlooked for the needs of the many does not make one group of needs any more important than another, imo. For example, what does it matter if most use album art for cover art? Some don't.

[infoleather]

Why do you think the different embedded art what do you use it? You can delete all the art, you still have three albums.

[Aminifu]

Originally Posted by infoleather

Why do you think the different embedded art what do you use it? You can delete all the art, you still have three albums.

I don't understand your question. If you're asking why use album art, then the answer is that it is nice to have. Images are not necessary, after all music is something you hear and feel.

But album art is nice 'eye candy'. Plus the default big Bento skin features it. Something to look at while listening, that triggers other mental images. Album art makes it faster selecting the album you want to play (as opposed to reading a list of text). The visual images help you remember albums in a large collection, that you may otherwise overlook or forget you have.

I use embedded art so that each track may have a different image, if I want them to. Mostly I use album covers, but sometimes I mix it up a little, with pictures of the artists and other images related to the songs.