Old 21st February 2007, 11:25   #1
Flieger
Junior Member
 
Join Date: Feb 2007
Posts: 7
Winamp 5.33: multiple entries in ML

Hi!

after each rescan i get multiply media library references on SAME files!

i have this problem since 5.33!
how can i repair my media library and solve this problem?

Thanks!
Flieger is offline   Reply With Quote
Old 21st February 2007, 12:34   #2
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Yeah, we're a bit embarrassed by this issue, especially since 5.33 was available as a public beta for 2 weeks prior to release, but no-one managed to find/report this bug.

I've added it to the Regressions section of the Known Bugs thread.


_____________________________________________________________


-Potential for some duplicate entries in Local Media after rescanning Watch Folders *

_____________________________________________________________


Legend
* = fixed in latest internal builds

_____________________________________________________________



Note that the problem cannot be reproduced on clean installs, only on upgrades.

It's fixed for 5.34, but in the meantime....

______________________________________________________________


EDIT

Fix now available below
DJ Egg is offline   Reply With Quote
Old 21st February 2007, 12:45   #3
Flieger
Junior Member
 
Join Date: Feb 2007
Posts: 7
does this mean you knew about this bug and it only happens by upgrading 5.32 to 5.33?

so if i uninstall my current corrupt version und reinstall a clean v5.33, it'll work?
Flieger is offline   Reply With Quote
Old 21st February 2007, 12:49   #4
DrO
 
Join Date: Sep 2003
Posts: 27,873
we didn't know about the issue until after 5.33 was released (most likely because people were trying the beta out in a separate clean install in most cases).

as Egg says, removing the duplicates/cleaning the library and starting again is the way to go

-daz
DrO is offline   Reply With Quote
Old 21st February 2007, 18:05   #5
Benski
Ben Allison
Former Winamp Developer
 
Benski's Avatar
 
Join Date: Jan 2005
Location: Brooklyn, NY
Posts: 1,057

Unzip this file into your Winamp folder and run it (make sure Winamp is closed first).

It will re-index your database and remove any exact duplicate entries.

Before you run, be sure to backup your old database (usually in C:\program files\winamp\plugins\ml\main.dat and main.idx - unless you installed with multi-user settings, then it's in c:\documents and settings\<yourname>\Application Data\Winamp\Plugins\ml).

Let me know if it works, and how many duplicate files it removed.
Attached Files
File Type: zip reindex.zip (4.0 KB, 1703 views)
Benski is offline   Reply With Quote
Old 23rd February 2007, 01:16   #6
decoy_oct
Junior Member
 
Join Date: Feb 2007
Posts: 1
Send a message via AIM to decoy_oct
Thanks for the patch. I just updated to 5.33 today and had the same bug.

It found 304 duplicates (9563 songs down to 9259). It works as long as Winamp isn't running I kept my original main.dat & main.idx as well, so if you want them for comparative purposes, let me know.
decoy_oct is offline   Reply With Quote
Old 24th February 2007, 15:44   #7
StimpyCat
Junior Member
 
StimpyCat's Avatar
 
Join Date: Nov 2005
Posts: 14
Reindex Works!

DJ Egg,
Thank you, thank you, thank you! Did I mention thank you! The reindexing of the media library worked. 233 duplicates were removed. All my tracks are OK. It was also nice to see that this bug was fixed in the upcoming 5.34 version. Once again, thanks. Keep up the good work.
StimpyCat is offline   Reply With Quote
Old 3rd March 2007, 02:05   #8
chimichanga15
Junior Member
 
Join Date: Jun 2001
Location: California/Minnesota
Posts: 19
Send a message via AIM to chimichanga15
Benski,

It removed 1122 duplicates out of 32,353 files. It worked great, except that it took about 10 minutes to execute.
chimichanga15 is offline   Reply With Quote
Old 4th March 2007, 04:46   #9
niZmO_Man
Junior Member
 
Join Date: Mar 2007
Posts: 3
Hey guys, I know its a bit off-topic, but I get the same duplicate entries problem with Playlist, so bring back the "remove duplicate entries" options in Remove...
Keep up the good work I only use Winamp, it is the #1 player in the universe, just ask all the aliens out there!
niZmO_Man is offline   Reply With Quote
Old 4th March 2007, 09:41   #10
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Wrong thread, niZmO_Man.

That menu item is only there as long as you don't uncheck "Extended Jump to File Support" in the Installer Options (ie. no jtfe / gen_jumpex.dll, no 'remove duplicates' option in the playlist).
DJ Egg is offline   Reply With Quote
Old 11th March 2007, 06:34   #11
Lenneth
Junior Member
 
Join Date: Jan 2007
Posts: 5
Does Winamp need to be closed in order to run this? Asking because I'm on a network and it's not so easy to shut it down.
Lenneth is offline   Reply With Quote
Old 11th March 2007, 09:53   #12
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Yes
DJ Egg is offline   Reply With Quote
Old 13th March 2007, 15:47   #13
Marshdweller
Junior Member
 
Join Date: Jan 2006
Posts: 18
It removed 82 duplicate entries!

Thanks a bundle good people of Winamp!
Marshdweller is offline   Reply With Quote
Old 15th March 2007, 06:10   #14
bitchtow
Junior Member
 
Join Date: Jun 2002
Posts: 40
What is nde.dll, reindex.exe won't run because I'm missing it, what specific runtimes does it run on?
bitchtow is offline   Reply With Quote
Old 15th March 2007, 08:07   #15
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
nde.dll is the Nullsoft Database Engine.
The Media Library won't run without it, never mind Reindex.

nde.dll is included with the Winamp installer, and is installed alongside the Media Library. It should already be in the Winamp root folder, where winamp.exe is. The reindex.exe file also needs to be placed in the Winamp root folder, and Winamp needs to be closed befor running the Reindexer.

All that said and done, Winamp 5.34 Beta is now available, which has the Reindex tool built in. The Reindexer in 5.34 will auomatically run on first Winamp startup, or if the ML isn't open on first run then when you first open the ML.
DJ Egg is offline   Reply With Quote
Old 15th March 2007, 08:39   #16
bitchtow
Junior Member
 
Join Date: Jun 2002
Posts: 40
Cheers, I must have misread the directions coz I dumped it in the ml directory and ran it from there. I did install the 5.34 beta and I didn't notice it reindexing, and I still have double ups, so I guess it didn't run. I tried a second time and same thing.

So then I tried using the reindex.exe from the root of the winamp directory and I get a weird error saying "the entry point ?Query@Scanner@@QAEHPA_W@Z could not be located in the dll nde.dll"

I'm not too fussed about blasting the library and rebuilding it from scratch, but I'd rather not.
bitchtow is offline   Reply With Quote
Old 15th March 2007, 10:47   #17
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
You've probably unchecked some items in the Installer Options, and therefore the newer ml components, including nde.dll, didn't get installed. Either that, or the external Reindexer tool is designed not to work with the nde.dll included with 5.34.

Open Winamp
Make sure the Media Library window is open
Close Winamp

Go to the Winamp\Plugins folder
Open gen_ml.ini in Notepad

If any of the following lines exist (probably at the bottom), then delete them:

reindex_533=1
reindex_534=1

Save the changes, and close Notepad.

Now reinstall 5.34 beta
Make sure the relevant items are not unchecked in the Intaller Options, eg.
Winamp Library > Core Components > Local Media

The 5.34b built-in Reindexer should kick-in as soon as Winamp and the ML open (after completion of the installation process).


ps. Note, if you selected 'shared settings for all users' on screen 5 of the installer options (default), then gen_ml.ini will be in the Program Files\Winamp\Plugins folder (or wherever you installed Winamp to), but if you chose 'separate settings per user' then gen_ml.ini will be in your %appdata%\winamp\plugins folder (where AppData on Win2k/XP is usually found at: C:\Documents and Settings\(username)\Application Data).
DJ Egg is offline   Reply With Quote
Old 16th March 2007, 05:57   #18
bitchtow
Junior Member
 
Join Date: Jun 2002
Posts: 40
Well, that got it reindexing, but it didn't remove the duplicates, well not the ones I noted. I guess I'll just blast the ml, it's not like it has a great deal of info anyway, just the play counts.
bitchtow is offline   Reply With Quote
Old 16th March 2007, 09:38   #19
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Strange. Are the duplicates definitely the exact same path and filename, and not just the same album/songs but stored in a different folder or with a different filename?
DJ Egg is offline   Reply With Quote
Old 16th March 2007, 09:49   #20
bitchtow
Junior Member
 
Join Date: Jun 2002
Posts: 40
Yeah, they definitely had the same path and file details, the only details that were different were the play count values, which I assume was because they were 'different' media library entries. But I've blasted away the ml now and it's duplicate free. So I can't help with any sort of debugging now.
bitchtow is offline   Reply With Quote
Old 18th March 2007, 13:01   #21
leebotton
Junior Member
 
Join Date: Dec 2006
Posts: 1
16471 duplicates removed!! :-D lol
Unfortunately though, when I upgraded, the play counts were removed but not the end of the world I guess...
leebotton is offline   Reply With Quote
Old 22nd March 2007, 12:00   #22
kenwoods
Member
 
kenwoods's Avatar
 
Join Date: Nov 2005
Posts: 73
Quote:
Originally posted by bitchtow
Yeah, they definitely had the same path and file details, the only details that were different were the play count values, which I assume was because they were 'different' media library entries. But I've blasted away the ml now and it's duplicate free. So I can't help with any sort of debugging now.
I have the same problem and I don't want to blast my media library!!!
I only want to remove the duplicates
kenwoods is offline   Reply With Quote
Old 22nd March 2007, 12:47   #23
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
@ kenwoods

I've already told you what to do if you require further help.
Ignoring our requests won't get you anywhere.

http://forums.winamp.com/showthread....19#post2158319
http://forums.winamp.com/showthread....40#post2158540
DJ Egg is offline   Reply With Quote
Old 24th March 2007, 23:01   #24
niZmO_Man
Junior Member
 
Join Date: Mar 2007
Posts: 3
Quote:
Originally posted by DJ Egg
Wrong thread, niZmO_Man.

That menu item is only there as long as you don't uncheck "Extended Jump to File Support" in the Installer Options (ie. no jtfe / gen_jumpex.dll, no 'remove duplicates' option in the playlist).
Sorry about that I only realised when I reinstalled, thanks for that anyway
niZmO_Man is offline   Reply With Quote
Old 27th March 2007, 13:26   #25
kenwoods
Member
 
kenwoods's Avatar
 
Join Date: Nov 2005
Posts: 73
DJ Egg, I've submitted the files in the relevant post...

Thanks Again...
kenwoods is offline   Reply With Quote
Old 27th March 2007, 13:48   #26
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
@ kenwoods

I forwarded the files to Benski, and he said he'd look into it...

In the meantime, could you please tell us what some of the duplicate entries are?
This might make it easier for us to diagnose the problem, if we know beforehand what entries we are looking for.

Artist, Title, and Filename info should do...
(though Album & Track# also would help even more, if possible).
DJ Egg is offline   Reply With Quote
Old 27th March 2007, 13:56   #27
kenwoods
Member
 
kenwoods's Avatar
 
Join Date: Nov 2005
Posts: 73
DJ Egg,
for example:
all of Pink Floyd's "The Wall (Pink Floyd - The Film)"
is duplicated...
here's one filename off of it for example:
"E:\My Music\Pink-Floyd-The-Wall,The-Film-(Soundtrack)-1982\09 - Mother.mp3"

another example is all of Rammstein's "Mutter" album (the filenames don't use any type of special unicode)
here's a filename for example: "E:\My Music\CD's I Own\Rammstein_-_Mutter\03-Sonne_Rammstein.mp3"
kenwoods is offline   Reply With Quote
Old 27th March 2007, 15:27   #28
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
I couldn't find "E:\My Music\Pink-Floyd-The-Wall,The-Film-(Soundtrack)-1982\09 - Mother.mp3" listed anywhere in main.dat

I found:

E:\My Music\CD's I Own\Pink Floyd - The Wall\06 - Mother.mp3

E:\My Music\Pink.Floyd.-.[Is.There.Anybody.Out.There,.The.Wall(Live)](MP3)\Is There Anybody Out There, The Wall Live 1980-81 - Cd 1\Pink Floyd - (07)Mother.mp3


And I found "E:\My Music\CD's I Own\Rammstein_-_Mutter\03-Sonne_Rammstein.mp3" listed once.


Are you sure that these media library entries are exact duplicates of the exact same files with the exact same path & filename?

The reindexer will only remove exact duplicates. It won't remove anything if the path\filename is different.


Have you tried doing: Library button > Remove missing files...?


[Edit]

Okay, I loaded your main.dat/main.idx into a clean install,
and sure enough, I can see the duplicate entries.

Weird!

Thanks :-)
DJ Egg is offline   Reply With Quote
Old 28th March 2007, 16:30   #29
aperture01
Junior Member
 
aperture01's Avatar
 
Join Date: Oct 2004
Posts: 45
Send a message via ICQ to aperture01 Send a message via AIM to aperture01
I have the same problem, I ran the reindexer and still have multiple database entries.
aperture01 is offline   Reply With Quote
Old 28th March 2007, 19:07   #30
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
@ kenwoods

We ran the Reindexer on your database, and it removed 1597 duplicates.

Did you definitely try the standalone Reindexer from above?

Be sure to read the instructions first.

(It's possible that the built-in one in 5.34 beta is borked, or maybe you didn't close winamp before running the standalone reindexer).
DJ Egg is offline   Reply With Quote
Old 28th March 2007, 20:21   #31
kenwoods
Member
 
kenwoods's Avatar
 
Join Date: Nov 2005
Posts: 73
DJ Egg,
the reindexer didn't work with 5.34 so I reinstalled 5.33
and used the reindexer and it worked...

Thanks...
(I've had this problem for ages with 5.33, I didn't know you've released a stand alone application for 5.33 back then...)
kenwoods is offline   Reply With Quote
Old 28th March 2007, 20:25   #32
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Doh!

Okay, cool

tfft
DJ Egg is offline   Reply With Quote
Old 4th April 2007, 13:14   #33
foxyshadis
Member
 
Join Date: May 2002
Location: The deserts of California
Posts: 60
Send a message via AIM to foxyshadis Send a message via Yahoo to foxyshadis
I've been living with this problem for a while, when the last version that caused duplicates cropped up. I could clear out, but once you rely on the library for a while that's hard, and I admit I've just been too lazy to make an ActiveWinamp script to fix it all, merging the stats from all the dupes.

So I offer up my media library if you want to look. Reindexer finds no duplicates, but a simple scan in a hex editor shows multiple copies of many files - buckethead is particularly afflicted, as many as 4 copies of some files; some copies are listed as ANSI, some as Unicode. Look up "05-buckethead-lurker_at_the_threshold_part_1.mp3". The ANSI files seem to be older holdover dupes, the unicode ones are newer. Dunno why reindex doesn't find them, maybe the newer version you have will.

btw, if you need to quickly search for unicode names, the tool XVI makes it quite easy, with a UTF-16 checkbox. Maybe that was why kenwoods' songs weren't coming up.

The ml is the 5.33 backup without reindex run on it, btw, so that it's clean for you.
foxyshadis is offline   Reply With Quote
Old 4th April 2007, 14:28   #34
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Hmm, strange.

I can see three "D:\foxy\stuff\05-buckethead-lurker_at_the_threshold_part_1.mp3" entries

Did you definitely place reindex.exe in the Winamp root folder and run it whilst Winamp was closed?

Note that the Reindexer worked for kenwoods.
DJ Egg is offline   Reply With Quote
Old 4th April 2007, 21:17   #35
foxyshadis
Member
 
Join Date: May 2002
Location: The deserts of California
Posts: 60
Send a message via AIM to foxyshadis Send a message via Yahoo to foxyshadis
I did, and used the 5.34 built-in indexer as well, and like you said it cleared some things out. Just left a lot of others.

Is it possible to get the reindexer source, if it can be built with the public SDK? (If not, well, no point.)
foxyshadis is offline   Reply With Quote
Old 4th April 2007, 21:34   #36
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
The reindexer in 5.34 b1195 is borked, but the standalone should have worked (you can still run it with 5.34 beta, as long as it's placed in the same dir as winamp.exe and winamp is closed at the time).

As for the source.... umm, I don't know (only Benski can answer that one).
DJ Egg is offline   Reply With Quote
Old 12th April 2007, 12:34   #37
5Pack
Guest
 
Posts: n/a
Reindex ran successfully.

Took 5 to 10 minutes. Removed 818 duplicates from a 20,000+ library. I had already manually removed about 400.

I had been using the Playlist|Remove Duplicates menu item to clean the playlist. You may want to add a similar command for the Media Library.
  Reply With Quote
Old 12th April 2007, 14:55   #38
DJ Egg
Techorator
Winamp & SHOUTcast Team
 
Join Date: Jun 2000
Posts: 35,821
Cool.

Note, there shouldn't ever be any duplicates (same path+filename) in the ML,
so there wouldn't be much point in adding a Remove Duplicates command
(ie. it was a 5.33-specific bug)
DJ Egg is offline   Reply With Quote
Old 15th April 2007, 18:50   #39
foxyshadis
Member
 
Join Date: May 2002
Location: The deserts of California
Posts: 60
Send a message via AIM to foxyshadis Send a message via Yahoo to foxyshadis
The new reindexer in r1239 took care of everything I could think of. Thanks!
foxyshadis is offline   Reply With Quote
Old 17th July 2008, 17:26   #40
matrix626
Junior Member
 
Join Date: Jul 2008
Posts: 1
I was wondering if it was possible to have a setting to remove dupilcate listings of the same song with partially different filenames....such as have a setting to remove duplicate titles for instance?
matrix626 is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Winamp > Winamp Bug Reports

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump