Big Idea... Small Brain!

[TenBaz]

Apologies in advance for the long post. I have a lot to ask and wasn't sure where to put it. I hope it's OK here...

Anyway, one of my tasks at a radio station I worked for was to maintain the MP3 library.

The library I inherited has some 100,000 files, some with incorrect filenames, many duplicated and most with missing or incorrect ID3 tags.

The only tag of interest to me though is the year tag as the MP3 files are stored in decade folders - with all the 1970s files in the Seventies folder and so on. Having the Year tag correctly filled in the ID3 tags makes locating, then dragging and dropping files in Windows Explorer so much easier...

I've already written a suite of programs for renaming files, finding and deleting duplicates and moving them from one location to another. The stumbling block at the moment is getting the correct year in the ID3 tag.

I've tried pretty much every auto-tagging program out there but they all seem to have the same problem - they are all ALBUM-based. If I query say a track released in 1973, then the year tag invariably ends up being set to 2007 (or whatever) as that is the year it appeared on some compilation CD - rather than the year it was first released on vinyl (the year I want).

So, with that in mind, does anyone know if there currently exists any fully automated way to do this with software?

If there isn't, (which I believe is the actual answer to the above question), then I'm going to have to do it all manually. If this is the case then I'm thinking I would like to set up my own online database - like CDDB, but track-based rather than album-based.

Currently, in CDDB/freeDB for example, 'Elton John - Your Song' appears many, many times because it appears on many albums. In my proposed database it would appear only once, with the year listed as the year it first appeared as a single or album track.

I'd like users to be able to query the database using just the MP3 filename (artiste - song title) and get back all of the ID3 tag info, (including the earliest album that the track appeared on).

It would also be pretty essential for the database to be added to by others who wish to contribute information (like MusicBrainz or Wiki) as I appreciate it would be far too much work for one person.

So, does anyone know how would you go about starting something like this?

I already know how much of a bad idea this is, so I would appreciate it if you could restrict you comments to how you get a project like this off the ground.

After all, CDDB and Google were once just a dream in someone's head!...

Thanks in advance.

TenBaz

[TenBaz]

I think enough time has passed since starting this thread for me to post again without being accused of being impatient!...

As an update, in the end, I decided to start a database of my own called the MYRIAD Project. It is online and already has around 86,000 tracks.

It might be of interest to those of you who want the ID3 year tag in your MP3s to contain the year of release - rather than the year of some compilation album the track appeared on years after the track was originally issued.

It's a free to access community database and will be updated/maintained by volunteers.

A website is being built for it at the moment, but if anyone is interested, I can supply further details (if the forum rules allow posing of links).

TenBaz

[Warrior of the Light]

First, I don't think a new piece of software would be needed. The cddb should just be maintained better to give the oldest album first.
Second, Legally downloaded tracks have this all filled out already and when it comes to ripped CD's, you/they could (should) have added this data when you ripped, especially if you knew this would be so important.

[k_rock923]

I've had a huge amount of success with Picard, although I assume you've tried it already.

[TenBaz]

First, I don't think a new piece of software would be needed.

Who's talking about a new piece of software? I clearly said MYRIAD was a new online database.

The cddb should just be maintained better to give the oldest album first.

I couldn't agree more - but I have no control over that so I doubt it's going to happen - anytime soon at least. And, as I said, CDDB (or Gracenote as it is now) is album-based.

What if you are looking for the release years of single tracks not grouped together as complete albums? Correct me if I'm wrong, but I think I'm right in saying that CDDB is of no use to you.

In fact, the auto ID3 tagger built-in to WinAmp uses Gracenote and suffers the exact same year problem I'm talking about.

when it comes to ripped CD's, you/they could (should) have added this data when you ripped, especially if you knew this would be so important.

To quote myself in the opening post: "The library I inherited". I wasn't involved in the MP3s being created. In any case, many of the tracks were taken off CDs like 'Best Of The Sixties' and when I look at the actual CD cases, invariably they are dated post-2000 and don't list the original year of release for each track.

I've had a huge amount of success with Picard

Yes thanks - I have tried it and it works well with MP3 album collections. However, even their tech support admitted that it's not that accurate with single tracks and original release years.

In fact, it was they who put me onto the ID3 tagger I currently use - Jaikoz, which actually achieves more or less what I need. Even so, as it uses the MusicBrainz database, it's not without it's date errors.

I've also received replies in a similar vein from tech support at TidySongs, and Media Monkey.

So, everything points to my particular problem being database-related rather than software-related. From what I've seen, I'm not the only one who simply wants to query an online database with the name of a single MP3 filename and get the actual year it was released back.

I created the MYRIAD database to do just that and it's there if anyone else wants to do the same. If not, then there's no harm done!

TenBaz

[QOAL]

Okay...

So how would making a whole new database help?

You're allowed to contribute to musicbrainz and it has an sdk/api/web service.
What more do you need?

Looking at the web service it's not too hard to get what you want to know.
http://musicbrainz.org/ws/1/track/MB...rn&title=trash
Would return every entry for Korn - Trash, okay so no year yet.
But it looks like the list of entries is returned in order of the albums release, so now lets find out about the first album given.
http://musicbrainz.org/ws/1/release/...l&title=issues
Okay, so it require a minor amount of work to crawl through the tree and find the earliest date, which in this case doesn't matter as they're all the same year - just need to handle the date field correctly.

So no, you don't need a new database, you just need to learn how to use current ones and contribute corrections and missing information.

Write yourself a client instead.


*Hours later*

http://qoal.110mb.com/musicbrainz.php (Source http://pastie.org/pastes/1258331/text)
You put in the artist and the track name, and it'll come back with the earliest year it finds.
(Okay so it would be nice to get the year by only doing one query, but still.)

[TenBaz]

Okay so it would be nice to get the year by only doing one query, but still.

Precisely!

Single queries instead of multiple queries for thousands and thousands of tracks could save a hell of a lot of time.

Apart from that, if MusicBrainz and the other databases already have the info for old tracks released before CDs were invented and the original release years are so easy to get, then why don't any of the current taggers out there work properly?

And it's not just me saying this - I've emailed them and explained what I want to do with my MP3 collection and they all say the same - "it can't easily be done at this moment in time".

So no, you don't need a new database, you just need to learn how to use current ones

Well, the people who write the tagging software and MusicBrainz tech support do not entirely agree with you. I'm no expert, but I have to assume they know what they are talking about.

Neither of your first two links result in any years returned and your example is with a post-CD era artiste. Try getting the year of release for a sixties or seventies track. My database returns the correct year with a single query.

Your last URL only proves my point.

I tried a number of tracks with it and only once was the correct year returned. For example, the Queen single 'Seven Seas Of Rhye' was apparently released in 1995!

Entering the same tracks on my database query page returned 1973 for the Queen track - and the correct release years for all the others.

Write yourself a client instead.

I did, but couldn't get the databases to return what I wanted. That's why I contacted MusicBrainz tech support and they told me it couldn't easily be done. I started my own database which sat in the same folder as my program and it's this database I uploaded onto the internet.

To be honest, I think it's a waste of time anyone posting just to tell me that you can already do what many of us want to do when you apparently can't. Otherwise, TidySongs, Media Monkey, Picard, Jaikoz, WinAmp and all the others would do it already. As I said, I've contacted them and they say it's not possible with the current online databases - not me.

I'm only posting this to let people know that there is an online database now available that will return the year of release from the artiste and song title - if that's what they want. If not, then it it's obviously of little interest.

TenBaz

[QOAL]

Okay then, I have once more updated my example. Same link / Source (err one sec on the uploading actually, host seems down)

It now has a couple more options and I changed the requests a bit, still the same amount though.
(But of course, a song can be released several times under the same name and still be different, so not having a single record for a song is actually quite helpful.)

But it will now correctly return the earliest year for a single (if you pick the "Every returned single" option) and thus Seven Seas of Rhye returns 1974.

So it'll probably work correctly for any other erroneous results you had before.

Of course in the case of nothing coming back (which I haven't tested for) that would most probably mean it's not in the database.
But you know what, it's a lot easier adding a new entry for a few songs than creating a whole new blank database and trying to fill it up accurately.


I'm done.


Good luck with your own project though.

[TenBaz]

The link no longer works, so I couldn't try it out. You just get a text page of swearing.

I'm not a PHP coder, but I'll spend some time examining the PHP source code, so many thanks for that.

But if what you have done does now work, what puzzles me is why auto tagging software is unable to return the same correct results - and why their tech support claims it can't. Is it because all the work is done in PHP and simply isn't possible doing a query from a client program?

so not having a single record for a song is actually quite helpful.

Of course - we all have different requirements. As I said earlier, I'm dealing with a rather large radio station library where it's quite important to know the original release year of any given track. To me, all the other ID3 tags are irrelevant.

Most - not all - stations are less bothered about what version of a song they play than the year the track was released.

All the contacts in the radio business that I've spoken to so far have been quite enthusiastic and think it will save them a lot of time - as they too have wasted a lot of time looking into auto ID3 taggers which don't do what they need.

I also know that there are a lot of other people out there who manage their MP3 libraries in the same way - in year or decade folders.

it's a lot easier adding a new entry for a few songs than creating a whole new blank database and trying to fill it up accurately.

You are correct of course. But I've already put a lot of work into the project and in the the words of Magnus Magnusson, "I've Started So I'll Finish"...

Good luck with your own project though

Thanks. I appreciate you have only been trying to save me wasting my time. You may well be right...

But it's been fun doing it. If the project fails miserably and no-one is interested in it then you have my permission to say "told you so"...

Thanks for your comments in any case.

TenBaz

[QOAL]

Yeah that page of swearing the most ultra-permissive software license I'm aware of.
(The server my account is on, with my free host, is dying on it's arse so I decided to pull the live version in the hope that if the server falls over it wouldn't be my fault)

No, not really. It's not a question of language it's "If you're not getting the answers you're looking for, then maybe you're asking the question wrong. -- or not enough follow up questions."

Anyway, in light of the fact you can't run php code I bring you a working stand alone version, coded in C!

(It's a command line app - I hope I packaged all the required dlls with it, I did test it in a VM)
http://stashbox.org/1023396/musicbrainz.zip

Usage:
musicbrainz "Artist" "Track" Mode First

Modes: (Default 0)
0 = All releases, 1 = Album, 2 = Single, 3 = EP

First: (Default 0)
Check the first returned record only. 0 or 1


musicbrainz "Queen" "Seven Seas Of Rhye" will return 1973
musicbrainz "Queen" "Seven Seas Of Rhye" 2 will return 1974 (yay)
musicbrainz "Queen" "Seven Seas Of Rhye" 2 1 will return 1995


If it crashes then it's most likely due to it not handling unicode chars at all.

(Yes, I know I said I was done last time, but I couldn't resist... I really tried too!)

Hope you're still enjoying working on your db.

[TenBaz]

"Oops! Google Chrome could not find stashbox.org"

Assuming it might be temporarily down, I'll try again later...

TenBaz