Do you have album art stored for those albums? Winamp can only display the album art under one of three conditions:
Have an image labeled "folder.jpg" in the folder of the album. This way, Winamp will connect that cover art to any songs in the album folder.
Embed the album art within each song's tag. Using MP3Tag
is a great, free program for doing this.
Right-click on an album in the album art view, and select "Get Album Art". Winamp's Cover Art Downloader will then attempt to retrieve a suitable artwork match for the album.
By following at least one of these methods, album art should start displaying correctly in Winamp.
Hope that helps,