Considerable size drop

Discussion in 'Acclaim Project' started by DevHackr, May 31, 2011.

  1. DevHackr

    DevHackr Enthusiastic Member

    Joined:
    Mar 10, 2009
    Messages:
    572
    Likes Received:
    85
    I have been going through these archives in an effort to preserve them. One thing I noticed was that the files are considerably bloated. This a mix of things. File formats that are outdated, older versions of photoshop, duplicates of the same files, and a healthy amount of junk.

    I began with 41.9gb worth of materials.

    #1 first step was combining all these torrents onto one folder called "Acclaim Archives"
    #2 Running a scan for exact duplicates of the same file. Same name/size I removed any duplicates
    #3 removal of the garbage 0kb files and empty directories.
    #4 converting the bloated and outdated TIFF format images to comparable PNG files of the same quality.
    #5 Updating the decade old photoshop files to newer, better, more compressed photoshop formats of today.
    #6 updating the outdated quark QXD files to the newer quark QXP format. No loss, better compression.
    #7 updated the outdated video files to current day standards.

    The work isn't done as of yet, but I have managed to compress 41.9GB down to 26.7GB. I still have more stuff to update, it's not nearly finished.

    I still need to:
    #1 finish the PSD file updates
    #2 finish updating the QXDs to QXPs
    #3 check the text and document files to see what can be shrunk
    #4 7z the entire thing and see how small I can get it. I'll keep you updated as I get further or finish.
     
    Last edited by a moderator: Feb 27, 2012
  2. bobzee

    bobzee [undefined]

    Joined:
    Jan 21, 2008
    Messages:
    338
    Likes Received:
    2
    There's something strangely addictive about compressing stuff. Good work.

    [​IMG]
     
  3. APE

    APE Site Supporter 2015

    Joined:
    Dec 5, 2005
    Messages:
    6,416
    Likes Received:
    138
    Not bad. Not bad at all.

    IMO the originals should be preserved for posterity, but for general consumption this should be more than adequate.
     
  4. DevHackr

    DevHackr Enthusiastic Member

    Joined:
    Mar 10, 2009
    Messages:
    572
    Likes Received:
    85
    Yes, I agree. Sadly my biggest failure is the fact that the dates of the newer files does not correctly represent the original documents. (except the tiff2png which the program matched the dates)

    I was a bit disappointed with the contents of these torrents. It was almost entirely strategy guide documents
     
  5. chalmo

    chalmo Spirited Member

    Joined:
    Mar 26, 2008
    Messages:
    189
    Likes Received:
    1
    When my brother first introduced me to PKZip back in the early 90s, I thought I was a genius by thinking of rezipping files over and over again until they were a couple of bytes. Yeah, I'm not the brightest.
     
  6. n64coder

    n64coder Robust Member

    Joined:
    Mar 25, 2009
    Messages:
    248
    Likes Received:
    1
    Could you really compress files over and over with pkzip until they're a few bytes? I didn't think that would be possible. I would think subsequent compressions would yield roughly the same or larger size.

    I guess there was a bug with pkzip or it used some sort of lossy compression algorithm.
     
  7. APE

    APE Site Supporter 2015

    Joined:
    Dec 5, 2005
    Messages:
    6,416
    Likes Received:
    138
    No you really can't. I can't claim to understand how the compression algorithm works but its my understanding that there is a small amount of overhead necessary to be able to take the compressed data and decompress it. If you compress something to its maximum potential compressing it again will just add overhead to it thus increasing the size of the resulting file.

    Not by much mind you, but if you take a tarball, 7zip it, RAR it, then ZIP it again you'll have a file larger than the original tarball.
     
  8. 7Force

    7Force Guardian of the Forum

    Joined:
    Mar 6, 2009
    Messages:
    4,547
    Likes Received:
    92
    And if you zip/7zip/rar/whatever something that's already been compressed with another method, like an Xvid video or something, the file will remain roughly the same size.
     
  9. syntax error

    syntax error Spirited Member

    Joined:
    Jan 29, 2011
    Messages:
    108
    Likes Received:
    0
    Do the have an ancient version of the Photoshop plugin Genuine Fractals on tape?
     
  10. DevHackr

    DevHackr Enthusiastic Member

    Joined:
    Mar 10, 2009
    Messages:
    572
    Likes Received:
    85
    Outcome after a few weeks of puttering 7.89gb in 81 97.5mb rar pieces
     
  11. DevHackr

    DevHackr Enthusiastic Member

    Joined:
    Mar 10, 2009
    Messages:
    572
    Likes Received:
    85
    7.49gb after a few weeks, and after rarin' them.

    So I shaved off 34.41 GB off the total archives. A little easier to distribute now
     
  12. Druidic teacher

    Druidic teacher Officer at Arms

    Joined:
    Jun 6, 2006
    Messages:
    3,643
    Likes Received:
    129
    x
     
    Last edited: Jun 22, 2017
  13. DevHackr

    DevHackr Enthusiastic Member

    Joined:
    Mar 10, 2009
    Messages:
    572
    Likes Received:
    85
    It would have been excessively difficult to make the video fies WORSE. They were 320/240 grainy pieces of crap. I did take care to make sure they had the same colors, framerates, audio, etc. However they're not bloated late 90's quicktime anymore
     
  14. Melchior

    Melchior Rapidly Rising Member

    Joined:
    Jun 12, 2011
    Messages:
    79
    Likes Received:
    3
    Be careful with the TIFF files!

    A file can contain more than one image, many utilities DO NOT COMPREHEND THIS!

    So you may be losing extra 'pages' of images...

    Blindly compressing and converting this is a good way to loose data. (If anyone cares)
     
  15. r34per

    r34per Rising Member

    Joined:
    Mar 16, 2011
    Messages:
    56
    Likes Received:
    0
    whats the status of this? :)
     
  16. Master13

    Master13 Spirited Member

    Joined:
    Apr 24, 2010
    Messages:
    159
    Likes Received:
    2
    I also am wondering when this version will be released? That is a major drop and could save a bunch of time Thanks for your work Devhackr
     
  17. ASSEMbler

    ASSEMbler Administrator Staff Member

    Joined:
    Mar 13, 2004
    Messages:
    19,394
    Likes Received:
    995
    I would like to replace the torrents with the streamlined version.
     
  18. Master13

    Master13 Spirited Member

    Joined:
    Apr 24, 2010
    Messages:
    159
    Likes Received:
    2
    Status? I'm still looking forward to a release.
     
  19. CodeAsm

    CodeAsm ohci_write: Bad offset 30

    Joined:
    Dec 22, 2010
    Messages:
    1,500
    Likes Received:
    176
    I stick with my originals, and believe they should stay acceseble.
     
  20. Master13

    Master13 Spirited Member

    Joined:
    Apr 24, 2010
    Messages:
    159
    Likes Received:
    2
    I know its just I don't have enough room on my hard drive right now for all of this
     
sonicdude10
Draft saved Draft deleted
Insert every image as a...
  1.  0%

Share This Page