Backup your PMs with wget before the site goes down

Discussion in 'Off Topic Discussion' started by -=FamilyGuy=-, Jun 2, 2019 at 3:38 PM.

Tags:
  1. -=FamilyGuy=-

    -=FamilyGuy=- Site Supporter 2049

    Joined:
    Mar 3, 2007
    Messages:
    3,031
    Likes Received:
    889
    You'll need to use the WGET program. It's available for Linux/OSX/Windows easily, just google it.

    1. Install Firefox and the export cookies addon: https://addons.mozilla.org/firefox/addon/export-cookies-txt/
    2. Log in into AG, making sure to tick "stay logged in". Use the addon to export the cookies for AG to a text file, mine is called "cookies-assemblergames-com.txt".
    3. Create a new folder, place the cookies file inside and open a console/terminal in the same folder.
    4. Run the following command (copy including the converssation URL):
      wget -mkEpnp --execute robots=off --load-cookies=cookies-assemblergames-com.txt https://assemblergames.com/conversations/
    5. WAIT!
    6. Rename the newly created "assemblergames.com" fodler to A.
    7. Run the following command (copy including the conversation URL):
      wget -mkEp --execute robots=off -I/attachments/ -I/data/ -I/conversations --load-cookies=cookies-assemblergames-com.txt https://assemblergames.com/conversations/
    8. Rename the folder "assemblergames.com" to B.
    9. Create a third folder called Final, copy the content of B to it.
    10. Copy the content of A over Final, overwriting/merging everything that was already there from B.
    11. Change the _bH variable to ./ (current dir) in every html files. In linux and probably OSX:
      find ./Final -type f -exec sed -i -e 's/_bH = "https:\/\/assemblergames\.com\/";/".\/"/g' {} \;
    12. Fix links of attachments:
      find ./Final/conversations/ -type f -exec sed -i -e 's/"https:\/\/assemblergames\.com\/attachments\//"\.\.\/\.\.\/attachments\//g' {} \;
    13. Profits $$$
    14. Like and subscribe! /jk
    You should now have an offline backup of your conversations/PMs in the Final folder. The main html file to open with your browser is Final/conversations/index.html

    Good luck!

    PS: My backup isn't done yet, so consider this untested. Use at your own risk.

    Edit: Some absolute links might have to be converted to local ones using external scripts afterwards, currently testing. All data is safe though, just inconvenient to browse.

    Everything should work pretty well now. When you click on an attachment, it'll open a basic file browsing page with an index.html file, that files is actually your attachment; right-click, save as.

    EDIT: Current bash script for Linux, should work under Windows via cygwin or on MacOS. To put in an empty folder along with cookies-assemblergames-com.txt and execute.
    Testing right now from a fresh folder with my own PMs.
    Code:
    #!/bin/bash
    
    wget -mkEpnp --execute robots=off --load-cookies=cookies-assemblergames-com.txt https://assemblergames.com/conversations/
    
    mv "assemblergames.com" A
    
    wget -mkEp --execute robots=off -I/attachments/ -I/data/ -I/conversations --load-cookies=cookies-assemblergames-com.txt https://assemblergames.com/conversations/
    
    mv "assemblergames.com" B
    
    mkdir Final
    
    cp -rf B Final
    
    cp -rf A Final
    
    find ./Final -type f -exec sed -i -e 's/_bH = "https:\/\/assemblergames\.com\/";/".\/"/g' {} \;
    
    find ./Final/conversations/ -type f -exec sed -i -e 's/"https:\/\/assemblergames\.com\/attachments\//"\.\.\/\.\.\/attachments\//g' {} \;
    
     
    Last edited: Jun 2, 2019 at 10:34 PM
    Denryu, pool7, Stipo360 and 3 others like this.
  2. -=FamilyGuy=-

    -=FamilyGuy=- Site Supporter 2049

    Joined:
    Mar 3, 2007
    Messages:
    3,031
    Likes Received:
    889
    There's a slight issue right now with absolute links, but I should be able to fix it using SED. Stay tuned.
    Fixed with FIND and SED, might need to adapt to your OS.

    Left to fix:
    • Some attachment might not get downloaded: Exploring wget options to try and fix this
      Should be fixed as of latest edit.
    Let me know if you find another issue.
     
    Last edited: Jun 2, 2019 at 10:09 PM
  3. SONIC3D

    SONIC3D Spirited Member

    Joined:
    Oct 30, 2008
    Messages:
    149
    Likes Received:
    33
  4. -=FamilyGuy=-

    -=FamilyGuy=- Site Supporter 2049

    Joined:
    Mar 3, 2007
    Messages:
    3,031
    Likes Received:
    889
    You're welcome.

    I just updated the procedure. It's a tad more complicated and completely unoptimised, but it also properly saves attachments.
    Let me know if anything is unclear.
     
    Last edited: Jun 2, 2019 at 10:17 PM
sonicdude10
Draft saved Draft deleted
Insert every image as a...
  1.  0%

Share This Page