Auto Archiver of Web URLs
A tool for automatically archiving web page links and saving the images and videos. Currently used by organisations monitoring human rights situations in Ukraine and Myanmar
Given a list of Web URL links entered into a Google Spreadsheet, we archive that page, so that you have a safe copy.
Our archiver is automatic and every minute looks to see if there are links which need to be processed. Below shows the archiver working on some Twitter URL's which contain images.
The results of the archiver for row 4 in the spreadsheet above are:
- Gets Images from the Link
- Save the image to the Archive Location either Google Drive or Digital Ocean Spaces
- Save a Screenshot
- Save a Hash of everything for veracity
- Create a Thumbnail of the image
- Create a Upload timestamp
1. Twitter Image
https://twitter.com/minmyatnaing13/status/1499415562937503751 is the entered Link
Screenshot we took of that Tweet and saved to our cloud storage in case it is every deleted from Twitter.
Image embedded in the Tweet which we have saved to our cloud storage
Raw twitter data, link to saved image (with hash). And this page is hashed and result populated in the spreadsheet
2. Facebook Video
https://www.facebook.com/Karenni-Free-Burma-Rangers-111193108038684/videos/343188674422293/ is the entered Link
Showing the successful archiving of the link
Automated screenshot of the Facebook page.
video.mkv is the raw video downloaded from Facebook.
Thumbnails of parts of the video so can see at a glance what happened.
Archivers
We have specific archivers for
- YouTube
- Telegram
- TikTok
There is a fallback archiver (which uses archive.org) for any other site.
Conclusion
This tool can save a huge amount of manual time archiving data from sites. It is being used by organisations capturing media for safeguarding.
We use the excellent https://github.com/bellingcat/auto-archiver code.
Please contact us here and we can do a demo for you