datahoarder

8166 readers
32 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 5 years ago
MODERATORS
126
 
 

What do you think of dual actuator hard drives? I never knew these even existed...

Here's a quick summary of the vid for those who want a TL;DW:

  • Dual actuator drives are a single drive with two actuator arms inside
  • These arms have their own platters, each with access to half of the drive's capacity
  • The SAS version shows up as two separate drives: one for each actuator
  • The SATA version shows up as a single drive, however can be partitioned at a specific LBA near the middle to use both actuators independently
  • Linux kernel updated to support these drives better when queuing commands
  • Capable of saturating a 5gbit SATA link

Personally, my concern is RAID setups, particularly in a SAS config. Will filesystems like ZFS and BTRFS know that two storage devices are the same physical drive... aside from that, and concern about more mechanical parts, this looks exciting especially for sequential speed throughput!

EDIT: fix typos

127
 
 

cross-posted from: https://feddit.uk/post/4478496

Veteran film collector John Franklin believes the answer is for the BBC to announce an immediate general amnesty on missing film footage.

This would reassure British amateur collectors that their private archives will not be confiscated if they come forward and that they will be safe from prosecution for having stored stolen BBC property, something several fear.

“Some of these collectors are terrified,” said Franklin, who knows the location of the two missing Doctor Who episodes, along with several other newly discovered TV treasures, including an episode of the The Basil Brush Show, the second to be unearthed this autumn. “We now need to catalogue and save the significant television shows that are out there. If we are not careful they will eventually be dumped again in house clearances, because a lot of the owners of these important collections are now in their 80s and are very wary,” he added.

Discarded TV film was secretly salvaged from bins and skips by staff and contractors who worked at the BBC between 1967 and 1978, when the corporation had a policy of throwing out old reels. And Hartnell’s Doctor Who episodes were far from the only ones to go. Many popular shows were lost and other Doctor Who adventures starring Patrick Troughton and Jon Pertwee were either jettisoned or erased. A missing early episode of the long-running sitcom Sykes, starring Eric Sykes and Hattie Jacques, has also been rediscovered in private hands in the last few weeks.

...

The BBC said it was ready to talk to anyone with lost episodes. “We welcome members of the public contacting us regarding programmes they believe are lost archive recordings, and are happy to work with them to restore lost or missing programmes to the BBC archives,” it said.

Whether this will be enough to prompt nervous collectors to come forward is doubtful. While collectors are in no real danger, the infamous arrest of comedian Bob Monkhouse in 1978 has not been forgotten, Franklin suspects: “Monkhouse was a private collector and was accused of pirating videos. He even had some of his archive seized. Sadly people still believe they could have their films confiscated.”

128
 
 

Apologies if this isn't the right community to ask in, I figured folks who use them would know, but if there is a better place to ask, please let me know!

So - I need to buy some external storage, and looking at prices and with the long run in mind, I was leaning towards a 18TB WD Elements or 16TB SEAGATE Expansion, but after looking at the reviews I am concerned about the noise levels, since I have sensory processing disorder, making me really sensitive to noise and this is going right by my bed and will hopefully function as an active backup, so will be always on.

Apparently there's a thing called Preventative Wear Levelling which will cause the drive to rev up every few seconds, and all drives (HDD anyway) do it, but it's the size of the drive that affects how much sound it makes, is that right?

If that's the case, is there a size drive where the noise becomes noticeable that I should stay under, or is it a case of trial and error and "my mileage may vary", and actually any drive could end up being noisy?

Alternatively, is there a quiet but affordable (in terms of ££ per TB, I wouldn't buy a significantly smaller SSD for the same price as those I mentioned, for example) alternative?

TIA

129
 
 

it would be great if the downloader isnt command line based and can list all the VODs of the channel at once after typing in the channel username/ link (but not too necessary). TYSM <3

P.S - I don't wanna watch but directly download the VOD (i am aware of the TwitchNoSub extension)

130
 
 

I've been seeding many Foss things for years but for some reason, people keep downloading Ubuntu versions that are more than 3 years old.

Any ideas why there is always someone downloading the ancient stuff, especially Ubuntu?

131
 
 

I bought a 15.36TB SSD SAMSUNG PM1633A SAS MZ-ILS15TA DELL EMC MZ1LS15THMLS-000D4

I am trying to figure out what to buy in order to connect it to my desktop PC via PCIE. Is this a viable or recommended solution?

SFF-8643 to SFF-8639 cable

Dell LSI 9311-8i 8-port Internal 12G SAS PCle x8 Host Bus RAID Adapter 3YDX4

132
 
 

I have an old computer that I use for storing and streaming my media. It has an attached external drive. I would like to increase my storage and build something that could be extensible to at least 100TB. I am not worried about backup.

I looked and I think I need a HDD rack or enclosure. Some people gave me links to good deals on ebay and some other sellers but they are based on the US and shipping fees are high. I saw this HDD enclosure and it seems to be what I am searching but I don't if they are good.

Do you have some advices for me?

133
 
 

I'm trying to archive all the images from a website, this one specifically: https://stevegallacci.com/archive/edf
However, when I use a tool like DownThemAll, it just pulls the thumbnails that link to the full image and not the image itself. Dunno if that's because I'm using the software wrong or that's just a limitation of DownThemAll. Is there any way to bulk download the full images without having to do so manually?

134
3
submitted 2 years ago* (last edited 2 years ago) by [email protected] to c/[email protected]
 
 

I want to setup a NAS (mainly for storing games and videos), that I'd also like to use to watch said videos on a WiFi TV and to install games on a separate PC connected via ethernet. This is the part list I came up with (plus whatever GPU I can get for as cheap as possible, I can probably get a ~~GT 730~~ GTX750 for free). I also don't need it to be on 24/7, if that's OK. I can place it in the same room as my main PC and hook it up to the same monitor to turn it on and start it up.

What's wrong with it?

PCPartPicker Part List

Type Item Price
CPU AMD Ryzen 3 3100 3.6 GHz Quad-Core Processor $50.00
Motherboard ASRock A520M-ITX/ac Mini ITX AM4 Motherboard $99.40
Memory Kingston Server Premier 8 GB (1 x 8 GB) DDR4-2666 CL19 Memory $36.00
Memory Kingston Server Premier 8 GB (1 x 8 GB) DDR4-2666 CL19 Memory $36.00
Storage Samsung 860 Evo 250 GB 2.5" Solid State Drive Purchased For $0.00
Storage Seagate IronWolf NAS 4 TB 3.5" 5400 RPM Internal Hard Drive $118.00
Storage Seagate IronWolf NAS 4 TB 3.5" 5400 RPM Internal Hard Drive $118.00
Video Card Gigabyte GV-N750OC-1GI GeForce GTX 750 1 GB Video Card Purchased For $0.00
Case Fractal Design Node 304 Mini ITX Tower Case $117.70
Power Supply be quiet! Pure Power 11 CM 400 W 80+ Gold Certified Semi-modular ATX Power Supply $58.10
Prices include shipping, taxes, rebates, and discounts
Total $633.20

PCPP says that R3 3100 isn't compatible with the RAM I picked (although I can't find why); it also says MoBo doesn't support ECC RAM, but on the producer's website it says it does (https://www.asrock.com/mb/AMD/A520M-ITXac/index.asp#Specification) , so I think PCPP is wrong.

I tried building around LGA 1150/1151 but motherboard prices are way higher (although CPU prices are lower).

I don't think I can make it much cheaper than this, since I'm buying everything, but if you can point me in a cheaper direction, feel free to do so!

Thanks in advance

135
136
 
 

Good afternoon all, I have half-assed my backups for 15 years, and it is not sustainable, and I need your help! I have the following setup: 1x Raspberry Pi 4 with a WD USB3 MyBook 4TB as a NAS target using OpenMediaVault. This works well enough, but is not in my mind a long term viable solution. 1x Apple Airport TimeCapsule A1355 2TB

I also have a smattering of other drives collected from over the years in MyBooks, all USB 2.0 drives, a 2TB mirror edition (2x 1TB drives in RAID 0 or RAID1), 1TB, and 500GB. This does not include the random 750 GBs, 500 GB and old 250 GB drives that I’ve taken out of my Macs and PCs over the years as I’ve upgraded them. I’ve got files scattered everywhere on them, plus on my MacBook and several other PCs and Macs around the house.

I need some help consolidating this into a single solution with priority to my photos and family home videos for data integrity. Then to a lesser extent, maybe PC backups and file storage.

Currently all of my photos are backed up to Google Photos or Amazon photos. With the fact that neither google or amazon is to be trusted with my photos, I’m ok with dumping them. Web based backup solutions are iffy, it takes forever for a backup to complete, as I am on a 60megabit download, with about a 5megabit upload connection. According to some things I’ve seen advertised nearby, fiber is being ran throughout the area, but it may be a year or two before it comes to my neighborhood.

For other hardware I have laying about, I have a 1st gen i7 980x system that is idle nowadays and is full of low capacity drives by today’s standards, a 2008 MacBook, the above mentioned 2012 MacBook Pro, an Atom n450 netbook, and an AMD Ryzen 5700g based prebuilt. None of them really seem to be something that would be useful as a ZFS based NAS or anything. But is a ZFS NAS or BTRFS system something that I need, or would my needs be better met by something else?

I have also looked at an OWC Mercurydisk M-Disc compatible burner for photo and video backup.

What are some options to look into? Preference would be on not breaking the bank and not necessarily set and forget it, but something I haven’t got to fight with to keep running.

137
138
 
 

Using Archive.org doesn't work on medium posts and ideally I want to archive every post. The blog I'm trying to archive is https://itsairborne.com in case the posts go down. Googling how to backup medium posts only gives me articles on how to do it if it were my blog. I found this extension called Monolith of Web that allows you to backup a website using the Rust tool Monolith and I just went to each article and clicked the extension and saved them all one by one

139
10
submitted 2 years ago* (last edited 2 years ago) by [email protected] to c/[email protected]
 
 

cross-posted from: https://l.antiope.link/post/43914

Hi all. I’m trying to choose a configuration for my home storage. Speed is not a priority, I want a balance of stability and performance. I was thinking of making a raid 6 array with an ext4 file system for 4 disks of 2 TB each. Asking for advice, will this configuration be optimal?

Note - I am going to make a raid array based on external usb drives, which I will plug into the orange pi

140
 
 

I'm having a hard time figuring out what case I want to get. Part of me thinks hot swap bays would be nice (I've had a drive failure and figuring out which one would have been 10x easier with hot swap). Of course in the future I'll have labels with S/N on the drives so it's easier to find the drive.

So provide me any case recommendations with 8+ drive bays if internal, and 6+ if hot swap. (I have a 5 drive pool now, but want to be semi future-proof).

141
 
 

I’m looking for a data archive of corporation ownership networks. For example, Alphabet owns Google, … and some metadata like when they are created/owned by Alphabet if possible. I was made aware of OpenCorporates but it doesn’t seem to have such data as far as I tried.

Apologies in advance if this is not an appropriate content for the community. I figured digital archivists may be aware of the existence of such archive. I couldn’t find a specific lemmy community solely for asking about data suggestions. If there’s a community better suited for this post, please let me know.

Thanks!

142
 
 

Google Books allows viewing the scans in colour, but when I click the option to download the PDF, I am provided only with a black-and-white version.

Is it known how to obtain the original colour images, outside of inspectelementing each page one by one?

143
11
submitted 2 years ago* (last edited 2 years ago) by [email protected] to c/[email protected]
 
 

I have 3 old SCSI HDDs that were in a hardware RAID, I don't have the RAID controller anymore but I have imaged them with DD and a SCSI PCI card I have.
Is there any way to assemble this array in software on Linux? I just want to get the data off so read only is fine.
Running blkid on the drive shows it as an Adaptec RAID member.
I believe the drives are in RAID 5.

EDIT: I got it working, but I had to use windows. I installed ReclaiMe Free RAID Recovery to find the RAID parameters then used the UFS explorer Pro free trial to image the array to a virtual disk. After a quick (actual quite long) chkdsk I managed to mount the NTFS file system on the array

EDIT2: There seem to be a lot of missing files, I don't think there was anything important on here anyway

EDIT3: wow, the found.000 folder is huge. I guess the recovery failed, or the array got pretty badly corrupted on the ~10 years in storage.

144
 
 

Anyone here use roboyoshi's datacurator-filetree? What do you think of it?

I write and record some my own music, some is personal and some is for my band. Under this filetree, where would it go in your opinion? In audio? Or in documents? Should I have all "band" media together, music, artwork, plans? etc? Should I have the band as a user on it's own and let it use it's own filetree?

145
 
 

In light of the recently announced price increase, I'm seriously considering moving all my backups from B2 to Storj as Storj is only charging $0.004/GB . As it's mostly just a backup, I don't really need the free egress and I travel a lot so not being tied to a single DC location is also appealing. What do you guys think? Anyone using Storj? What has your experience been like so far?

146
 
 

a TorrentFreak article got me spooked so I fired up the ol' yt-dlp. Got the entire channel, including comments, description metadata, and thumbnail images.

A significant number of videos were actually unavailable because of an odd YouTube bug where 15+ year old videos were listed as "currently being processed". I may re-run this later (since I ran it in archive file mode) to get the missing videos, as it seems there may be about 300 out of 4911 videos missing.

147
 
 

I'm asking because I have been looking for some videos about the game Metroid Prime Hunters, these are some online gameplay videos almost since the beginning of YouTube...

I only remember them in my mind, I don't even recall specific channel names.

Is there any like archive of YT videos? Or is this lost media now?

148
149
12
ZFS backup strategy (lemmy.sdfeu.org)
submitted 2 years ago* (last edited 2 years ago) by [email protected] to c/[email protected]
 
 

Hello,

I've been lately thinking about my backup strategy as I'm finalising building my NAS. I want to use ZFS and my idea was to have two drives in mirror (RAID-1) configuration and just execute periodical snapshots on such dataset. I want to the same thing in a second location, so in the end my files would be on 4 different drives in 2 different locations and protected by snapshots from deletion or any other unwanted modification.

Would be possible with this setup to just swap one of the drives in one location and have ZFS automatically rebuild data on the new drive and then I take the drive to second location and do the same so all drives would be exactly the same, instead of copying data manually? Though I believe all of the drives would need to be exactly the same size, is that right?

Is it a good idea in general or should I ditch it, or maybe just ditch the part with ZFS rebuilding and use instead some kind of software for that?

Thank you for your help in advance!

150
view more: ‹ prev next ›