hpr-knowledge-base/hpr_transcripts/hpr2996.txt

Episode: 2996
Title: HPR2996: Spideroak Update
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr2996/hpr2996.mp3
Transcribed: 2025-10-24 14:37:22

---

This is Hacker Public Radio Episode 2996 for Monday, 27 January 2020.
Today's show is entitled Spider Oak Update. It is hosted by Operator
and is about eight minutes long
and carries an explicit flag. The summer is.
I give you an update on my cloud backup solution and fixes.
This episode of HPR is brought to you by archive.org.
Support universal access to all knowledge
by heading over to archive.org forward slash donate.
Hello everyone and welcome to another episode of Hacker Public Radio.
This one is going to be over the Spider Oak backup software
that I'm using for my media music and other stuff.
I've been using it here for two or three years now.
I might have done an episode on backups.
I think I did one if you want to look back on board backup,
which is used for incremental backups and Linux and in Windows.
They have a Windows sort of hack job. I don't know if they've improved that,
but I don't have a board running on any Windows boxes.
I use it to backup my root partition and then I back that up onto Spider Oak.
I noticed I had placed some new music from Bandcamp
and that wasn't getting backed up to Spider Oak.
I went and looked around and I had some permission issues
on some of the root folders when I'm backing up my root.
I have to set the permissions after I'm done backing up root
to the right permissions so that when those backups get written,
the backup software under the limited user or the normal user
can access those files to back them up.
I'm thinking that's what was happening.
I was kind of in a stale state and it was trying to access these files
that it couldn't access and kind of just drooling all over itself.
When in actuality, I think what happened here several weeks later
is I figured out kind of the root cause.
I had moved some backup surround and I had changed path locations and stuff
and there's a configuration file called I think it's config.ini basically.
I'm guessing it defines what's going to be backed up in Spider Oak
and you can modify this file,
but it actually doesn't do anything when you modify the file directly
from what I can tell.
This config.text file gets generated through the UI
and there might be a way to do this through the command line,
but I essentially had to use the UI,
remove the old path and use the new path for the same backup.
I had a folder called MoreData,
which was a different dried and that got moved to two of those folders
where in a folder called MoreData
and then that got moved to a different mount point on a different dried called backup.
So I was taking my backup drives instead of having three backups.
I used to have a local backup and then I'll take that backup and sync it to the cloud.
Now I just have a backup that gets sync to the cloud.
So if my backup drive craps out,
I only have one other place and that's in the cloud.
So I just have to ensure that my backup drive actually gets backed up to the cloud
and that's what I was doing here.
This config.text file with spider oak gets updated through the UI at best
and I had to remove the old paths
and there's all these warnings that says don't allow this to be backed up in the future
and I'm not too clear as to what that's going to exclude this from backups
and I don't know if that means that path or the path and everything under it is unclear.
So a lot of the configuration for the spider oak thing is kind of unclear
and it's kind of weird. So I just kind of let it run and I check in a couple days
and if it hasn't done what it's supposed to do then I have to monkey around with it,
which is not something you want in your backup software.
And again, it's probably just limitations of my time to understand the UI
and the difference between the UI and the command line.
So the way I traditionally run it is dash dash headless dash dash verbose
and then I'll look at those logs while I'm adding new content
to see if they get backed up.
And there's automatic is the setting for the scheduling,
which I don't know what that means.
So I actually set the scheduling to like one in the morning for the backups
and after check after fixing the paths in the backup locations in the UI client,
then everything started to show up.
So all of a sudden I had added 22 gigs worth of stuff.
It wasn't there. I don't understand why.
And then I removed the old paths that I had moved to a different location
and I removed those from the backup software and I hit save and then I hit run.
And then all of a sudden 22 gigs ends up in the folder that I had specified.
So it's almost like I had a folder the same exact name and it was keeping the old name
or is keeping the old metadata in that folder.
So I had a folder called like backup dump.
And the backup dump folder was not getting updated on the website or the spider oxide
because it was like two of the same folders with the same name with the same content in it.
So when I got rid of the more data path, I reran it, hit run and synced everything up
and then boom 22 gigs instantly shows up in the UI.
There's some other options you can run when you start running out of this space.
There's a dash dash empty dash garbage dash bin and a dash dash purge dash historical dash virgins
which will help clean up some space.
Initially when I hit the sync button, I got warnings from the UI saying I had negative 88 gigs of space
out of my two terabytes, which in actuality it ended up being right at around.
I want to say 1.4 terabytes or should be 1.2 terabytes, something like that.
So that was why I'm not sure what was happening there.
But I'm thinking it was adding up all of the new additional stuff
that it hadn't already verified that it was already uploaded.
So when you upload something, somehow it knows, even though it's encrypted,
somehow it knows that with some kind of metadata analysis, it knows that those encrypted files were already uploaded once.
We're already uploaded on the system.
So there's like some local caching that gets done and then that local database gets fed to something and says,
okay, this test check matches, this test check.
So air go, these two files have been, I've already been uploaded, something to that effect I can assume.
I'm just making it up.
That's pretty much it.
So I run the backup job that will download my web server stuff
and it will download my web server back up to my web server
and then it will run board backup and then that will all get synced to spider up.
So it feels like now and we'll check here.
In a couple of months, it feels like that it's doing what it's supposed to be doing.
But then again, this software, I have to touch a file
and then make sure that that file gets updated within the next couple of days.
I'll touch a file and then check back on it and make sure that it's there
and then I'll go back and look at a specific path or something
and make sure that that path matches the path that I've got within a recent change.
But I think my problem was that I had had those paths duplicated in a different older path
that didn't exist anymore, so spider up was all confused.
But I think I fix all that now.
Anyways, that's pretty much it.
It's kind of a quick tip, so it's only under 10 minutes.
Let me know if you have any questions about board backup or spider oak and go from there.
You've been listening to HackerPublicRadio at HackerPublicRadio.org.
We are a community podcast network that releases shows every weekday, Monday through Friday.
Today's show, like all our shows, was contributed by an HBR listener like yourself.
If you ever thought of recording a podcast and click on our contributing
to find out how easy it really is.
HackerPublicRadio was founded by the digital dog pound and the infonomicon computer club
and is part of the binary revolution at binrev.com.
If you have comments on today's show, please email the host directly,
leave a comment on the website or record a follow-up episode yourself.
Unless otherwise status, today's show is released under a creative comments,
attribution, share a like, 3.0 license.