Well, it happened again. I broke my server.
On Friday afternoon, I was working on someone’s site that is hosted on my server. I had PuTTY and WinSCP opened while I was working.
I remember months ago typing in “whereami” and found out that it’s a real package. At first instinct, I thought it was a command to state what directory you are in. So I decided to install it.
That was the mistake, installing whereami.
The install stopped at about 78% on my screen, and froze. All of sudden WinSCP disconnected but my SSH session was still up at the frozen install. At first, I thought that me indexing a folder killed it.
Luckily, my server is hosted via DigitalOcean. They have a nice little console window that works no matter what. Thank God.
After many hours that night figuring out what the fuck I did, I went to bed in defeat. I worked all that weekend so I had no time to investigate. I decided to create a support ticket to see what they can do since I had no idea.
They told me hours later to run some commands to make sure my network config is correct, and it was from what I could tell. The main issue at that point were a couple of things:
- All outbound traffic times out while trying to connect
- Anything trying to connect outside the server fails
- Since nothing could connect, nameservers died
So I investigated more on Sunday night and found a major issue with my config. The default gateway was set to 0.0.0.0! Ha! I found the issue!
I quickly set the gateway back to what it should be, and heaven’s light shined on me. All was working!
So RIP to the 5 websites and 1 Twitch bot that are all hosted on this server.
Turns out whereami is some networking setup package? I still don’t really know but it reset my entire network config.
I’ve always wanted to run some type of bot on some website. At first, I wanted to run a Reddit bot. However, simple reply bots aren’t that great so I decided against it.
So I found Tootbot. It’s a Twitter bot that mirrors hot posts from given subreddits.
So I got to work and decided to mirror posts from /r/FortniteBR because why not. Anyways, setup is very simple.
First I cloned the repository and installed the dependencies.
pip3 install -r requirements.txt
After that, I simply ran the bot using
tootbot.py. It asked for some API keys which were easy to retrieve. After that, the bot was running!
Now the only problem was trying to get it running 24/7. As you know, closing off the SSH window will stop the script. So I just used
nohup to run the script in the background for me.
So great, I have a Twitter bot now! It runs fine and you can change how often it posts, the threshold to post, etc.
If you want to check the bot out, you can follow @r_fortnitebr!
If you saw on my Twitter, I may have broken my website on Friday night. I lost file and SSH access to it. How did it happen?
This all started with me moving my main domain over to hyper.lol. I thought I could just change all configs to the new domain and it would work. Some things were broken obviously. So I thought the issue was with file and folder ownership. So I change the directory to the site and did this command:
sudo chown -R www-data:www-data /
To anyone that is stupid enough, don’t run that command above. It WILL break your computer/VPS. That command destroys the file/folder perms making your install break. My thought process was that the
/ was relative to where I currently was. That was not the case. At that point, I was unable to login at all. The server couldn’t accept my key.
I realized what I did after I typed the command. Luckily, I was still logged in via WinSCP. To fix it, I just went to my root directory and mass changed the ownership back to root. I had to leave it on overnight; however, that still didn’t work.
I ended up contacting my hosting’s support. After about 10 emails, we decided that these were the best steps to fix this:
- Support would mount a recovery tool. From there, I could access the files.
- Download everything I want.
- Support would unmount the recovery tool.
- I rebuild the server’s OS.
That is what we did, and now everything is working once again.
I also did change my domain over to hyper.lol, since I had the chance to rebuild everything.
TLDR, idiot tries to change file and folder perms but messes up the location on the command. Contacts support to fix.