CS 470: Unix/Linux Sysadmin Summer 2021 Lab 7 public cloud: Microsoft Azure

This lab is to get you familiar with the concept of public cloud, while introducing you to SSH tunneling, VPNs, and GitLab to set up a source repository. Amazon didn’t reply in time for this lab to be graded, so we’ve fired them and replaced them with Azure, where you can get $100 of free credits for having a .edu e-mail address without the pain of waiting for worthless bureaucrats at the university or at the retail-killer in Seattle to

CS 470: Unix/Linux Sysadmin
Summer 2021 Lab 7
public cloud: Microsoft Azure
This lab is to get you familiar with the concept of public cloud, while introducing you to SSH tunneling,
VPNs, and GitLab to set up a source repository. Amazon didn’t reply in time for this lab to be graded,
so we’ve fired them and replaced them with Azure, where you can get $100 of free credits for having
a .edu e-mail address without the pain of waiting for worthless bureaucrats at the university or at the
retail-killer in Seattle to
The modern cloud really started with Amazon, and with AWS, which began as a way for Amazon to
manage its vast internet commerce site behind the scenes. In order to become the proverbial “everything store” it’s become today, web services had to become separated from the actual “data warehouses” of product information behind them, and these in turn separated from the payment systems
… and all of these services, for all of those customers, had to become fault-tolerant, expendable,
clonable, and scalable in a distributed fleet across multiple fault-tolerant geographical zones … in fact,
the first public brand name for what is now known as “AWS” was “EC2,” short for “elastic computing
cloud,” to emphasize the scalability services behind it. When one subset of services needed more
computing power, memory, storage … there became an API for that … and rapidly, with the
knowledge you’ve developed if you didn’t have it already, you can see the budding beginnings of a
first public cloud offering underpinning the various needs of running the largest e-commerce site on
the internet.
Microsoft, like in a lot of things, was not the first to market, but didn’t want to be left out of the party
… and in typical Microsoft fashion, Azure is a few years behind its competition, didn’t get it right for
the first few versions, and still has lots of catching up to do.
You have limited time and “credits” available on here, while you’re learning … so please don’t idle on
the website, especially with a VM running. The more you use, the less credits you have left. Continue.
Enough cloud history, though. Let’s do some stuff.
part 0: get free stuff
1. First, let’s sign up for a free $100 of Microsoft Azure credits. Go to the following URL …
https://azure.microsoft.com/en-us/free/students/
… and click on the “start free” button. Microsoft will start interrogating you about your eligibility and your status as a student. As such, it’s really important to use your @sdsu.edu email address.
Auditors, you can use your cssc*@edoras.sdsu.edu account as an e-mail address to sign up
here … but before you do, create a ~/.forward file on EDORAS with vi and put your real email address you want mail to your EDORAS account to forward to.
1
They will ask you for your phone number, to further validate you with a phone number, both
for multi-factor authentication, and so they have somebody to call before they essentially
lease you a virtual server in their data center. This is primarily intended to deter spammers,
who plagued the early cloud by exploiting trial offers like this one, and using cloud trial credits
to fill all our inboxes with advertisements.
Enter a valid phone number, because they will send a verification code. Yes, Microsoft is evil,
but this free is still free.
Once you have been successfully verified as a student, you will be greeted by this pleasant
Windows 10-looking screen, and when you click on “Overview” in the left-hand side toolbar,
you should be shown that you have $100 in Azure credits, good for a year.
2. This step is optional, but extremely useful to take advantage of while you’re still in school and
can get free loot: sign up for the Github Student Pack at the following URL …
https://education.github.com/benefits?type=student
This will also ask link your school email (.edu) to your existing Github account to verify that
you’re a student. Github Student Pack also provides another way to get at $100 of Azure credits, so it ties in with this lab.
2
part 1: set up a VM
Generally, there’s a point-and-click way to do everything inside Azure, and there’s an API for everything, so you can do it with the command line, or with code, in what we now call “Infrastructure as a
service” (IaaS … remember our thing about as-a-services before?).
We’re just going to do it the easy way, via the web interface, because we’ve got a lot to cover.
3. Click on the three lines menu in the upper left-hand corner of the Azure screen and select
“Virtual Machines.” Then click “Create” to create a new VM.
Choose “Ubuntu Server 20.04 LTS – Gen 1” as your operating system, and use the following
settings:
Virtual Machine Name: gitlab
Region: choose whatever is closest to you, something US West for most of us
Availability Options: No infrastructure redundancy required.
Size: Standard_D2s_v3 – 2 vcpus, 8 GiB memory
Authentication type: SSH public key
Username: your username used for previous labs
SSH public key source: Generate new key pair
Key pair name: azure_gitlab
Public inbound ports: Allow selected ports
Select inbound ports: click on the dropdown, check SSH (22) and HTTPS (443)
3
Now click on “Next : Disks” at the bottom. Don’t create the VM yet!
!! IMPORTANT NOTE: your VM is billed hourly, and you must shut down your VM at the end of
this lab to avoid draining your free credits. Don’t worry, you won’t be billed $70 right away.
You’re billed a proorated amount for every hour your VM is running.
4. On the “Disks” screen, select “Standard SSD.” We’re saving a few credits here. Leave the rest
untouched. Then click on “Next : Networking.”
5. On the “Networking” screen, click “Create new” under “Public IP,” then in the pane that comes
in from the right, select a static assignment. This may cost a couple more credits, but is well
worth it if this were to be a real service VM.
Then click on “Next : Management.”
6. Microsoft Azure has thoughtfully provided us with an option to auto-shutdown VMs to avoid
draining our credits, just in case you forget. This is an important fail-safe mechanism. Turn on
“Enable auto-shutdown,” then change the time zone to (UTC-08:00) Pacific Time if you live in
California, or adjust accordingly to your time zone. Also note: make sure the shutdown time is
far enough from your current time, so your VM doesn’t decide to shut down in the middle of
you doing this lab.
7. Click on “Next : Advanced,” go ahead and look around, and leave everything untouched.
Then click on “Next : Tags,” read about them, and leave it untouched.
8. Click on “Next : Review + create,” and pay attention to pricing. This is what I got …
… so about a dime per hour. If this lab takes you 2 hours to complete, you’ll be billed about 20
cents, provided that you remember to shut it down after you’re done.
Check all your settings one more time, and then click on “Create.” Make sure to download
your new private key in the dialog that pops up … you cannot download it later, and if you
misplace it you will have to re-key your VM via the resource group UI.
4
Azure will tell you the deployment is in progress, and when it’s done, it will tell you that your
deployment is complete. Note the relative beauty of Microsoft’s Azure interface. Microsoft
isn’t typically known for aesthetics … but when and if you try Amazon’s AWS, you’ll see what
we mean.
Click on “Go to resource,” and in the resource screen, note your VM’s public IP address.
part 2: configuring your instance
9. First, of course, let’s set up DNS for our new system. As we did at the end of lab 2, add a
lookup to your host computer’s hosts file for your Azure VM’s IP address, and the hostname
gitlab.cs470.local.
If you’re using Windows and WSL, you’ll need to restart your WSL instance to have it
regenerate your hosts file in WSL. Close all WSL windows, and then run the following
command in a command prompt or PowerShell window:
wsl.exe –shutdown
5
10. In the name server from lab one, on OpenBSD, create a forward lookup to gitlab.cs470.local
for your Azure VM’s IP address, and make sure to increment the serial number of your
cs470.local zone file before restarting your name server. Make sure the lookup works.
11. Go to wherever you saved your SSH key file (likely your Downloads folder). It will have a .pem
extension.
In case you haven’t noticed already, WSL mounts your Windows operating system drive on
/mnt/c … so if you’re on Windows, cd /mnt/c using Ubuntu in WSL. Now you can view (and
modify) your Windows directories and files through the Ubuntu shell … cool, huh? For me,
when in Windows and WSL, I use the command …
cd /mnt/c/Users/peter/Downloads
… to get to my downloads folder. Of course, you’ll insert whatever your username is in place
of mine to get there on your system.
Those of you using Macs or Linux, will have a folder called Downloads in your home directory,
and remember, the tilde character (~) is a shortcut for the path to your home directory, and
your .ssh folder is at ~/.ssh.
Whichever operating system you’re on, once you’re there, move the key file (the one ending
in .pem) into your .ssh directory, and give it the name ~/.ssh/id_azure …
12. cd into your ~/.ssh directory and change the permissions of your key file to be read-only for
the owner only, with no permissions for anyone else. This is important for SSH to not complain loudly at you about how you store your key material. You’ll be using chmod to do this, of
course … and you need to figure out the octal permissions “number” or proper switches to
use, but should you either have the skills to figure this out by now, or you’re leeching off your
friends, and should knock it off.
13. Note that you logged in using only an SSH key. Also note, this SSH key had no password to
protect it. So, if that keyfile got out … so would access to your instance. Let’s fix that.
$ ssh-keygen -p
ssh-keygen will ask you to identify the file, and to choose a password. If you haven’t made
sure that your permissions are appropriate for crypto key material, ssh-keygen is going to po-
litely have a fit.
14. Let’s copy over our SSH public key into the authorized_keys keyring file on our Azure VM.
Mika not only was the chief author of this lab, but also found a command the other night for
doing just this …
$ ssh-copy-id -f -i ~/.ssh/id_rsa -o ‘IdentityFile ~/.ssh/id_azure’
gitlab.cs470.local
6
You can ignore most of the output. If you did everything correctly, you should now be able to
log into your new Azure instance with your
15. As in step #13 from lab 4, let’s take a couple minutes to install packages used later by your instructor for grading purposes: csh (via tcsh), GNU binutils, and net-tools.
$ sudo apt update
$ sudo apt install tcsh binutils net-tools gcc
part 3: GitLab
We’re going to install GitLab on the Azure VM to serve as our own git repository manager. Think of it
like hosting our version of GitHub. We’re using GitLab, though, not GitHub … which not coincidentally
is owned by Microsoft, who uses it to spy on our coding and sell us the results.
https://copilot.github.com/
If you’ve ever used git or Github, you’ve only worked with individual repositories. GitLab is a comprehensive repository management system that will manage multiple repositories … and documenting its use is outside the scope of this exercise, but you are encouraged to look into it for sure, as we
chose our examples here to be utilitarian to your future endeavors …
https://docs.gitlab.com/
… and GitLab comes both in an Enterprise Edition (“EE”) with lots of features for business and professional support, and a Community Edition (“CE”) that comes free of charge, and with only community
support. We are, of course, going to use the free CE.
16. Let’s install GitLab. First, let’s get the initial setup script …
$ curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlabce/script.deb.sh | sudo bash
… as you may have noticed …
The repository is setup! You can now install packages.
… the initial setup script is merely adding repositories to apt for us to be able to grab more
software, directly from the people at GitLab. With that done, we can now install GitLab … except for one thing.
17. Log back in to your Azure instance, and at long last, let’s install GitLab …
$ sudo EXTERNAL_URL=http://gitlab.cs470.local apt -y install gitlab-ce
7
Gitlab should be running at this point. How can you tell? Well, we went over it in class so you
should be able to find out.
If it’s not running look in /var/log/gitlab …
Next, we need to access the GitLab launch webpage. If we’re on our private software-defined
network inside VMware on each of our computers, we’re a whole lot less concerned about encrypting connections with important data, like passwords … so at every other juncture thus
far, we’ve took shortcuts and cut corners. This time, however, we’re going over the internet,
and we’re going to do this The Right Way™.
18. We could use more SSH port forward to provide encryption here, but then we’d want to access HTTP content under URLs at gitlab.cs470.local … both on its public IP, and via a loopback
adapter SSH port forward … but that would be two different IP addresses for different services
on the same hostname, and this not a tenable configuration.
The Right Way™ is to use HTTPS, instead of HTTP … in order to do this, we’ll need an SSL/TLS
certificate for the hostname gitlab.cs470.local. In a real-world scenario, we’d have real, publicly-usable domain name and would get a certificate from a third-party certificate provider,
such as Let’s Encrypt. For the purposes of this exercise, we’ll be creating a self-signed certificate, and we’ll get plenty of warnings about our self-signed certificate, and they are mostly
correct … but what we are doing is much better than just ignoring transport encryption like
we’ve done before.
You will, however, rapidly notice why we skipped it previously, on our private network.
19. Edit /etc/gitlab/gitlab.rb file and change external_url to have https instead of http in
the protocol portion of the URL.
20. Next, make a directory for certificates for GitLab.
$ sudo mkdir -m 700 /etc/gitlab/ssl
21. Now, let’s make an RSA key for our certificate…
$ sudo openssl genrsa -out /etc/gitlab/ssl/gitlab.cs470.local.key 4096
22. Now, let’s generate a certificate request (often called a “CSR”), with the above key.
$ sudo openssl req -new -key /etc/gitlab/ssl/gitlab.cs470.local.key -out
/etc/gitlab/ssl/gitlab.cs470.local.csr
This command will prompt you for some info.
You are about to be asked to enter information that will be incorporated
into your certificate request.
8
What you are about to
There are quite a few
For some fields there
If you enter ‘.’, the
enter is what is called a Distinguished Name or a DN.
fields but you can leave some blank
will be a default value,
field will be left blank.
Fill out each field as you see fit … in real life, you would want all this data to be acccurance, but
for the purposes of this lab, the only one here that really cannot change is the hostname.
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:Californistan
Locality Name (eg, city) [Default City]:Sandy Eggo
Organization Name (eg, company) [Default Company Ltd]:CS470
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server’s hostname) []:gitlab.cs470.local
Email Address []:de_la_terrible@sdsu.edu
I strongly recommend you not configure a challenge password.
Please enter the following ‘extra’ attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Now, let’s generate a self-signed certificate, to be saved in the x509 format, valid for one year,
signed with the key we generated earlier, in the file /etc/gitlab/ssl/gitlab.crt …
$ sudo openssl x509 -req -days 365 -in /etc/gitlab/ssl/gitlab.cs470.local.csr
-signkey /etc/gitlab/ssl/gitlab.cs470.local.key -out
/etc/gitlab/ssl/gitlab.cs470.local.crt
IMPORTANT: The above command is line-wrapped here in text processing, and should all be
on a single line.
Since we’re self-signing it, no browser, nobody is going to trust this certificate … except us, because we know that we generated it, and when our browser warns us about it soon, we can
verify that it’s ours, and agree to accept the risk of using self-signed, unattested-to encryption
materials.
You should see something like this in your certificate directory when you’re done …
$ sudo ls -l
total 12
-rw-r–r– 1
-rw-r–r– 1
-rw——- 1
/etc/gitlab/ssl/
root root 2021 Aug 12 05:51 gitlab.cs470.local.crt
root root 1752 Aug 12 05:50 gitlab.cs470.local.csr
root root 3243 Aug 12 05:49 gitlab.cs470.local.key
9
23. Now let’s tell GitLab to reconfigure itself for HTTPS, and reload its settings
$ sudo gitlab-ctl reconfigure
If you got it right, you should see a listener active on port 443 with netstat …
% netstat -an | grep 443
tcp
0
0 0.0.0.0:443
0.0.0.0:*
LISTEN
… if you didn’t get it right, look in /var/log/gitlab for error messages.
For me during testing, I had to use gitlab-ctl to stop and start the services to get port 443
to come up instead of 80.
24. Now, open up a web browser, and go to https://gitlab.cs470.local … you should get a certificate or security warning; this is expected. On Safari, it looks like this, and you should “show
details” and then click “visit this website” …
… on Firefox, it looks like this, and you should click “advanced,” then to “accept the risk and
continue” …
10
… if you’re using any other browser, unless it’s something obscure like Brave, Epic, or Opera,
you’re using spyware, and it’s we don’t support malware here.
!! IMPORTANT NOTE: all browsers give you the option of inspecting the certificate, to make
sure it’s the one you set up, and that nobody is snooping your connection.
Once you tell your browser to trust the self-signed certificate, bam! You should have a login
splash for GitLab, asking you to change your password. This is the password for the root GitLab
account, so set one up, and then sign in as root using that password you just made. Then, register a user using the web interface. You should receive an email from GitLab once you’ve
completed your user registration, or you’ve done something wrong.
IMPORTANT: If the webpage is saying it’s taking too long to respond, just refresh it. This means
you will not be prompted to change your root password.
You can find the default password in /etc/gitlab/initial_root_password. Copy it and
login in with the username root (You typically copy the space at the end too so watch out for
that). After you login go to your profile in the top right and click edit profile. Then go to password in the left sidebar and change it.
Congrats, GitLab is now set up.
25. Let’s set up your account in GitLab. Log in, and pull down the menu in the upper right-hand
corner down to preferences. You’ll find yourself at the user settings screen.
In the menu at left, choose “Account” and change your username to your regular username,
then hit “update username.”
Then, in the menu at left, choose “Password” and set a password you’ll remember, instead of
11
the random one you got from GitLab. You’ll have to log out and log back in.
26. Go back to the user settings screen again after you log back in, and select “SSH Keys” from the
menu at left. Use the following command …
$ cat ~/.ssh/id_rsa.pub
… to show your SSH public key, and then copy and paste it into the text entry box at right.
27. Pull down the GitLab menu from the top bar in your browser, and select to create a project.
Choose to create a new blank project. Name it hello_world, and check the box to initialize the
repository with a README file.
28. Once you’re inside the repository, click the blue “clone” button and copy the text to clone the
repository with SSH. Then, on the command line (from within any machine you like now), you
should be able to use that data to clone the repository you just created, and authenticate with
SSH …
$ git clone git@gitlab.cs470.local:peter/hello_world.git
12
29. cd into the hello_world folder and create a hello_world.c file to do everybody’s first programming project. The output of your program should be precisely …
Hello World!
… with a newline at the end. Once you have it working, remove any binaries, so that you only
have the README.md and hello_world.c and then mark all files in the current directory to be
added to the repository.
$ git add .
Then let’s set up our local git client to tag up our first commit properly.
$
$
$
$
git
git
git
git
config –global user.email “yourname@yourdomain.com”
config –global user.name “Your name or nickname here”
commit -am “my first commit”
push origin main
Yeah, that right … it’s main, not master. Thanks, snowflakes.
Open up the browser again and go to the repo again … voila! It’s there … and you’re done with
lab 7.
In future iterations of this lab, we’ll also be setting our VM the less-easy way, with code, and
exploring more of the ridiculous number of things you can do with Azure, and AWS.
13