In this tutorial we'll install the GitLab Community Edition repository manager in a virtual machine, setup the continuous integration tool that comes with it (GitLab CI) and push through Git a minimal test suite from the host OS to see how a successful/failed commit works.
To follow this you need to be comfortable with the Bash *nix shell and to have an idea about version control systems (VCS) and continuous integration (CI) through behavior-driven and/or test-driven development.
There are other ways to install GitLab but I've done it in the recommended way and tried to automate as much as I could. The idea is to create in the end an unattended install through Vagrant provisioning.
Let's see where this goes through the next steps!
Step 1 - create Vagrant virtual machine
I'm using a Windows 7 host OS with Git Bash, here we go:
1. $ mkdir gitlab
2. $ cd gitlab
3. $ vagrant init ubuntu/xenial64
4. edit the generated 'Vagrantfile':
-> set the virtual machine RAM to 2GB minimum, I put 4GB for this test
config.vm.provider "virtualbox" do |vb| # # Display the VirtualBox GUI when booting the machine # vb.gui = true # # # Customize the amount of memory on the VM: vb.memory = "4096" end
-> add port forwarding for GitLab server - you can skip this step and leave port 80 if it's not in use by some other process on your localhost, if you do that don't forget to remove port 8080 from the next steps 🙂
config.vm.network "forwarded_port", guest: 80, host: 8080
5. $ vagrant up
6. $ vagrant ssh
Now we should be logged in the virtual machine and we can go to the next step.
Step 2 - add a swap file to our virtual machine
Even if you set more than 2GB of memory to the VM (4GB like I did) it is possible to run into 'out of memory' situations under heavy-load. Adding a 4GB swap file is a simple way to extend the total available memory.
1. create swap file:
$ sudo fallocate -l 4G /swapfile sudo: unable to resolve host ubuntu-xenial
If you encounter the annoying error above running sudo (don't worry, the command was executed even with this message!) you need to add the 'ubuntu-xenial' hostname to '/etc/hosts' for 127.0.0.1 IP:
a) $ sudo nano /etc/hosts
b) edit first line:
127.0.0.1 localhost ubuntu-xenial
c) save & exit
we can automate this in a one-liner (and we will see during this tutorial how important is this ability in *nix systems to automate everything!):
$ sudo sed -i 's/127.0.0.1 localhost/127.0.0.1 localhost ubuntu-xenial/' /etc/hosts
2. check if swap file was created:
$ ls -lh /swapfile -rw-r--r-- 1 root root 4.0G Jun 1 20:20 /swapfile
3. change permission to swap file to be accessible only by 'root' user:
$ sudo chmod 600 /swapfile
4. verify new permissions:
$ ls -lh /swapfile -rw------- 1 root root 4.0G Jun 1 20:21 /swapfile
OR (because we are not logged in as 'root'):
$ cat /swapfile cat: /swapfile: Permission denied
5. set swap file as Linux swap space:
$ sudo mkswap /swapfile Setting up swapspace version 1, size = 4 GiB (4294963200 bytes) no label, UUID=ff93a772-4f06-48e8-b8fe-bc0ae3a732fc
6. make swap available to system:
$ sudo swapon /swapfile
7. check if swap was enabled:
$ sudo swapon --show NAME TYPE SIZE USED PRIO /swapfile file 4G 0B -1
8. check total available system memory:
$ free -m -t total used free shared buff/cache available Mem: 3951 42 3694 5 215 3861 Swap: 4095 0 4095 Total: 8047 42 7790
9. add swap to system init:
a) we can do this the manual way
$ sudo nano /etc/fstab
append this line to the end of file:
/swapfile none swap sw 0 0
save & exit
b) we could do this without firing the nano editor (like a true 'enterprise automation junkie devops evangelist' - yes, such a funny job title really exists... 😛 )
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
Step 3 - install GitLab Community Edition
1. install Postfix and other useful packages (setting before the Postfix configuration so we can do an unattended setup):
$ sudo debconf-set-selections <<< "postfix postfix/mailname string localhost" $ sudo debconf-set-selections <<< "postfix postfix/main_mailer_type string 'Internet Site'" $ sudo apt-get install -y curl openssh-server ca-certificates postfix
2. run remote installation script of the GitLab CE repositories used by Ubuntu's 'apt-get install' command:
$ curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
3. $ sudo apt-get install gitlab-ce -y
4. $ sudo gitlab-ctl reconfigure
5. check that GitLab CE is installed and running with curl:
$ curl http://ubuntu-xenial <html><body>You are being <a href="http://ubuntu-xenial/users/sign_in">redirected</a>.</body></html>
It's OK, we can access GitLab from the virtual machine!
9. set GitLab external url as 'ubuntu-xenial' instead of 'localhost':
$ sudo sh -c "sed -i 's|external_url '\''http://localhost'\''|external_url '\''http://ubuntu-xenial'\''|' /etc/gitlab/gitlab.rb"
10. reload GitLab configuration:
$ sudo gitlab-ctl reconfigure
11. edit 'hosts' file for host operating system, add this line:
12. access in host OS:
13. set up 'root' password for GitLab - different from the 'root' user of the guest OS, for this example I've used something simple like 'rootgitlab':
$ sudo gitlab-rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=rootgitlab GITLAB_ROOT_EMAILfirstname.lastname@example.org force=yes
After this you need to reconfigure GitLab again with:
$ sudo gitlab-ctl reconfigure
14. sign in with user 'root' and password 'rootgitlab' and you will see the GitLab Admin Area homepage:
Step 4 - install GitLab Runner
To use continuous integration on your GitLab server you need to install a runner. This piece of software runs your tests and sends the results to GitLab. GitLab CI is the open-source continuous integration service included with GitLab that coordinates the testing.
1. $ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb.sh | sudo bash
a) set gitlab-ci coordinator url to http://ubuntu-xenial/ci
b) enter gitlab-ci token from host OS page http://ubuntu-xenial:8080/admin/runners
c) leave gitlab-ci description as 'ubuntu-xenial', press [ENTER]
d) add a gitlab-ci tag like 'runner'
e) finally select the executor, we will use 'shell' for this tutorial
After completing all data you will receive this message:
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
4. refreshing the Runners page of the GitLab Admin Area in your host OS should look like this:
Step 5 - set up a Git project
1. access from host OS http://ubuntu-xenial:8080/projects/new to add your new project.
Let's name it 'test' and choose 'Public' for visibility to keep things simple.
The project will be accesible by Git on http://ubuntu-xenial:8080/root/test.git (make sure that you use the correct forwarded port).
2. open a Git console in the host OS and clone the 'test' project above:
$ git clone http://ubuntu-xenial:8080/root/test.git project Cloning into 'project'... warning: You appear to have cloned an empty repository. Checking connectivity... done.
3. $ cd project
4. add a file using your favorite editor, for example 'test.txt' with content '123abc'
5. $ git add test.txt
6. $ git commit -m 'first commit'
7. $ git push -u origin master (enter user: 'root', password: 'rootgitlab')
Counting objects: 3, done. Writing objects: 100% (3/3), 222 bytes | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) To http://ubuntu-xenial:8080/root/test.git * [new branch] master -> master Branch master set up to track remote branch master from origin.
8. alright, let's check out first commit in the GitLab Admin Area on http://ubuntu-xenial:8080/root/test
Step 6 - setup Continous Integration
1. create a file named '.gitlab-ci.yml' in the root folder of the Git project created in the step above with content:
stages: - test test: script: - bash test.sh
2. create a bash script called 'test.sh' that will be used to test exit codes. First we will use a command that returns the exit status 0 which is for successful execution.
This is a simple and valid Linux command that will execute with no error.
3. add these 2 new files to Git:
$ git add .
4. commit files:
$ git commit -m 'continous integration - success' [master 12ad840] continous integration - success 2 files changed, 7 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 test.sh
5. push to GitLab server:
$ git push -u origin master Counting objects: 4, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (4/4), 378 bytes | 0 bytes/s, done. Total 4 (delta 0), reused 0 (delta 0) To http://ubuntu-xenial:8080/root/test.git 196d832..12ad840 master -> master Branch master set up to track remote branch master from origin.
6. we can see the successful commit at this address http://ubuntu-xenial:8080/root/test
8. commit 'test.sh':
$ git commit -a -m 'continous integration - fail' [master 6c858eb] continous integration - fail 1 file changed, 1 insertion(+), 1 deletion(-)
9. push to GitLab server:
$ git push -u origin master Counting objects: 3, done. Delta compression using up to 4 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 336 bytes | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) To http://ubuntu-xenial:8080/root/test.git 12ad840..6c858eb master -> master Branch master set up to track remote branch master from origin.
10. we can see the failed commit refreshing the http://ubuntu-xenial:8080/root/test page:
Step 7 - create Vagrant provisioning script
Vagrant comes with the powerful provisioning feature to automatically install software and make configurations in your virtual machine. In this step we will write a Bash script to install and setup GitLab as much as we can without user intervention (this is called an unattended installation).
Before we start this step we need to destroy the GitLab VM created above (don't worry it will be back up in short time) OR create a new folder and copy there the 'Vagrantfile'. 🙂
To destroy the VM run 'vagrant halt && vagrant destroy' in your host OS and confirm destruction.
1. create a file called 'gitlab.sh' in the same directory as your 'Vagrantfile' with this content:
#!/usr/bin/env bash # create swap file sudo sed -i 's/127.0.0.1 localhost/127.0.0.1 localhost ubuntu-xenial/' /etc/hosts sudo fallocate -l 4G /swapfile ls -lh /swapfile sudo chmod 600 /swapfile ls -lh /swapfile sudo mkswap /swapfile sudo swapon /swapfile sudo swapon --show free -m -t echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab # install GitLab sudo debconf-set-selections <<< "postfix postfix/mailname string localhost" sudo debconf-set-selections <<< "postfix postfix/main_mailer_type string 'Internet Site'" DEBIAN_FRONTEND=noninteractive sudo apt-get install -y curl openssh-server ca-certificates postfix curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash sudo apt-get install gitlab-ce -y sudo gitlab-ctl reconfigure sudo gitlab-rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=rootgitlab GITLAB_ROOT_EMAILemail@example.com force=yes sudo sh -c "sed -i 's|external_url '\''http://localhost'\''|external_url '\''http://ubuntu-xenial'\''|' /etc/gitlab/gitlab.rb" sudo gitlab-ctl reconfigure # install GitLab Runner curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb.sh | sudo bash sudo apt-get install -y gitlab-ci-multi-runner
2. setup provisioning in 'Vagrantfile', we can add it under the 'config.vm.box' setting:
config.vm.box = "ubuntu/xenial64" config.vm.provision :shell, path: "gitlab.sh"
4. $ vagrant up
5. access http://localhost:8080/ from host OS and login with user: 'root', password: 'rootgitlab'
access http://ubuntu-xenial:8080/ if you didn't forgot to setup the hostname in your host OS like I did when writing this tutorial... 😀
6. setup the GitLab runner settings using the correct token, see Step 4.3
7. create your project and go to Steps 5 & 6 in this tutorial to setup Git and continuous integration
The 'Vagrantfile' and related Bash script 'gitlab.sh' are available on my GitHub, you can get them with:
$ git clone https://github.com/mihailj/gitlabci-ubuntu1604.git
Then enter the project directory, run $ vagrant up and you're good to go!
Alright, this is the end of this tutorial!
I've tried to keep it very simple but the opportunities and advantages of using Continuous Integration are huge. We can add unit testing to every modern language and make sure that our code pushed to Git passes these tests with visual confirmation in the GitLab Admin Panel.