From subversion to git with the help of gitlab

Here are the steps I recently followed to migrate a subversion repo into git:

	# Installation
	* See the restore instructions if you need to restore from backups
	* Make sure /srv/gitlab/ is completely removed in the host
	* Use the below ~/docker-compose.yml file. The gitlab root user takes effect *only* for brand new installations
	```
	gitlab:
	image: 'gitlab/gitlab-ce:8.7.0-ce.0'
	restart: always
	hostname: 'gitlab.sample.com'
	container_name: 'gitlab'
	environment:
	GITLAB_OMNIBUS_CONFIG: |
	external_url 'http://gitlab.sample.com:9090'
	gitlab_rails['gitlab_shell_ssh_port'] = 2222
	gitlab_rails['gitlab_email_enabled'] = true
	gitlab_rails['gitlab_email_from'] = 'gitlab@sample.com'
	gitlab_rails['gitlab_email_display_name'] = 'sample Gitlab'
	gitlab_rails['gitlab_email_reply_to'] = 'noreply@sample.com'
	gitlab_rails['smtp_enable'] = true
	gitlab_rails['smtp_address'] = "samplembx.krco.com"
	gitlab_rails['smtp_port'] = 25
	gitlab_rails['smtp_domain'] = "sample.com"
	gitlab_rails['smtp_authentication'] = "plain" #plain, login
	gitlab_rails['smtp_enable_starttls_auto'] = false
	gitlab_rails['backup_keep_time'] = 604800
	GITLAB_ROOT_EMAIL: 'ci@sample.com'
	ports:
	- '9090:9090'
	- '2222:22'
	volumes:
	- '/srv/gitlab/config:/etc/gitlab'
	- '/srv/gitlab/logs:/var/log/gitlab'
	- '/srv/gitlab/data:/var/opt/gitlab'
	```
	Note that we must use a specific version otherwise backup/restore would fail.
	# Image version upgrade
	Must be tested: Apparently just stoping, removing and pulling the new version will take care of upgrading the database.
	# Troubleshooting
	* To connect to local gitlab running docker container:
	```
	sudo docker exec -i -t gitlab /bin/bash
	```
	# Manual configuration
	Additional manual configurations are not possible when using docker so changing /etc/gitlab/gitlab.rb and then running the below will not work as expected:
	```
	gitlab-ctl reconfigure
	```
	Instead, use docker-compose.yml to introduce the changes and kill/start the container.
	# Project creation:
	* Better to be created under a specific group (like for example dev)
	* Should be private always and access granted user by user
	# git client configuration
	* Use Profile Settings | SSH Keys to add your public SSH key.
	* Command line instructions to work with git are available in each project landing page
	# Administration
	* To kill:
	```
	docker-compose kill gitlab
	```
	* To start:
	```
	docker-compose up -d gitlab
	```
	# Migrating from subversion
	First of all consider keeping for some months your old subversion in read only and start fresh with git. If that is a possibility (leaving without full history in git) you will save an outrageous amount of time in your conversion.
	Assuming you want a full copy, start by hitting your gitlab instance and:
	* Create group dev and your own user
	* Assign master role to your user
	* Assign your user to the group (members link on the left after selecting the group)
	* Create project in the group namespace
	On the client side:
	* I had to follow https://github.com/nirvdrum/svn2git/issues/50#issuecomment-14707504 to resolve git svn clone issues.
	* Other than taht just follow the below steps and make sure you run this from a machine that will not be off for some time or probably use nohup. It will take a while ...
	```
	git config --global user.name "Nestor Urquiza"
	git config --global user.email "nurquiza@sample.com"
	sudo apt-get install -y git-svn
	cat ~/.ssh/id_rsa.pub
	svn log -q http://subversion.sample.com/repos/reporting/projects/fe/ | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > ~/authors.txt
	# Warning: Better if you can update ~/authors.txt with proper email for users
	git svn clone --stdlayout --no-metadata -A ~/authors.txt http://subversion.sample.com/repos/reporting/projects/fe/
	cd fe
	git remote add gitlab ssh://git@gitlab.sample.com:2222/dev/sample-fe.git
	git push --set-upstream gitlab master
	git for-each-ref refs/remotes/tags | cut -d / -f 4- | grep -v @ | while read tagname; do git tag "$tagname" "tags/$tagname"; git branch -r -d "tags/$tagname"; done
	git for-each-ref refs/remotes | cut -d / -f 3- | grep -v @ | while read branchname; do git branch "$branchname" "refs/remotes/$branchname"; git branch -r -d "$branchname"; done
	git branch -d trunk
	git push gitlab --all
	git push gitlab --tags
	```
	# Add .gitignore based on svn:ignore property
	From a fresh copy of the subversion repo I used https://gist.github.com/iegik/3450385 to generate .gitignore:
	```
	svn propget -R svn:ignore | grep -v "^$" | sed "s/\(\(.*\) - \)\(.*\)/\2\/\3/g" | sort >> .gitignore
	```
	Clone the git repo and add the .gitignore file, commit and push back to the origina master.
	# Shrink the git repo
	If the repo is big this will take a long time. Once done clone your repo and run the below command. If your repo is big consider looking for a specific culprit like big binary files that were committed before (regardless if they were already deleted). If that is not the apparent cause of the problems then most likely history alone is the culprit. You will need to identify then full directories not in use anymore. Follow the steps from http://naleid.com/blog/2012/01/17/finding-and-purging-big-files-from-git-history to determine what can be eliminated. In our case history and not specific big files were the culprits. The below command for example calculates the total size being used by the webapp directory which was needed in old pure server side MVC application but not longer needed in new SPA based application:
	```
	cat bigtosmall.txt | grep webapp | awk '{total=total+$2} END{print total}'
	```
	Per the link eliminating these would require the below command:
	```
	git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch webapp' --prune-empty --tag-name-filter cat -- --all
	```
	Note that after running the command you need to deparately clone the git repo to confirm the size has shrunk.
	# Backup
	Backup is done via environment/production/scripts/tools.sample.com/tool_backup.sh
	# Restore
	To restore:
	* Put the backup file in /srv/gitlab/data/backups/ and run the below. It will import projects, users etc. You must not try to recover from a backup done with a different version of gitlab!:
	```
	sudo find /srv/gitlab/data/backups/ -name "*gitlab_backup.tar" | xargs sudo chmod 666
	sudo docker exec gitlab /bin/bash -c 'export force=yes && gitlab-rake gitlab:backup:restore'
	```
	# Upgrade
	* One more time: You must not try to recover from a backup done with a different version of gitlab! This means you need to test the new version in a separated server before. Shouldn't you always do this anyway?
	# Maintenance
	* To change password inside container
	```
	root@gitlab:/# gitlab-rails console production
	irb(main):001:0> user = User.where(id: 1).first
	irb(main):001:0> user.password = '*****'
	irb(main):001:0> user.password_confirmation = '******'
	irb(main):001:0> user.save
	```

view raw gitlab production installation: good bye subversion hosted with ❤ by GitHub