A repository template

I decided to do a little experiment for a mini service I am playing with at the moment : have more than just code in the repository.

More than code

A web app, service or backend usually has a few dependencies to work with. At the very least there is usually a database running somewhere on the host and storing its data locally. If you are only working with one project you might have one single instance of PostgreSQL running locally. If you are running projects for a few clients at the same time (a few small ones and a big one) chances are you are using multiple versions of one or more databases. That’s what I usually do.

So I end up with a docker-compose in each service repository with the right database engine and its version configured. A volume is used to store the data within the code repository folder (with a gitignore and dockerignore rule). So I end up with some data folder which are not really usable or related to the code within that repository. Of course it’s ignored when it matters but it’s kind of sitting there still.

$> ls
.
..
app/
lib/
pgdata/
public/
docker-compose.yml	
Dockerfile

And then, I would end up with separate repositories for plenty of things on the side of this code repository : one for the documentation, one for infrastructure as code, etc …

I started experimenting with a way to have multiple things in the same repository using the root as a place for common things and then having subdirectories for each service or part.

$> ls
.
..
code/
documentation/
pgdata/
docker-compose.yml

But the documentation was not very handy to have like this, as just a folder with markdown or asciidoc files. So I thought a Jekyll container could help with that. That’s one more service to declare in the docker-compose.ymlfile.

Linking up

Remembering every single TCP port to use for separate services isn’t practical. Instead I also added a ha-proxy container with a config file loaded from a volume. That way we can link up all the services behind it and use simple hostnames locally, it just needs a little edit in the /etc/hosts file.

I have dropped all this base of a repository into a sample GitHub repository.

The setup is fairly easy :

  • edit the docker-compose.yml to your liking and needs
  • edit the haproxydata/haproxy.cfg based on those changes (hostnames for one)
  • edit the documentation/_config.yml based on those changes (hostnames)
  • edit /etc/hosts to match the above changes
  • run docker-compose up
  • hit http://myteam.local:8000 (or whatever hostname you picked)

And voila : http://web.myteam.local:8000 is where your web service is accessible, http://doc.myteam.local:8000 is where the documentation for this service or your team stuff is.

Reasons

Well, I wanted to play. More importantly I want to try using this model to alleviate the need for third parties such as Github to do code reviews and Slite to do documentation for a team.
Everything the team working on a product has can exist within a git repository. With such a setup the team can have, after one git clone, all the code and the documentation that describes how to work with this code base and the team.

No doc, no test : no review ?

Some of my previous teams would have a culture of “no test : no review”. In my experience, writing that down in a technical culture slide helps a lot drive the good practice in the team.
It does not mean there is no way to pass some code in without tests. Rather it sets an expectation and habit for the team to follow.
Many teams have unsaid rules and habits that are setup de facto during the early days within the limited team. As the team grows a lot of these rules, this culture, are so ingrained within the team that old timers aren’t necessarily aware that they follow them. New comers, if they are lucky, get through an onboarding process and might get some of those cultural rules and tips passed onto them. Some team decided that it’s better to put those rules and tips within a wiki, others in a README, and finally some figure out a way to get as much of it as they can into automation.

Using a documentation service within the code repository might help keeping the topic of culture and documentation of team processes, product processes, release processes, in the place where all the work happens. Thus it could help ensuring documentation is not going stale : you change something in the code, then update the documentation in the same branch.

Templates, automation

Reducing the amount of repetitive tasks to do through automation is key to improve reliability and reduce some of the work fatigue among a team. Google SRE Book has a good chapter on this topic : Eliminating toil

Once you have picked a few rules as a start for your culture you should rely on automated tools to nudge everyone towards following those rules. If you have a style guide then use a linter called within a git hook or the CI Pipeline to get warnings and errors if there are infractions within a commit or branch. If you find yourself reminding people to add some specific details into issues to help with proper triage and resolution you can usually create templates (in Github, Gitlab, Bitbucket, …) that contain a list of the points you are most likely needing every time.

Have a look at this collection of github issue templates for inspiration.

Templates are not rules set in stone, they are nudges for the team to keep on using the best practices they have decided to follow. They can be bypassed at times of need and updated from time to time to match the evolution of the team too.
Automation are good help to have some of those cultural points checked without the need for a human to think of every single one of them when writing or reviewing code, they are the ones doing the nudges.

Conclusion

So, this brings us back to this template I have setup. It’s a base that you can start off or get inspiration for your own. Feel free to create issues to comment or suggest possible changes, I am interested to hear your thoughts on the matter.

Have fun.

Need help ?

We specialise in helping small and medium teams transform the way they build, manage and maintain their Internet based services.

With more than 10 years of experience in running Ruby based web and network applications, and 6 years running products servicing from 2000 to 100000 users daily we bring skills and insights to your teams.

Wether you have a small team looking for insights to quickly get into the right gear to support a massive usage growth, or a medium sized one trying to tackle growth pains between software engineers and infrastructure : we can help.

We are based in France, EU and especially happy to respond to customers from Denmark, Estonia, Finland, France, Italy, Netherlands, Norway, Spain, and Sweden.

We can provide training, general consulting on infrastructure and design, and software engineering remotely or in house depending on location and length of contract.

Contact us to talk about what we can do : sales@imfiny.com.