Category: Uncategorized

Posted on
by

GitLab has a built-in CI system. A GitLab CI pipeline consists of a number of stages. In a stage, a number of jobs are executed in parallel. Jobs are executed as scripts within a docker container.

In the last days I set out to expand my basic knowledge of GitLab CI pipelines (e.g. compiling a Java project with gradle) in order to write a pipelines that compiles, tests and deploys a project to a CloudFoundry instance. Here are 3 things I learnt about.

Reusing jobs with “extends”

“extends” lets us reuse a job. It’s kind of “extract to method” refactoring: suppose I write a job to publish to CloudFoundry. It consists of three steps:

  1. install CloudFoundry CLI
  2. Login to CloudFoundry
  3. Publish app to CloudFoundry

A job that publishes an app to “dev” stage only differs in step 3 to a job that publishes the app to “test” stage. So we write a job that publishes an app to CloudFoundry based on some variables (e.g. “STAGE”)

.deploy-to-cf:
  script:
    - install cf cli
    - cf login -u "$CF_USER" -p "$CF_PASSWORD" -a "$CF_URL"
    - cf push -f manifest.yml --var STAGE=$STAGE

and then our job “deploy-to-dev” just extends this job and redefines STAGE

deploy-to-dev:
  extends: .deploy-to-cf
  variables:
    STAGE: "dev"
  when:
  - develop

We can further “refactoring” our CI pipeline by “extracting” the install&login part because we have two CF foundations and therefore, two different values for “CF_USER” and “CF_PASSWORD”

Test results

GitLab CI understands JUnit test result XML and we just need to provides it as artifact report:

test:
  script:
    - gradlew test
  artifact:
    reports:
      junit: build/reports/**/TEST-*xml

You can see the test results in the pipeline and in the merge request screen.

Speed up caches

Our pipeline uses the cache to avoid downloading gradle (our build tool) and all dependencies everytime a job builds or tests the code. But it is painfully slow, especially collecting the files and pushing the cache.

In my case it takes several minutes (!) to collect 3000+ files in .gradle dir. Things sped up significantly when adding:

variables:
  FF_USE_FASTZIP: "true"
  CACHE_COMPRESSION_LEVEL: "fast"

Here is the documentation.

Posted on
by

Say your git repo consists of a subdirectory “app” and a subdirectory “database” (or “frontend” and “backend” or “team-a” and “team-b”) and you realize each directories content should be in its own repository. Here is how to split the repo.

To create a repo that consists of the content of the “database” dir and its history, execute

git filter-branch --prune-empty --tag-name-filter cat --subdirectory-filter database -- --all

and then push it to a new remote:

git remote rename origin old-origin
git remote add origin <new-remote>
git push -u origin --all
git push -u origin --tags

This is what Gitlab states on a newly created project page. However in git push -u origin –all “-all” should be replaced with “–mirror” because “-all” pushes all local branches only so you would need to checkout all branches you want to keep. “–mirror” pushes remote branches, too.

In the same fashion apply “filter-branches” to the “app” directory.

Posted on
by

My name is Erik. I am a software developer. Most often, I program business “web” software in Java. My main focus is “software integration” that is using other software services in my services. So I’ll focus on this theme in this blog.

I have got a blog called kopf.lastig for 10 years that covers some software development stuff but also other things. Now I decided to move the sw dev stuff to this new blog “Erik on software integration”. I will write here in English and on my old blog in German.

This is a result of me taking a “free email course on building a blog that boosts your career” by John Sonmez from SimpleProgrammer.com .

At first, I will repost some of my old blog posts but later I will add new content.