Tag: Gitlab

Posted on
by

GitLab has a built-in CI system. A GitLab CI pipeline consists of a number of stages. In a stage, a number of jobs are executed in parallel. Jobs are executed as scripts within a docker container.

In the last days I set out to expand my basic knowledge of GitLab CI pipelines (e.g. compiling a Java project with gradle) in order to write a pipelines that compiles, tests and deploys a project to a CloudFoundry instance. Here are 3 things I learnt about.

Reusing jobs with “extends”

“extends” lets us reuse a job. It’s kind of “extract to method” refactoring: suppose I write a job to publish to CloudFoundry. It consists of three steps:

  1. install CloudFoundry CLI
  2. Login to CloudFoundry
  3. Publish app to CloudFoundry

A job that publishes an app to “dev” stage only differs in step 3 to a job that publishes the app to “test” stage. So we write a job that publishes an app to CloudFoundry based on some variables (e.g. “STAGE”)

.deploy-to-cf:
  script:
    - install cf cli
    - cf login -u "$CF_USER" -p "$CF_PASSWORD" -a "$CF_URL"
    - cf push -f manifest.yml --var STAGE=$STAGE

and then our job “deploy-to-dev” just extends this job and redefines STAGE

deploy-to-dev:
  extends: .deploy-to-cf
  variables:
    STAGE: "dev"
  when:
  - develop

We can further “refactoring” our CI pipeline by “extracting” the install&login part because we have two CF foundations and therefore, two different values for “CF_USER” and “CF_PASSWORD”

Test results

GitLab CI understands JUnit test result XML and we just need to provides it as artifact report:

test:
  script:
    - gradlew test
  artifact:
    reports:
      junit: build/reports/**/TEST-*xml

You can see the test results in the pipeline and in the merge request screen.

Speed up caches

Our pipeline uses the cache to avoid downloading gradle (our build tool) and all dependencies everytime a job builds or tests the code. But it is painfully slow, especially collecting the files and pushing the cache.

In my case it takes several minutes (!) to collect 3000+ files in .gradle dir. Things sped up significantly when adding:

variables:
  FF_USE_FASTZIP: "true"
  CACHE_COMPRESSION_LEVEL: "fast"

Here is the documentation.

Posted on
by

Many software projects use 3rd party libraries aka “dependencies”. You often want to use the most recent version of these dependencies but how do you know when a new release of a dependency is published? The more dependencies your project have the more tiresome a manual approach to “tracking dependency updates” is.

In this post I explore some solutions that tracks dependency updates for you. I cover broad solutions (libraries.io and dependabot) and Java-only solutions (“artifact listener” and a Gradle/Maven plugin).

Why update?

But why do we want to update dependencies at all?

A new version of a dependency

  • may fix bugs that affects your project
  • may introduce new features that you could use
  • may fix a security issue that affects your project
  • may have other optimizations to the code

Of course there is a risk as well: a new version may introduce a bug that affects your project. Plus, there might be API changes that require changes in your code.

Tracking solutions

Renovate

(update) I now use renovatebot because it integrates nicely with Gitlab CI. Much like dependabot (see below), it scans “dependency files” like “build.gradle”, “pom.xml” or “package.json” and creates merge requests for dependency updates.

Libraries.io

From their own words

Libraries.io can automatically keep track of all of the packages that your repositories depend upon across many different package managers.

Once synced, Libraries.io will email you about new versions of your dependencies, if you add or remove a new dependency it will change the notifications settings for that package as soon as you push to your repositories.

Repositories on Github, Gitlab and Bitbucket are supported. Plus, you can subscribe to dependencies manually, ie without a repository on any of these platforms.

Beside email notifications you can also subscribe to an RSS feed of your dependency updates.

Libraries.io is an open source project.

artifact listener

Artifact Listener is a small service and only available for Java / Maven Central. You can search for libraries and “follow” them. Alternatively you can upload a POM and then choose which dendencies to follow. Updates of libraries you follow are emailed to you.

You can provide additional email adresses to notify, e.g, email addresses of other team members. This is a small but lovely feature for me.

The service is an open source project.

Dependabot

Dependabot checks the “dependency files” (where your dependencies are definied) in your Github repos for updates. If there is an update it creates a PR for it. The PR may contain links, release notes, a list of commits etc.

So this service not only notifies you about an update but even creates a PR that applies it. You just have to merge it (at least if your project is on Github).

Dependabout has been aquired by Github.com and is free of charge.

Gradle plugin

If you are using Gradle (a Java build system) to declare dependencies and build your project you can use the Gradle versions plugin to detect dependency updates and report them. It is easy to use. You just need to execute it on a regular basis.

Maven plugin

Of course, there is a similar plugin for Maven (another Java build system).