In my previous blog posts, I explained how parallelization and resource allocation optimization help developers accelerate their CI workflows. Today, I would like to introduce you to another method of optimizing software development processes, which I call “incremental CI”. As its name suggests, it relies on analyzing the source code differences between two versions.

Incremental processing

Nowadays, in order to further increase their productivity, developers include tasks other than builds and tests in their CI workflows. The most common examples are the tools to check code styles for adherence to coding standards and to measure code metrics. Some of these tools, such as gofmt and golint, are designed to process only the files that are passed as inputs and are not affected by any changes in other files. In the case of those tools, the results about input files with no changes are always exactly the same, no matter how many times they are executed. By reusing the result of the previous execution, Inspecode brings a significant performance gain.
To enable this behavior in Inspecode, you should set the value of the incremental parameter to true, as shown in the below example:

inspecode:
    experimental:
        incremental: true

It is important to note that, with Inspecode, incremental processing can also be enabled or disabled on a tool-by-tool basis. In the example below, incremental processing is enabled on gofmt and disabled on golint.

inspecode:
    experimental:
        incremental: true
    gofmt: default
    golint:
        experimental:
            incremental: false

It is also important to note that there are several cases that might cause different results, even if the input source files are not modified:

• Changes are made in the tool’s option settings
• The tool’s configuration file is updated
• The tool is upgraded to a newer version

To avoid any issues in the above scenarios, we take a conservative approach, where the tools are set to process all the input files, no matter if these were updated or not.

Automatic task skipping

Automatic task skipping greatly increases the speed of CI workflows. If incremental processing is enabled and Inspecode does not detect any change on all the target files of a tool, the execution task of the tool will be automatically skipped. You don’t need to write special words like [ci skip] in commit messages. For example, if the all the updates exclusively concern the README file, while the Go source code files are untouched, the execution of golint is skipped. Inspecode reports the same golint issues as detected in the previous task. Hence, the status of the skipped task also remains unchanged from the previous task.

Similarly, if the input and ignore clauses in rocro.yml includes no target files, Inspecode will skip the execution of the tool automatically. The below snippet illustrates a case where all the target files for golint are ignored:

inspecode:
    experimental:
        ignore: "**/*.go"
    golint: default

In the above example, the task execution is skipped regardless of whether incremental processing is enabled or disabled. The status of the task will always be Skipped. You should review the input and ignore settings if the task status is Skipped, although you did not cancel the job including the task.

This blog post introduced you to incremental processing, one of the methods Inspecode uses to help developers speed up their CI workflows. Currently, Inspecode analyses file changes and conservatively decides if incremental processing can be executed so that all the issue reports of tools never change. In the future, we are thinking about making incremental processing the default option. I hope this blog post included useful information that will help you further optimize your CI workflows.

I’m Kenji Hanakawa from Rocro and I’d like to introduce you to one of the most useful features of Inspecode – input filters. First, let me explain what input filters are. By default, Rocro processes all the source code files found in a GitHub or BitBucket repository, including those located in subdirectories.
However, in some cases, you might want to avoid this behavior. For example, if your application uses a third-party library, its source code doesn’t need to be processed. Here is exactly where input filters come to help. With input filters, developers can easily specify which files and directories should be processed and which files and directories should be ignored.

How to Set-up the Input Filters

Rocro provides two directives that you can use in order to set-up input filters in the rocro.yml configuration file:

• input : a file or directory included in an input directive is going to be processed.
• ignore : conversely, a file or directory included in an ignore directive will be excluded from processing.

The syntax follows the .dockerignore format which extends UNIX glob patterns. Let’s see how it actually works:

inspecode:
    gofmt:
        input: mycode # Process mycode file/directory located in the root directory.
        ignore: mycode/sample* #Do not process files and directories starting with sample and located in the mycode directory.

Note that, if input directive is missing, Rocro will process all files and directories, except those that match the ignore directive. It is important to mention that multiple patterns can be mixed:

inspecode:
    gofmt:
        ignore:
            - sample* # Do not process files and directories starting with sample and located in the root directory.
            - example* # Do not process files and directories starting with example and located in the root directory.

How to Use Wildcards

To provide maximum flexibility, Inspecode was engineered to offer support for wildcards. For example, you could use the * wildcard to refer to a hierarchical directory tree:

inspecode:
    gofmt:
        ignore: */*/sample* # Do not process files and directories starting with sample and located in all directories in the second level.

Another useful pattern is **. It matches any directory structure:

inspecode:
    gofmt:
        ignore: **/sample* # Do not process files and directories starting with sample, no matter the location.

Besides ignore, developers can use ! to specify files and directories they want to exclude from processing. Basically, ignore and ! have the same effect.
Here is how our first example can be written using ! instead of ignore:

inspecode:
    gofmt:
        input:
            - mycode # process the files under the mycode directory located in the root directory.
            - !mycode/sample* # Do not process files and directories starting with sample and located in the mycode directory.

Note that, even if ! and ignore can be used simultaneously in the same configuration file, it is quite complicated to do it. Also, the behavior is difficult to understand for most people. To keep your code easy to read and maintain, we advise you to avoid the use of ! within an input clause.

How to Set-up Global Input Filters

Input filters can be applied at the global level too. By doing so, you will configure a set of rules that apply to all tools. Here is an example of how input settings can be applied at the global level:

inspecode:
    experimental:
        input: mycode # Applies to all tools. Process only the mycode file/directory in the root directory.
        ignore: **/sample* # Applies to all tools. Do not process files and directories starting with sample and located in the mycode directory.

Furthermore, it is possible to use both global settings and tool specific settings at the same time. In this scenario, the tool settings are going to be added to the global settings.
This approach is useful when you need apply several general settings to all tools and then further customize them as needed.
Let’s say you need to exclude from processing all directories starting with sample. Additionally, gofmt should ignore all files located in directories starting with gosample. Here is how you can do it:

inspecode:
    experimental:
        input: mycode # We will process only the mycode file/directory, located in the root. It applies to all tools.
        ignore: **/sample* # Do not process files and directories starting with sample and located in the mycode directory. It applies to all tools.
    gofmt:
        input: **/gocode # Process only the gocode files and directories located in mycode directory. It only applies to gofmt.
        ignore: **/gosample* # Do not process files and directories starting with gosample. It only applies to gofmt.

By using both global level filters and tool level ones, you can build sophisticated and easy to understand configuration rules with just a few lines.

The aim of this blog post is to help developers recognize when Inspecode detects problems in their code and also show them how to customize the default settings.

As you probably remember from our previous posts, Inspecode uses the following terminology:

• A job is comprised of several processes. These processes are triggered by various events such as Git push or pull requests. For example, the entire process described in YAML is considered a job.
• A  task is a set of processes executed in individual containers. For example, the execution of each tool is a task. Thus, a job usually contains several tasks.

For each completed task or job, Inspecode assigns a status that you can check to find out if a particular job or task was executed successfully or if it failed. It is important to note that, when the default settings are used, the status of an executed job does not depend on the number of issues detected in your code. Hence, to find out if an issue was detected in the code,  a developer should either check the report or the console log.

How to Configure Job Status Using Thresholds

Fortunately, there is an easier way to spot issues. Inspecode can be configured to automatically set the job status to Failed if the number of issues exceeds a certain threshold. The example below shows how to configure rocro.yml in order to set the status to Failed if the number of issues for the job is greater than 10:

inspecode:
  thresholds:
    num-issues: 10 # Allow up to 10 Issues for the entire job

As you guessed, with these settings, the job status will be set to Failed if more than 10 issues are found. So, when the job status is set to Failed, developers should fix the issues in their code.

Inspecode also lets developers set a threshold at the task level. When the threshold is set at the task level, the status of the job will be set to Failed if any of its tasks will fail. Here is how to configure rocro.yml in order to set a threshold at the task level:

inspecode:
  rubocop:
    thresholds:
      num-issues: 5 # Allow up to 5 Issues of RuboCop
  misspell:
    thresholds:
      num-issues: 10 # Accept Issue of misspelling with 10 tickets

Inspecode was engineered to allow the highest level of customization. Hence, job level thresholds and task level threshold can be mixed. Below you can find an example:

inspecode:
  thresholds:
    num-issues: 10 # Allow up to 10 Issues in the entire job
  rubocop:
    thresholds:
      num-issues: 5 # Allow up to 5 Issues of RuboCop
  misspell:
    thresholds:
      num-issues: 10 # Accept Issue of misspelling with 10 tickets

Configuring Job Status Using Severity Levels

With Inspecode, developers can set the thresholds with even more granularity. For example, a threshold can be set for the number of issues that have a level of severity greater or equal with the one specified. As you already know, Rocro uses four different severity levels:  Info, Warning, Error, and Critical. Let’s have a look at another example:

inspecode:
  rubocop:
    thresholds:
      num-issues:
        total: 10
        warning: 8
        critical: 0

This snippet will set the status to Failed if any of these conditions are met:

• The total number of issues is greater than 10 or
• More than 8 issues with warning, error, or critical security level are detected or
• One or more critical issues are detected

Additional Severity Levels

Not all tools provide the same severity levels. For example, RuboCop supports additional severity levels such as Fatal, Refactor, Convention. Inspecode also allows to set thresholds for those additional severity levels. Here is an example config specifying additional severity levels:

inspecode:
  rubocop:
    thresholds:
      num-issues:
        error: 5
        fatal: 3
        refactor: 10
        convention: 10

To help developers spot any issue with ease, Inspecode maps those external severity levels to the built-in ones. For example, Rubocop’s Fatal is equivalent to Critical while Refactor and Convention are equivalent to Info.

We encourage you to check the documentation for each tool to find out more details about its severity levels.

Conclusion

As we have seen, Inspecode was designed to provide the highest amount of flexibility. By allowing developers to set the threshold levels as shown in this blog post, it is always easy to spot any issue introduced with the latest code change.