The aim of this blog post is to help developers recognize when Inspecode detects problems in their code and also show them how to customize the default settings.

As you probably remember from our previous posts, Inspecode uses the following terminology:

• A job is comprised of several processes. These processes are triggered by various events such as Git push or pull requests. For example, the entire process described in YAML is considered a job.
• A  task is a set of processes executed in individual containers. For example, the execution of each tool is a task. Thus, a job usually contains several tasks.

For each completed task or job, Inspecode assigns a status that you can check to find out if a particular job or task was executed successfully or if it failed. It is important to note that, when the default settings are used, the status of an executed job does not depend on the number of issues detected in your code. Hence, to find out if an issue was detected in the code,  a developer should either check the report or the console log.

How to Configure Job Status Using Thresholds

Fortunately, there is an easier way to spot issues. Inspecode can be configured to automatically set the job status to Failed if the number of issues exceeds a certain threshold. The example below shows how to configure rocro.yml in order to set the status to Failed if the number of issues for the job is greater than 10:

inspecode:
  thresholds:
    num-issues: 10 # Allow up to 10 Issues for the entire job

As you guessed, with these settings, the job status will be set to Failed if more than 10 issues are found. So, when the job status is set to Failed, developers should fix the issues in their code.

Inspecode also lets developers set a threshold at the task level. When the threshold is set at the task level, the status of the job will be set to Failed if any of its tasks will fail. Here is how to configure rocro.yml in order to set a threshold at the task level:

inspecode:
  rubocop:
    thresholds:
      num-issues: 5 # Allow up to 5 Issues of RuboCop
  misspell:
    thresholds:
      num-issues: 10 # Accept Issue of misspelling with 10 tickets

Inspecode was engineered to allow the highest level of customization. Hence, job level thresholds and task level threshold can be mixed. Below you can find an example:

inspecode:
  thresholds:
    num-issues: 10 # Allow up to 10 Issues in the entire job
  rubocop:
    thresholds:
      num-issues: 5 # Allow up to 5 Issues of RuboCop
  misspell:
    thresholds:
      num-issues: 10 # Accept Issue of misspelling with 10 tickets

Configuring Job Status Using Severity Levels

With Inspecode, developers can set the thresholds with even more granularity. For example, a threshold can be set for the number of issues that have a level of severity greater or equal with the one specified. As you already know, Rocro uses four different severity levels:  Info, Warning, Error, and Critical. Let’s have a look at another example:

inspecode:
  rubocop:
    thresholds:
      num-issues:
        total: 10
        warning: 8
        critical: 0

This snippet will set the status to Failed if any of these conditions are met:

• The total number of issues is greater than 10 or
• More than 8 issues with warning, error, or critical security level are detected or
• One or more critical issues are detected

Additional Severity Levels

Not all tools provide the same severity levels. For example, RuboCop supports additional severity levels such as Fatal, Refactor, Convention. Inspecode also allows to set thresholds for those additional severity levels. Here is an example config specifying additional severity levels:

inspecode:
  rubocop:
    thresholds:
      num-issues:
        error: 5
        fatal: 3
        refactor: 10
        convention: 10

To help developers spot any issue with ease, Inspecode maps those external severity levels to the built-in ones. For example, Rubocop’s Fatal is equivalent to Critical while Refactor and Convention are equivalent to Info.

We encourage you to check the documentation for each tool to find out more details about its severity levels.

Conclusion

As we have seen, Inspecode was designed to provide the highest amount of flexibility. By allowing developers to set the threshold levels as shown in this blog post, it is always easy to spot any issue introduced with the latest code change.