Category Extending the pass pipeline

Let’s define a simple record for an instruction:

def ADD {
string Mnemonic = “add”;
int Opcode = 0xA0;
}

The def keyword signals that you define a record. It is followed by the name of the record. The record body is surrounded by curly braces, and the body consists of field definitions, similar to a structure in C++.
You can use the llvm-tblgen tool to see the generated records. Save the preceding source code in an inst.td file and run the following:

$ llvm-tblgen –print-records inst.td
————- Classes —————–
————- Defs —————–
def ADD {
string Mnemonic = “add”;
int Opcode = 160;
}

This is not yet exciting; it only shows the defined record was parsed correctly.
Defining instructions using single records is not very comfortable. A modern CPU has hundreds of instructions, and with this amount of records, it is very easy to introduce typing errors in the field names. And if you decide to rename a field or add a new field, then the number of records to change becomes a challenge. Therefore, a blueprint is needed. In C++, classes have a similar purpose, and in TableGen, it is also called a class. Here is the definition of an Inst class and two records based on that class:

class Inst {
string Mnemonic = mnemonic;
int Opcode = opcode;
}
def ADD : Inst<“add”, 0xA0>;
def SUB : Inst<“sub”, 0xB0>;

The syntax for classes is similar to that of records. The class keyword signals that a class is defined, followed by the name of the class. A class can have a parameter list. Here, the Inst class has two parameters, mnemonic and opcode, which are used to initialize the records’ fields. The values for those fields are given when the class is instantiated. The ADD and SUB records show two instantiations of the class. Again, let’s use llvm-tblgen to look at the records:

$ llvm-tblgen –print-records inst.td
————- Classes —————–
class Inst {
string Mnemonic = Inst:mnemonic;
int Opcode = Inst:opcode;
}
————- Defs —————–
def ADD { // Inst
string Mnemonic = “add”;
int Opcode = 160;
}
def SUB { // Inst
string Mnemonic = “sub”;
int Opcode = 176;
}

Now, you have one class definition and two records. The name of the class used to define the records is shown as a comment. Please note that the arguments of the class have the default value ?, which indicates int is uninitialized.

The tinylang compiler we developed in the previous chapters performs no optimizations on the IR code. In the next few subsections, we’ll add an optimization pipeline to the compiler to achieve this accordingly.
Creating an optimization pipeline
The PassBuilder class is central to setting up the optimization pipeline. This class knows about all registered passes and can construct a pass pipeline from a textual description. We can use this class to either create the pass pipeline from a description given on the command line or use a default pipeline based on the requested optimization level. We also support the use of pass plugins, such as the ppprofiler pass plugin we discussed in the previous section. With this, we can mimic part of the functionality of the opt tool and also use similar names for the command-line options.
The PassBuilder class populates an instance of a ModulePassManager class, which is the pass manager that holds the constructed pass pipeline and runs it. The code generation passes still use the old pass manager. Therefore, we have to retain the old pass manager for this purpose.
For the implementation, we will extend the tools/driver/Driver.cpp file from our tinylang compiler:

1. We’ll use new classes, so we’ll begin with adding new include files. The llvm/Passes/PassBuilder.h file defines the PassBuilder class. The llvm/Passes/PassPlugin.h file is required for plugin support. Finally, the llvm/Analysis/TargetTransformInfo.h file provides a pass that connects IR-level transformations with target-specific information:

include “llvm/Passes/PassBuilder.h”
include “llvm/Passes/PassPlugin.h”
include “llvm/Analysis/TargetTransformInfo.h”

2. To use certain features of the new pass manager, we must add three command-line options, using the same names as the opt tool does. The –passes option allows the textual specification of the pass pipeline, while the –load-pass-plugin option allows the use of pass plugins. If the –debug-pass-manager option is given, then the pass manager prints out information about the executed passes:

static cl::opt
DebugPM(“debug-pass-manager”, cl::Hidden,
cl::desc(“Print PM debugging information”));
static cl::opt PassPipeline(
“passes”,
cl::desc(“A description of the pass pipeline”));
static cl::list PassPlugins(
“load-pass-plugin”,
cl::desc(“Load passes from plugin library”));

3. The user influences the construction of the pass pipeline with the optimization level. The PassBuilder class supports six different optimization levels: no optimization, three levels for optimizing speed, and two levels for reducing size. We can capture all levels in one command-line option:

static cl::opt OptLevel(
cl::desc(“Setting the optimization level:”),
cl::ZeroOrMore,
cl::values(
clEnumValN(3, “O”, “Equivalent to -O3”),
clEnumValN(0, “O0”, “Optimization level 0”),
clEnumValN(1, “O1”, “Optimization level 1”),
clEnumValN(2, “O2”, “Optimization level 2”),
clEnumValN(3, “O3”, “Optimization level 3”),
clEnumValN(-1, “Os”,
“Like -O2 with extra optimizations “
“for size”),
clEnumValN(
-2, “Oz”,
“Like -Os but reduces code size further”)),
cl::init(0));

4. The plugin mechanism of LLVM supports a plugin registry for statically linked plugins, which is created during the configuration of the project. To make use of this registry, we must include the llvm/Support/Extension.def database file to create the prototype for the functions that return the plugin information:

define HANDLE_EXTENSION(Ext) \
llvm::PassPluginLibraryInfo getExtPluginInfo();
include “llvm/Support/Extension.def”

With the –-passes option, you can not only name a single pass but you can also describe a whole pipeline. For example, the default pipeline for optimization level 2 is named default<O2>. You can run the ppprofile pass before the default pipeline with the–-passes=”ppprofile,default<O2>” argument. Please note that the pass names in such a pipeline description must be of the same type.

Now, let’s turn to using the new pass with clang.

Plugging the new pass into clang

In the previous section, you learned how you can run a single pass using opt. This is useful if you need to debug a pass but for a real compiler, the steps should not be that involved.

To achieve the best result, a compiler needs to run the optimization passes in a certain order. The LLVM pass manager has a default order for pass execution. This is also called the default pass pipeline. Using opt, you can specify a different pass pipeline with the –passes option. This is flexible but also complicated for the user. It also turns out that most of the time, you just want to add a new pass at very specific points, such as before optimization passes are run or at the end of the loop optimization processes. These points are called extension points. The PassBuilder class allows you to register a pass at an extension point. For example, you can call the registerPipelineStartEPCallback() method to add a pass to the beginning of the optimization pipeline. This is exactly the place we need for the ppprofiler pass. During optimization, functions may be inlined, and the pass will miss those inline functions. Instead, running the pass before the optimization passes guarantees that all functions are instrumented.

To use this approach, you need to extend the RegisterCB() function in the pass plugin. Add the following code to the function:
  PB.registerPipelineStartEPCallback(
      [](ModulePassManager &PM, OptimizationLevel Level) {
        PM.addPass(PPProfilerIRPass());
      });

Whenever the pass manager populates the default pass pipeline, it calls all the callbacks for the extension points. We simply add the new pass here.

To load the plugin into clang, you can use the -fpass-plugin option. Creating the instrumented executable of the hello.c file now becomes almost trivial:
$ clang -fpass-plugin=./PPProfiler.so hello.c runtime.c

Please run the executable and verify that the run creates the ppprofiler.csv file.

Recall the ppprofiler pass that we developed as a plugin out of the LLVM tree in the Developing the ppprofiler pass as a plugin section. Here, we’ll learn how to use this pass with LLVM tools, such as opt and clang, as they can load plugins.
Let’s look at opt first.
Run the pass plugin in opt
To play around with the new plugin, you need a file containing LLVM IR. The easiest way to do this is to translate a C program, such as a basic “Hello World” style program:

include
int main(int argc, char *argv[]) {
puts(“Hello”);
return 0;
}

Compile this file, hello.c, with clang:

$ clang -S -emit-llvm -O1 hello.c

You will get a very simple IR file called hello.ll that contains the following code:

$ cat hello.ll
@.str = private unnamed_addr constant [6 x i8] c”Hello\00″,
align 1
define dso_local i32 @main(
i32 noundef %0, ptr nocapture noundef readnone %1) {
%3 = tail call i32 @puts(
ptr noundef nonnull dereferenceable(1) @.str)
ret i32 0
}

This is enough to test the pass.
To run the pass, you have to provide a couple of arguments. First, you need to tell opt to load the shared library via the –load-pass-plugin option. To run a single pass, you must specify the–-passes option. Using the hello.ll file as input, you can run the following:

$ opt –load-pass-plugin=./PPProfile.so \
–passes=”ppprofiler” –stats hello.ll -o hello_inst.bc

If statistic generation is enabled, you will see the following output:

===——————————————————–===
… Statistics Collected …
===——————————————————–===
1 ppprofiler – Number of instrumented functions.

Otherwise, you will be informed that statistic collection is not enabled:

Statistics are disabled. Build with asserts or with
-DLLVM_FORCE_ENABLE_STATS

The bitcode file, hello_inst.bc, is the result. You can turn this file into readable IR with the llvm-dis tool. As expected, you will see the calls to the __ppp_enter() and __ppp_exit() functions and a new constant for the name of the function:

$ llvm-dis hello_inst.bc -o –
@.str = private unnamed_addr constant [6 x i8] c”Hello\00″,
align 1
@0 = private unnamed_addr constant [5 x i8] c”main\00″,
align 1
define dso_local i32 @main(i32 noundef %0,
ptr nocapture noundef readnone %1) {
call void @__ppp_enter(ptr @0)
%3 = tail call i32 @puts(
ptr noundef nonnull dereferenceable(1) @.str)
call void @__ppp_exit(ptr @0)
ret i32 0
}

This already looks good! It would be even better if we could turn this IR into an executable and run it. For this, you need to provide implementations for the called functions.

Implementing a new pass as a plugin is useful if you plan to use it with a precompiled clang, for example. On the other hand, if you write your own compiler, then there can be good reasons to add your new passes directly to the LLVM source tree. There are two different ways you can do this – as a plugin and as a fully integrated pass. The plugin approach requires fewer changes.

Utilizing the plugin mechanisms inside the LLVM source tree

The source of passes that perform transformations on LLVM IR is located in the llvm-project/llvm/lib/Transforms directory. Inside this directory, create a new directory called PPProfiler and copy the source file, PPProfiler.cpp, into it. You do not need to make any source changes!

To integrate the new plugin into the build system, create a file called CMakeLists.txt with the following content:
add_llvm_pass_plugin(PPProfiler PPProfiler.cpp)

Finally, in the CmakeLists.txt file in the parent directory, you need to include the new source directory by adding the following line:
add_subdirectory(PPProfiler)

You are now ready to build LLVM with PPProfiler added. Change into the build directory of LLVM and manually run Ninja:
$ ninja install

CMake will detect a change in the build description and rerun the configuration step. You will see an additional line:
— Registering PPProfiler as a pass plugin (static build: OFF)

This tells you that the plugin was detected and has been built as a shared library. After the installation step, you will find that shared library, PPProfiler.so, in the <install directory>/lib directory.

So far, the only difference to the pass plugin from the previous section is that the shared library is installed as part of LLVM. But you can also statically link the new plugin to the LLVM tools. To do this, you need to rerun the CMake configuration and add the -DLLVM_PPPROFILER_LINK_INTO_TOOLS=ON option on the command line. Look for this information from CMake to confirm the changed build option:
— Registering PPProfiler as a pass plugin (static build: ON)

After compiling and installing LLVM again, the following has changed:

• The plugin is compiled into the static library, libPPProfiler.a, and that library is installed in the <install directory>/lib directory.
• The LLVM tools, such as opt, are linked against that library.
• The plugin is registered as an extension. You can check that the <install directory>/include/llvm/Support/Extension.def file now contains the following line:

HANDLE_EXTENSION(PPProfiler)

In addition, all tools that support this extension mechanism pick up the new pass. In the Creating an optimization pipeline section, you will learn how to do this in your compiler.

This approach works well because the new source files reside in a separate directory, and only one existing file was changed. This minimizes the probability of merge conflicts if you try to keep your modified LLVM source tree in sync with the main repository.

There are also situations where adding the new pass as a plugin is not the best way. The passes that LLVM provides use a different way for registration. If you develop a new pass and propose to add it to LLVM, and the LLVM community accepts your contribution, then you will want to use the same registration mechanism.