Category Creating a new TableGen tool

14. For the code generation process, we have to use the old pass manager. We must simply declare the CodeGenPM instances and add the pass, which makes target-specific information available at the IR transformation level: legacy::PassManager CodeGenPM;
    CodeGenPM.add(createTargetTransformInfoWrapperPass(
    TM->getTargetIRAnalysis()));
15. To output LLVM IR, we must add a pass that prints the IR into a stream: if (FileType == CGFT_AssemblyFile && EmitLLVM) {
    CodeGenPM.add(createPrintModulePass(Out->os()));
    }
16. Otherwise, we must let the TargetMachine instance add the required code generation passes, directed by the FileType value we pass as an argument: else {
    if (TM->addPassesToEmitFile(CodeGenPM, Out->os(),
    nullptr, FileType)) {
    WithColor::error()
    << “No support for file type\n”;
    return false;
    }
    }
17. After all this preparation, we are now ready to execute the passes. First, we must run the optimization pipeline on the IR module. Next, the code generation passes are run. Of course, after all this work, we want to keep the output file: MPM.run(M, MAM); CodeGenPM.run(M);
    Out->keep();
    return true;
    }
18. That was a lot of code, but the process was straightforward. Of course, we have to update the dependencies in the tools/driver/CMakeLists.txt build file too. Besides adding the target components, we must add all the transformation and code generation components from LLVM. The names roughly resemble the directory names where the source is located. The component name is translated into the link library name during the configuration process:

set(LLVM_LINK_COMPONENTS ${LLVM_TARGETS_TO_BUILD}
AggressiveInstCombine Analysis AsmParser
BitWriter CodeGen Core Coroutines IPO IRReader
InstCombine Instrumentation MC ObjCARCOpts Remarks
ScalarOpts Support Target TransformUtils Vectorize
Passes)

19. Our compiler driver supports plugins, and we must announce this support:

add_tinylang_tool(tinylang Driver.cpp SUPPORT_PLUGINS)

20. As before, we have to link against our own libraries:

target_link_libraries(tinylang
PRIVATE tinylangBasic tinylangCodeGen
tinylangLexer tinylangParser tinylangSema)

These are necessary additions to the source code and the build system.

21. To build the extended compiler, you must change into your build directory and type the following:

$ ninja

Changes to the files of the build system are automatically detected, and cmake is run before compiling and linking our changed source. If you need to re-run the configuration step, please follow the instructions in Chapter 1, Installing LLVM, the Compiling the tinylang application section.
As we have used the options for the opt tool as a blueprint, you should try running tinylang with the options to load a pass plugin and run the pass, as we did in the previous sections.
With the current implementation, we can either run a default pass pipeline or we can construct one ourselves. The latter is very flexible, but in almost all cases, it would be overkill. The default pipeline runs very well for C-like languages. However, what is missing is a way to extend the pass pipeline. We’ll look at how to implement this in the next section.

5. Now, we must replace the existing emit() function with a new version. Additionally, we must declare the required PassBuilder instance at the top of the function:

bool emit(StringRef Argv0, llvm::Module *M,
llvm::TargetMachine *TM,
StringRef InputFilename) {
PassBuilder PB(TM);

6. To implement the support for pass plugins given on the command line, we must loop through the list of plugin libraries given by the user and try to load the plugin. We’ll emit an error message if this fails; otherwise, we’ll register the passes: for (auto &PluginFN : PassPlugins) {
   auto PassPlugin = PassPlugin::Load(PluginFN);
   if (!PassPlugin) {
   WithColor::error(errs(), Argv0)
   << “Failed to load passes from ‘” << PluginFN << “‘. Request ignored.\n”; continue; } PassPlugin->registerPassBuilderCallbacks(PB);
   }
7. The information from the static plugin registry is used in a similar way to register those plugins with our PassBuilder instance:

define HANDLE_EXTENSION(Ext) \
getExtPluginInfo().RegisterPassBuilderCallbacks( \
PB);
include “llvm/Support/Extension.def”

8. Now, we need to declare variables for the different analysis managers. The only parameter is the debug flag: LoopAnalysisManager LAM(DebugPM);
   FunctionAnalysisManager FAM(DebugPM);
   CGSCCAnalysisManager CGAM(DebugPM);
   ModuleAnalysisManager MAM(DebugPM);
9. Next, we must populate the analysis managers with calls to the respective register method on the PassBuilder instance. Through this call, the analysis manager is populated with the default analysis passes and also runs registration callbacks. We must also make sure that the function analysis manager uses the default alias-analysis pipeline and that all analysis managers know about each other: FAM.registerPass(
   [&] { return PB.buildDefaultAAPipeline(); });
   PB.registerModuleAnalyses(MAM);
   PB.registerCGSCCAnalyses(CGAM);
   PB.registerFunctionAnalyses(FAM);
   PB.registerLoopAnalyses(LAM);
   PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
10. The MPM module pass manager holds the pass pipeline that we constructed. The instance is initialized with the debug flag: ModulePassManager MPM(DebugPM);
11. Now, we need to implement two different ways to populate the module pass manager with the pass pipeline. If the user provided a pass pipeline on the command line – that is, they have used the –passes option – then we use this as the pass pipeline: if (!PassPipeline.empty()) {
    if (auto Err = PB.parsePassPipeline(
    MPM, PassPipeline)) {
    WithColor::error(errs(), Argv0)
    << toString(std::move(Err)) << “\n”;
    return false;
    }
    }
12. Otherwise, we use the chosen optimization level to determine the pass pipeline to construct. The name of the default pass pipeline is default, and it takes the optimization level as a parameter: else {
    StringRef DefaultPass;
    switch (OptLevel) {
    case 0: DefaultPass = “default”; break;
    case 1: DefaultPass = “default”; break;
    case 2: DefaultPass = “default”; break;
    case 3: DefaultPass = “default”; break;
    case -1: DefaultPass = “default”; break;
    case -2: DefaultPass = “default”; break;
    }
    if (auto Err = PB.parsePassPipeline(
    MPM, DefaultPass)) {
    WithColor::error(errs(), Argv0)
    << toString(std::move(Err)) << “\n”;
    return false;
    }
    }
13. With that, the pass pipeline to run transformations on the IR code has been set up. After this step, we need an open file to write the result to. The system assembler and LLVM IR output are text-based, so we should set the OF_Text flag for them: std::error_code EC;
    sys::fs::OpenFlags OpenFlags = sys::fs::OF_None;
    CodeGenFileType FileType = codegen::getFileType();
    if (FileType == CGFT_AssemblyFile)
    OpenFlags |= sys::fs::OF_Text;
    auto Out = std::make_unique(
    outputFilename(InputFilename), EC, OpenFlags);
    if (EC) {
    WithColor::error(errs(), Argv0)
    << EC.message() << ‘\n’;
    return false;
    }

With the –-passes option, you can not only name a single pass but you can also describe a whole pipeline. For example, the default pipeline for optimization level 2 is named default<O2>. You can run the ppprofile pass before the default pipeline with the–-passes=”ppprofile,default<O2>” argument. Please note that the pass names in such a pipeline description must be of the same type.

Now, let’s turn to using the new pass with clang.

Plugging the new pass into clang

In the previous section, you learned how you can run a single pass using opt. This is useful if you need to debug a pass but for a real compiler, the steps should not be that involved.

To achieve the best result, a compiler needs to run the optimization passes in a certain order. The LLVM pass manager has a default order for pass execution. This is also called the default pass pipeline. Using opt, you can specify a different pass pipeline with the –passes option. This is flexible but also complicated for the user. It also turns out that most of the time, you just want to add a new pass at very specific points, such as before optimization passes are run or at the end of the loop optimization processes. These points are called extension points. The PassBuilder class allows you to register a pass at an extension point. For example, you can call the registerPipelineStartEPCallback() method to add a pass to the beginning of the optimization pipeline. This is exactly the place we need for the ppprofiler pass. During optimization, functions may be inlined, and the pass will miss those inline functions. Instead, running the pass before the optimization passes guarantees that all functions are instrumented.

To use this approach, you need to extend the RegisterCB() function in the pass plugin. Add the following code to the function:
  PB.registerPipelineStartEPCallback(
      [](ModulePassManager &PM, OptimizationLevel Level) {
        PM.addPass(PPProfilerIRPass());
      });

Whenever the pass manager populates the default pass pipeline, it calls all the callbacks for the extension points. We simply add the new pass here.

To load the plugin into clang, you can use the -fpass-plugin option. Creating the instrumented executable of the hello.c file now becomes almost trivial:
$ clang -fpass-plugin=./PPProfiler.so hello.c runtime.c

Please run the executable and verify that the run creates the ppprofiler.csv file.

Recall the ppprofiler pass that we developed as a plugin out of the LLVM tree in the Developing the ppprofiler pass as a plugin section. Here, we’ll learn how to use this pass with LLVM tools, such as opt and clang, as they can load plugins.
Let’s look at opt first.
Run the pass plugin in opt
To play around with the new plugin, you need a file containing LLVM IR. The easiest way to do this is to translate a C program, such as a basic “Hello World” style program:

include
int main(int argc, char *argv[]) {
puts(“Hello”);
return 0;
}

Compile this file, hello.c, with clang:

$ clang -S -emit-llvm -O1 hello.c

You will get a very simple IR file called hello.ll that contains the following code:

$ cat hello.ll
@.str = private unnamed_addr constant [6 x i8] c”Hello\00″,
align 1
define dso_local i32 @main(
i32 noundef %0, ptr nocapture noundef readnone %1) {
%3 = tail call i32 @puts(
ptr noundef nonnull dereferenceable(1) @.str)
ret i32 0
}

This is enough to test the pass.
To run the pass, you have to provide a couple of arguments. First, you need to tell opt to load the shared library via the –load-pass-plugin option. To run a single pass, you must specify the–-passes option. Using the hello.ll file as input, you can run the following:

$ opt –load-pass-plugin=./PPProfile.so \
–passes=”ppprofiler” –stats hello.ll -o hello_inst.bc

If statistic generation is enabled, you will see the following output:

===——————————————————–===
… Statistics Collected …
===——————————————————–===
1 ppprofiler – Number of instrumented functions.

Otherwise, you will be informed that statistic collection is not enabled:

Statistics are disabled. Build with asserts or with
-DLLVM_FORCE_ENABLE_STATS

The bitcode file, hello_inst.bc, is the result. You can turn this file into readable IR with the llvm-dis tool. As expected, you will see the calls to the __ppp_enter() and __ppp_exit() functions and a new constant for the name of the function:

$ llvm-dis hello_inst.bc -o –
@.str = private unnamed_addr constant [6 x i8] c”Hello\00″,
align 1
@0 = private unnamed_addr constant [5 x i8] c”main\00″,
align 1
define dso_local i32 @main(i32 noundef %0,
ptr nocapture noundef readnone %1) {
call void @__ppp_enter(ptr @0)
%3 = tail call i32 @puts(
ptr noundef nonnull dereferenceable(1) @.str)
call void @__ppp_exit(ptr @0)
ret i32 0
}

This already looks good! It would be even better if we could turn this IR into an executable and run it. For this, you need to provide implementations for the called functions.

To fully integrate the new pass into LLVM, the source of the plugin needs to be structured slightly differently. The main reason for this is that the constructor of the pass class is called from the pass registry, which requires the class interface to be put into a header file.
Like before, you must put the new pass into the Transforms component of LLVM. Begin the implementation by creating the llvm-project/llvm/include/llvm/Transforms/PPProfiler/PPProfiler.h header file. The content of that file is the class definition; put it into the llvm namespace. No other changes are required:

ifndef LLVM_TRANSFORMS_PPPROFILER_PPPROFILER_H
define LLVM_TRANSFORMS_PPPROFILER_PPPROFILER_H
include “llvm/IR/PassManager.h”
namespace llvm {
class PPProfilerIRPass
: public llvm::PassInfoMixin {
public:
llvm::PreservedAnalyses
run(llvm::Module &M, llvm::ModuleAnalysisManager &AM);
private:
void instrument(llvm::Function &F,
llvm::Function *EnterFn,
llvm::Function *ExitFn);
};
} // namespace llvm
endif

Next, copy the source file of the pass plugin, PPProfiler.cpp, into the new directory, llvm-project/llvm/lib/Transforms/PPProfiler. This file needs to be updated in the following way:

1. Since the class definition is now in a header file, you must remove the class definition from this file. At the top, add the include directive for the header file:

include “llvm/Transforms/PPProfiler/PPProfiler.h”

2. The llvmGetPassPluginInfo() function must be removed because the pass wasn’t built into a shared library of its own.
   As before, you also need to provide a CMakeLists.txt file for the build. You must declare the new pass as a new component:

add_llvm_component_library(LLVMPPProfiler
PPProfiler.cpp
LINK_COMPONENTS
Core
Support
)

After, like in the previous section, you need to include the new source directory by adding the following line to the CMakeLists.txt file in the parent directory:

add_subdirectory(PPProfiler)

Inside LLVM, the available passes are kept in the llvm/lib/Passes/ PassRegistry.def database file. You need to update this file. The new pass is a module pass, so we need to search inside the file for the section in which module passes are defined, for example, by searching for the MODULE_PASS macro. Inside this section, add the following line:

MODULE_PASS(“ppprofiler”, PPProfilerIRPass())

This database file is used in the llvm/lib/Passes/PassBuilder.cpp class. This file needs to include your new header file:

include “llvm/Transforms/PPProfiler/PPProfiler.h”

These are all required source changes based on the plugin version of the new pass.
Since you created a new LLVM component, it is also necessary to add a link dependency in the llvm/lib/Passes/CMakeLists.txt file. Under the LINK_COMPONENTS keyword, you need to add a line with the name of the new component:

PPProfiler

Et voilà – you are ready to build and install LLVM. The new pass, ppprofiler, is now available to all LLVM tools. It has been compiled into the libLLVMPPProfiler.a library and available in the build system as the PPProfiler component.
So far, we have talked about how to create a new pass. In the next section, we will examine how to use the ppprofiler pass.