Category ITCertification Exams

4. The run() method calls all the emitting methods. It also times the length of each phase. You specify the –time-phases option, and then the timing is shown after all code is generated:

void TokenAndKeywordFilterEmitter::run(raw_ostream &OS) {
// Emit Flag fragments.
Records.startTimer(“Emit flags”);
emitFlagsFragment(OS);
// Emit token kind enum and functions.
Records.startTimer(“Emit token kind”);
emitTokenKind(OS);
// Emit keyword filter code.
Records.startTimer(“Emit keyword filter”);
emitKeywordFilter(OS);
Records.stopTimer();
}

5. The emitFlagsFragment() method shows the typical structure of a function emitting C++ source code. The generated code is guarded by the GET_TOKEN_FLAGS macro. To emit the C++ source fragment, you loop over all records that are derived from the Flag class in the TableGen file. Having such a record, it is easy to query the record for the name and the value. Please note that the names Flag, Name, and Val must be written exactly as in the TableGen file. If you rename Val to Value in the TableGen file, then you also need to change the string in this function. All the generated source code is written to the provided stream, OS:

void TokenAndKeywordFilterEmitter::emitFlagsFragment(
raw_ostream &OS) {
OS << “ifdef GET_TOKEN_FLAGS\n”; OS << “undef GET_TOKEN_FLAGS\n”; for (Record *CC : Records.getAllDerivedDefinitions(“Flag”)) { StringRef Name = CC->getValueAsString(“Name”);
int64_t Val = CC->getValueAsInt(“Val”);
OS << Name << ” = ” << format_hex(Val, 2) << “,\n”;
}
OS << “endif\n”;
}

6. The emitTokenKind() method emits a declaration and definition of token classification functions. Let’s have a look at emitting the declarations first. The overall structure is the same as the previous method – only more C++ source code is emitted. The generated source fragment is guarded by the GET_TOKEN_KIND_DECLARATION macro. Please note that this method tries to generate nicely formatted C++ code, using new lines and indentation as a human developer would do. In case the emitted source code is not correct, and you need to examine it to find the error, this will be tremendously helpful. It is also easy to make such errors: after all, you are writing a C++ function that emits C++ source code.
   First, the TokenKind enumeration is emitted. The name for a keyword should be prefixed with a kw_ string. The loop goes over all records of the Token class, and you can query the records if they are also a subclass of the Keyword class, which enables you to emit the prefix: OS << “ifdef GET_TOKEN_KIND_DECLARATION\n” << “undef GET_TOKEN_KIND_DECLARATION\n” << “namespace tok {\n” << ” enum TokenKind : unsigned short {\n”; for (Record *CC : Records.getAllDerivedDefinitions(“Token”)) { StringRef Name = CC->getValueAsString(“Name”);
   OS << ” “; if (CC->isSubClassOf(“Keyword”))
   OS << “kw_”;
   OS << Name << “,\n”;
   }
   OS << „ NUM_TOKENS\n”
   << „ };\n”;

Since parsing and creation of records are done through an LLVM library, you only need to care about the backend implementation, which consists mostly of generating C++ source code fragments based on the information in the records. First, you need to be clear about what source code to generate before you can put it into the backend.
Sketching the source code to be generated
The output of the TableGen tool is a single file containing C++ fragments. The fragments are guarded by macros. The goal is to replace the TokenKinds.def database file. Based on the information in the TableGen file, you can generate the following:

1. The enumeration members used to define flags. The developer is free to name the type; however, it should be based on the unsigned type. If the generated file is named TokenKinds.inc, then the intended use is this:

enum Flags : unsigned {
define GET_TOKEN_FLAGS
include “TokenKinds.inc”
}

2. The TokenKind enumeration, and the prototypes and definitions of the getTokenName(), getPunctuatorSpelling(), and getKeywordSpelling() functions. This code replaces the TokenKinds.def database file, most of the TokenKinds.h include file and the TokenKinds.cpp. source file.
3. A new lookupKeyword() function that can be used instead of the current implementation using the llvm::StringMap. type. This is the function you want to optimize.
   Knowing what you want to generate, you can now turn to implementing the backend.
   Creating a new TableGen tool
   A simple structure for your new tool is to have a driver that evaluates the command-line options and calls the generation functions and the actual generator functions in a different file. Let’s call the driver file TableGen.cpp and the file containing the generator TokenEmitter.cpp. You also need a TableGenBackends.h header file. Let’s begin the implementation with the generation of the C++ code in the TokenEmitter.cpp file:
4. As usual, the file begins with including the required headers. The most important one is llvm/TableGen/Record.h, which defines a Record class, used to hold records generated by parsing the .td file:

include “TableGenBackends.h”
include “llvm/Support/Format.h”
include “llvm/TableGen/Record.h”
include “llvm/TableGen/TableGenBackend.h”
include

2. To simplify coding, the llvm namespace is imported:

using namespace llvm;

3. The TokenAndKeywordFilterEmitter class is responsible for generating the C++ source code. The emitFlagsFragment(), emitTokenKind(), and emitKeywordFilter() methods emit the source code, as described in the previous section, Sketching the source code to be generated. The only public method, run(), calls all the code-emitting methods. The records are held in an instance of RecordKeeper, which is passed as a parameter to the constructor. The class is inside an anonymous namespace:

namespace {
class TokenAndKeywordFilterEmitter {
RecordKeeper &Records;
public:
explicit TokenAndKeywordFilterEmitter(RecordKeeper &R)
: Records(R) {}
void run(raw_ostream &OS);
private:
void emitFlagsFragment(raw_ostream &OS);
void emitTokenKind(raw_ostream &OS);
void emitKeywordFilter(raw_ostream &OS);
};
} // End anonymous namespace

The TokenKinds.def database file defines three different macros. The TOK macro is used for tokens that do not have a fixed spelling – for example, for integer literals. The PUNCTUATOR macro is used for all kinds of punctuation marks and includes a preferred spelling. Lastly, the KEYWORD macro defines a keyword that is made up of a literal and a flag, which is used to indicate at which language level this literal is a keyword. For example, the thread_local keyword was added to C++11.
One way to express this in the TableGen language is to create a Token class that holds all the data. You can then add subclasses of that class to make the usage more comfortable. You also need a Flag class for flags defined together with a keyword. And last, you need a class to define a keyword filter. These classes define the basic data structure and can be potentially reused in other projects. Therefore, you create a Keyword.td file for it. Here are the steps:

1. A flag is modeled as a name and an associated value. This makes it easy to generate an enumeration from this data:

class Flag {
string Name = name;
int Val = val;
}

2. The Token class is used as the base class. It just carries a name. Please note that this class has no parameters:

class Token {
string Name;
}

3. The Tok class has the same function as the corresponding TOK macro from the database file. it represents a token without fixed spellings. It derives from the base class, Token, and just adds initialization for the name:

class Tok : Token {
let Name = name;
}

4. In the same way, the Punctuator class resembles the PUNCTUATOR macro. It adds a field for the spelling of the token:

class Punctuator : Token {
let Name = name;
string Spelling = spelling;
}

5. And last, the Keyword class needs a list of flags:

class Keyword flags> : Token {
let Name = name;
list Flags = flags;
}

6. With these definitions in place, you can now define a class for the keyword filter, called TokenFilter. It takes a list of tokens as a parameter:

class TokenFilter tokens> {
string FunctionName;
list Tokens = tokens;
}

With these class definitions, you are certainly able to capture all the data from the TokenKinds.def database file. The TinyLang language does not utilize the flags, since there is only this version of the language. Real-world languages such as C and C++ have undergone a couple of revisions, and they usually require flags. Therefore, we use keywords from C and C++ as an example. Let’s create a KeywordC.td file, as follows:

1. First, you include the class definitions created earlier:

Include “Keyword.td”

2. Next, you define flags. The value is the binary value of the flag. Note how the !or operator is used to create a value for the KEYALL flag:

def KEYC99 : Flag<“KEYC99”, 0x1>;
def KEYCXX : Flag<“KEYCXX”, 0x2>;
def KEYCXX11: Flag<“KEYCXX11”, 0x4>;
def KEYGNU : Flag<“KEYGNU”, 0x8>;
def KEYALL : Flag<“KEYALL”, !or(KEYC99.Val, KEYCXX.Val, KEYCXX11.Val , KEYGNU.Val)>;

3. There are tokens without a fixed spelling – for example, a comment:

def : Tok<“comment”>;

4. Operators are defined using the Punctuator class, as in this example:

def : Punctuator<“plus”, “+”>;
def : Punctuator<“minus”, “-“>;

5. Keywords need to use different flags:

def kw_auto: Keyword<“auto”, [KEYALL]>;
def kw_inline: Keyword<“inline”, [KEYC99,KEYCXX,KEYGNU]>;
def kw_restrict: Keyword<“restrict”, [KEYC99]>;

6. And last, here’s the definition of the keyword filter:

def : TokenFilter<[kw_auto, kw_inline, kw_restrict]>;

Of course, this file does not include all tokens from C and C++. However, it demonstrates all possible usages of the defined TableGen classes.
Based on these TableGen files, you’ll implement a TableGen backend in the next section.

In the previous section, you defined records in the TableGen language. To make use of those records, you need to write your own TableGen backend that can produce C++ source code or do other things using the records as input.

In Chapter 3, Turning the Source File into an Abstract Syntax Tree, the implementation of the Lexer class uses a database file to define tokens and keywords. Various query functions make use of that database file. Besides that, the database file is used to implement a keyword filter. The keyword filter is a hash map, implemented using the llvm::StringMap class. Whenever an identifier is found, the keyword filter is called to find out if the identifier is actually a keyword. If you take a closer look at the implementation using the ppprofiler pass from Chapter 6, Advanced IR Generation, then you will see that this function is called quite often. Therefore, it may be useful to experiment with different implementations to make that functionality as fast as possible.

However, this is not as easy as it seems. For example, you can try to replace the lookup in the hash map with a binary search. This requires that the keywords in the database file are sorted. Currently, this seems to be the case, but during development, a new keyword might be added in the wrong place undetected. The only way to make sure that the keywords are in the right order is to add some code that checks the order at runtime.

You can speed up the standard binary search by changing the memory layout. For example, instead of sorting the keywords, you can use the Eytzinger layout, which enumerates the search tree in breadth-first order. This layout increases the cache locality of the data and therefore speeds up the search. Personally speaking, maintaining the keywords in breadth-first order manually in the database file is not possible.

Another popular approach for searching is the generation of minimal perfect hash functions. If you insert a new key into a dynamic hash table such as llvm::StringMap, then that key might be mapped to an already occupied slot. This is called a key collision. Key collisions are unavoidable, and many strategies have been developed to mitigate that problem. However, if you know all the keys, then you can construct hash functions without key collisions. Such hash functions are called perfect. In case they do not require more slots than keys, then they are called minimal. Perfect hash functions can be generated efficiently – for example, with the gperf GNU tool.

In summary, there is some incentive to be able to generate a lookup function from keywords. So, let’s move the database file to TableGen!

In some cases, using a multiclass like in the previous example can lead to repetitions. Assume that the CPU also supports memory operands, in a way similar to immediate operands. You can support this by adding a new record definition to the multiclass:

multiclass InstWithOps {
def “”: Inst;
def “I”: Inst;
def “M”: Inst;
}

This is perfectly fine. But now, imagine you do not have 3 but 16 records to define, and you need to do this multiple times. A typical scenario where such a situation can arise is when the CPU supports many vector types, and the vector instructions vary slightly based on the used type.
Please note that all three lines with the def statement have the same structure. The variation is only in the suffix of the name and of the mnemonic, and the delta value is added to the opcode. In C, you could put the data into an array and implement a function that returns the data based on an index value. Then, you could create a loop over the data instead of manually repeating statements.
Amazingly, you can do something similar in the TableGen language! Here is how to transform the example:

1. To store the data, you define a class with all required fields. The class is called InstDesc, because it describes some properties of an instruction:

class InstDesc {
string Name = name;
string Suffix = suffix;
int Delta = delta;
}

2. Now, you can define records for each operand type. Note that it exactly captures the differences observed in the data:

def RegOp : InstDesc<“”, “”, 0>;
def ImmOp : InstDesc<“I”, “””, 1>;
def MemOp : InstDesc””””,””””, 2>;

3. Imagine you have a loop enumerating the numbers 0, 1, and 2, and you want to select one of the previously defined records based on the index. How can you do this? The solution is to create a getDesc class that takes the index as a parameter. It has a single field, ret, that you can interpret as a return value. To assign the correct value to this field, the !cond operator is used:

class getDesc {
InstDesc ret = !cond(!eq(n, 0) : RegOp,
!eq(n, 1) : ImmOp,
!eq(n, 2) : MemOp);
}

This operator works similarly to a switch/case statement in C.

4. Now, you are ready to define the multiclass. The TableGen language has a loop statement, and it also allows us to define variables. But remember that there is no dynamic execution! As a consequence, the loop range is statically defined, and you can assign a value to a variable, but you cannot change that value later. However, this is enough to retrieve the data. Please note how the use of the getDesc class resembles a function call. But there is no function call! Instead, an anonymous record is created, and the values are taken from that record. Lastly, the past operator () performs a string concatenation, similar to the !strconcat operator used earlier:

multiclass InstWithOps {
foreach I = 0-2 in {
defvar Name = getDesc.ret.Name;
defvar Suffix = getDesc.ret.Suffix;
defvar Delta = getDesc.ret.Delta;
def Name: Inst;
}
} Now, you use the multiclass as before to define records: defm ADD : InstWithOps<“add”, 0xA0>; Please run llvm-tblgen and examine the records. Besides the various ADD records, you will also see a couple of anonymous records generated by the use of the getDesc class.
This technique is used in the instruction definition of several LLVM backends. With the knowledge you have acquired, you should have no problem understanding those files.
The foreach statement used the syntax 0-2 to denote the bounds of the range. This is called a range piece. An alternative syntax is to use three dots (0…3), which is useful if the numbers are negative. Lastly, you are not restricted to numerical ranges; you can also loop over a list of elements, which allows you to use strings or previously defined records. For example, you may like the use of the foreach statement, but you think that using the getDesc class is too complicated. In this case, looping over the InstDesc records is the solution: multiclass InstWithOps {
foreach I = [RegOp, ImmOp, MemOp] in {
defvar Name = I.Name;
defvar Suffix = I.Suffix;
defvar Delta = I.Delta;
def Name: Inst;
}
} So far, you only defined records in the TableGen language, using the most commonly used statements. In the next section, you’ll learn how to generate C++ source code from records defined in the TableGen language.

Another often-used statement is multiclass. A multiclass allows you to define multiple records at once. Let’s expand the example to show why this can be useful.
The definition of an add instruction is very simplistic. In reality, a CPU often has several add instructions. A common variant is that one instruction has two register operands while another instruction has one register operand and an immediate operand, which is a small number. Assume that for the instruction having an immediate operand, the designer of the instruction set decided to mark them with i as a suffix. So, we end up with the add and addi instructions. Further, assume that the opcodes differ by 1. Many arithmetic and logical instructions follow this scheme; therefore, you want the definition to be as compact as possible.
The first challenge is that you need to manipulate values. There is a limited number of operators that you can use to modify a value. For example, to produce the sum of 1 and the value of the field opcode, you write:

!add(opcode, 1)

Such an expression is best used as an argument for a class. Testing a field value and then changing it based on the found value is generally not possible because it requires dynamic statements that are not available. Always remember that all calculations are done while the records are constructed!
In a similar way, strings can be concatenated:

!strconcat(mnemonic,”i”)

Because all operators begin with an exclamation mark (!), they are also called bang operators. You find a full list of bang operators in the Programmer’s Reference: https://llvm.org/docs/TableGen/ProgRef.htmlappendix-a-bang-operators.
Now, you can define a multiclass. The Inst class serves again as the base:

class Inst {
string Mnemonic = mnemonic;
int Opcode = opcode;
}

The definition of a multiclass is a bit more involved, so let’s do it in steps:

1. The definition of a multiclass uses a similar syntax to classes. The new multiclass is named InstWithImm and has two parameters, mnemonic and opcode:

multiclass InstWithImm {

2. First, you define an instruction with two register operands. As in a normal record definition, you use the def keyword to define the record, and you use the Inst class to create the record content. You also need to define an empty name. We will explain later why this is necessary: def “”: Inst;
3. Next, you define an instruction with the immediate operand. You derive the values for the mnemonic and the opcode from the parameters of the multiclass, using bang operators. The record is named I: def I: Inst;
4. That is all; the class body can be closed, like so:

}

To instantiate the records, you must use the defm keyword:

defm ADD : InstWithImm<“add”, 0xA0>;

These statements result in the following:

1. The Inst<“add”, 0xA0> record is instantiated. The name of the record is the concatenation of the name following the defm keyword and of the name following def inside the multiclass statement, which results in the name ADD.
2. The Inst<“addi”, 0xA1> record is instantiated and, following the same scheme, is given the name ADDI.
   Let’s verify this claim with llvm-tblgen:

$ llvm-tblgen –print-records inst.td
————- Classes —————–
class Inst {
string Mnemonic = Inst:mnemonic;
int Opcode = Inst:opcode;
}
————- Defs —————–
def ADD { // Inst
string Mnemonic = “add”;
int Opcode = 160;
}
def ADDI { // Inst
string Mnemonic = “addi”;
int Opcode = 161;
}

Using a multiclass, it is very easy to generate multiple records at once. This feature is used very often!
A record does not need to have a name. Anonymous records are perfectly fine. Omitting the name is all you need to do to define an anonymous record. The name of a record generated by a multiclass is made up of two names, and both names must be given to create a named record. If you omit the name after defm, then only anonymous records are created. Similarly, if the def inside the multiclass is not followed by a name, an anonymous record is created. This is the reason why the first definition in the multiclass example used the empty name “”: without it, the record would be anonymous.

To get a more detailed dump of the records, you can use the –-print-detailed-records option. The output includes the line numbers of record and class definitions, and where record fields are initialized. They can be very helpful if you try to track down why a record field was assigned a certain value.
In general, the ADD and SUB instructions have a lot in common, but there is also a difference: addition is a commutative operation but subtraction is not. Let’s capture that fact in the record, too. A small challenge is that TableGen only supports a limited set of data types. You already used string and int in the examples. The other available data types are bit, bits, list, and dag. The bit type represents a single bit; that is, 0 or 1. If you need a fixed number of bits, then you use the bits type. For example, bits<5> is an integer type 5 bits wide. To define a list based on another type, you use the list type. For example, list is a list of integers, and list is a list of records of the Inst class from the example. The dag type represents directed acyclic graph (DAG) nodes. This type is useful for defining patterns and operations and is used extensively in LLVM backends.
To represent a flag, a single bit is sufficient, so you can use one to mark an instruction as commutable. The majority of instructions are not commutable, so you can take advantage of default values:

class Inst {
string Mnemonic = mnemonic;
int Opcode = opcode;
bit Commutable = commutable;
}
def ADD : Inst<“add”, 0xA0, 1>;
def SUB : Inst<“sub”, 0xB0>;

You should run llvm-tblgen to verify that the records are defined as expected.
There is no requirement for a class to have parameters. It is also possible to assign values later. For example, you can define that all instructions are not commutable:

class Inst {
string Mnemonic = mnemonic;
int Opcode = opcode;
bit Commutable = 0;
}
def SUB : Inst<“sub”, 0xB0>;

Using a let statement, you can overwrite that value:

let Commutable = 1 in
def ADD : Inst<“add”, 0xA0>;

Alternatively, you can open a record body to overwrite the value:

def ADD : Inst<“add”, 0xA0> {
let Commutable = 1;
}

Again, please use llvm-tblgen to verify that the Commutable flag is set to 1 in both cases.
Classes and records can be inherited from multiple classes, and it is always possible to add new fields or overwrite the value of existing fields. You can use inheritance to introduce a new CommutableInst class:

class Inst {
string Mnemonic = mnemonic;
int Opcode = opcode;
bit Commutable = 0;
}
class CommutableInst
: Inst {
let Commutable = 1;
}
def SUB : Inst<“sub”, 0xB0>;
def ADD : CommutableInst<“add”, 0xA0>;

The resulting records are always the same, but the language allows you to define records in different ways. Please note that, in the latter example, the Commutable flag may be superfluous: the code generator can query a record for the classes it is based on, and if that list contains the CommutableInst class, then it can set the flag internally.

Let’s define a simple record for an instruction:

def ADD {
string Mnemonic = “add”;
int Opcode = 0xA0;
}

The def keyword signals that you define a record. It is followed by the name of the record. The record body is surrounded by curly braces, and the body consists of field definitions, similar to a structure in C++.
You can use the llvm-tblgen tool to see the generated records. Save the preceding source code in an inst.td file and run the following:

$ llvm-tblgen –print-records inst.td
————- Classes —————–
————- Defs —————–
def ADD {
string Mnemonic = “add”;
int Opcode = 160;
}

This is not yet exciting; it only shows the defined record was parsed correctly.
Defining instructions using single records is not very comfortable. A modern CPU has hundreds of instructions, and with this amount of records, it is very easy to introduce typing errors in the field names. And if you decide to rename a field or add a new field, then the number of records to change becomes a challenge. Therefore, a blueprint is needed. In C++, classes have a similar purpose, and in TableGen, it is also called a class. Here is the definition of an Inst class and two records based on that class:

class Inst {
string Mnemonic = mnemonic;
int Opcode = opcode;
}
def ADD : Inst<“add”, 0xA0>;
def SUB : Inst<“sub”, 0xB0>;

The syntax for classes is similar to that of records. The class keyword signals that a class is defined, followed by the name of the class. A class can have a parameter list. Here, the Inst class has two parameters, mnemonic and opcode, which are used to initialize the records’ fields. The values for those fields are given when the class is instantiated. The ADD and SUB records show two instantiations of the class. Again, let’s use llvm-tblgen to look at the records:

$ llvm-tblgen –print-records inst.td
————- Classes —————–
class Inst {
string Mnemonic = Inst:mnemonic;
int Opcode = Inst:opcode;
}
————- Defs —————–
def ADD { // Inst
string Mnemonic = “add”;
int Opcode = 160;
}
def SUB { // Inst
string Mnemonic = “sub”;
int Opcode = 176;
}

Now, you have one class definition and two records. The name of the class used to define the records is shown as a comment. Please note that the arguments of the class have the default value ?, which indicates int is uninitialized.

LLVM comes with its own domain-specific language (DSL) called TableGen. It is used to generate C++ code for a wide range of use cases, thus reducing the amount of code a developer has to produce. The TableGen language is not a full-fledged programming language. It is only used to define records, which is a fancy word for a collection of names and values. To understand why such a restricted language is useful, let’s examine two examples.

Typical data you need to define one machine instruction of a CPU is:

• The mnemonic of the instruction
• The bit pattern
• The number and types of operands
• Possible restrictions or side effects

It is easy to see that this data can be represented as a record. For example, a field named asmstring could hold the value of the mnemonic; say, “add”. Also, a field named opcode could hold the binary representation of the instruction. Together, the record would describe an additional instruction. Each LLVM backend describes the instruction set in this way.

Records are such a general concept that you can describe a wide variety of data with them. Another example is the definition of command-line options. A command-line option:

• Has a name
• May have an optional argument
• Has a help text
• May belong to a group of options

Again, this data can be easily seen as a record. Clang uses this approach for the command-line options of the Clang driver.

The TableGen language

In LLVM, the TableGen language is used for a variety of tasks. Large parts of a backend are written in the TableGen language; for example, the definition of a register file, all instructions with mnemonic and binary encoding, calling conventions, patterns for instruction selection, and scheduling models for instruction scheduling. Other uses of LLVM are the definition of intrinsic functions, the definition of attributes, and the definition of command-line options.

You’ll find the Programmer’s Reference at https://llvm.org/docs/TableGen/ProgRef.html and the Backend Developer’s Guide at https://llvm.org/docs/TableGen/BackGuide.html.

To achieve this flexibility, the parsing and the semantics of the TableGen language are implemented in a library. To generate C++ code from the records, you need to create a tool that takes the parsed records and generates C++ code from it. In LLVM, that tool is called llvm-tblgen, and in Clang, it is called clang-tblgen. Those tools contain the code generators required by the project. But they can also be used to learn more about the TableGen language, which is what we’ll do in the next section.

Experimenting with the TableGen language

Very often, beginners feel overwhelmed by the TableGen language. But as soon as you start experimenting with the language, it becomes much easier.

To allow the user to add passes at every extension point, you need to add the preceding code snippet for each extension point.

3. Now is a good time to try out the different pass manager options. With the –debug-pass-manager option, you can follow which passes are executed in which order. You can also print the IR before or after each pass, which is invoked with the –print-before-all and –print-after-all options. If you created your own pass pipeline, then you can insert the print pass in points of interest. For example, try the –passes=”print,inline,print” option. Furthermore, to identify which pass changes the IR code, you can use the –print-changed option, which will only print the IR code if it has changed compared to the result from the pass before. The greatly reduced output makes it much easier to follow IR transformations.
   

The PassBuilder class has a nested OptimizationLevel class to represent the six different optimization levels. Instead of using the “default<O?>” pipeline description as an argument to the parsePassPipeline() method, we can also call the buildPerModuleDefaultPipeline() method, which builds the default optimization pipeline for the request level – except for level O0. This optimization level means that no optimization is performed.

Consequently, no passes are added to the pass manager. If we still want to run a certain pass, then we can add it to the pass manager manually. A simple pass to run at this level is the AlwaysInliner pass, which inlines a function marked with the always_inline attribute into the caller. After translating the command-line option value for the optimization level into the corresponding member of the OptimizationLevel class, we can implement this as follows:
    PassBuilder::OptimizationLevel Olevel = …;
    if (OLevel == PassBuilder::OptimizationLevel::O0)
      MPM.addPass(AlwaysInlinerPass());
    else
      MPM = PB.buildPerModuleDefaultPipeline(OLevel, DebugPM);

Of course, it is possible to add more than one pass to the pass manager in this fashion. PassBuilder also uses the addPass() method when constructing the pass pipeline.

Running extension point callbacks

Because the pass pipeline is not populated for optimization level O0, the registered extension points are not called. If you use the extension points to register passes that should also run at O0 level, this is problematic. You can call the runRegisteredEPCallbacks() method to run the registered extension point callbacks, resulting in a pass manager populated only with the passes that were registered through the extension points.

By adding the optimization pipeline to tinylang, you created an optimizing compiler similar to clang. The LLVM community works on improving the optimizations and the optimization pipeline with each release. Due to this, it is very seldom that the default pipeline is not used. Most often, new passes are added to implement certain semantics of the programming language.

Summary

In this chapter, you learned how to create a new pass for LLVM. You ran the pass using a pass pipeline description and an extension point. You extended your compiler with the construction and execution of a pass pipeline similar to clang, turning tinylang into an optimizing compiler. The pass pipeline allows the addition of passes at extension points, and you learned how you can register passes at these points. This allows you to extend the optimization pipeline with your developed passes or existing passes.

In the next chapter, you will learn the basics of the TableGen language, which is used extensively in LLVM and clang to significantly reduce manual programming.