Just in Time Compiler

Before programming languages, people used to write machine code. You would enter a set of commands for the computer to execute. There were no functions, no if statements, no classes. Then a couple of smart people started thinking of other ways to program.

They noticed they write some sets of instructions over and over again. The code for a conditional jump (AKA an if statement) could be written more concisely. They invented a shorter way to write their code, and so, a programming language was born. They then made a second program that will translate their shorthand into the actual machine code. A compiler was born.

Compiler or Interpreter?

Both a compiler and interpreter translate your code into code that the computer understands. The difference is in when they do it.

Compilers take your blob of code, and translate it into a new file. This new file is ready to be executed, which is why it's called an executable file.

Interpreters are different. They read your code while the program is running, and translate it into machine code on the fly. In compilers, you have to compile the whole program before it runs, but an interpreter continuously translates code while running.

This means that interpreters often lead to slower programs than compilers. A compiler can read all your code at once, giving it a lot of opportunities to optimize your code. It's very hard to make an interpreter run as fast a compiler.

But interpreters have their advantages. The time between writing and running the program is almost instant, because there is no compiling to do ahead of time. Because the code can be change on the fly, it's easier to implement dynamic typing, meta-programming and other dynamic features with an interpreter. Interpreted programs are easier to debug, since you're working directly with the program, and not machine code.

Lots of programmers swear by compilers and the safety it gives them. Others love how productive they can be with interpreters. Should you use a compiler or an interpreter? The answer in a lot of languages is "Why not both?".

Compiler and Interpreter

The best way to deal with trade-offs is to take the pros of both options, without taking the cons. That's what JIT compilers do. JIT stands for just-in-time, because these compilers work kind of like interpreters, they don't compile ahead of time but instead while you're running the program.

The difference between JIT compilers and interpreters is that once something is compiled, the translated code is cached. If your program is using that piece of code twice, the compiler can just refer to the already compiled line of code.

The reason this works really well is that programs tend to go over lines of code multiple times. Loops, like for and while, will go over their blocks a bunch of times. Or you might have a few functions which are used all over the place. Compiling that function only once saves the compiler a bunch of time.

The result is a compiler which gives you most of the benefits of interpretation, (mostly) without the tradeoffs. Because there's very little compilation ahead of time, your program runs instantly. Because the results of the compilation are cached, the impact on performance is minimal.

The way most JIT compilers work is that your code is compiled ahead of time into something called byte code. Byte code is like machine code, with one key advantage. Each CPU has its own set of machine code instructions. This means that machine code compiled for a Samsung phone most likely won't work on your PC. On the other hand, byte code is the same on all platforms. But you still need to compile byte code into real machine code. Thankfully, since the languages are similar, that process is very fast.

Most languages are compiled to byte code or some other form of cross-platform asseembly-like language before they're compiled into the actual machine code for the platform they run on.

Once your program is compiled into byte code, it starts running. While running, sections of the byte code will be compiled into machine code, and the results cached.

Some JIT compilers use techniques where they look at your code while it's running, and detect which parts of code you are running the most. They then rearrange things or optimize that chunk of code so it runs the fastest, without having to optimize the rest of the code.

If you read a couple of PWOTD articles you'll notice one theme: optimization opportunities are proportional to the amount of information a compiler has about your code. For instance, it's easier to optimize statically typed languages, because the compiler has a lot of information.

JIT compilers can, in theory, be faster than ahead of time compiling, because they have another piece of information: how and where is your code run. For instance, they can detect which hardware your program is being run on, and make platform-specific optimizations. They can see how much memory you have and know how much they can keep in the RAM. Or, they see that there's a few lines of code you're running over and over, and make sure those lines are as optimized as they can be.

A JIT compiler that optimizes most-used lines of code is called a tracing JIT and is used in lots of languages including Java and JavaScript.

Almost all mainstream languages have some form of a JIT compiler, sometimes multiple different ones. For instance, in Firefox, Javascript dynamically selects from three different kinds of compilers, based on how much you call certain lines of code. Java has its HotSwap virtual machine that uses tracing JIT. So does PyPy, an alternative implementation of Python.

Always remember, if you have to make a hard choice, there's probably a middle way.

References

A crash course in (JIT) compilers (Lin Clark, Mozzila)
https://hacks.mozilla.org/2017/02/a-crash-course-in-just-in-time-jit-compilers/

HotSpot - Wikipedia
https://en.wikipedia.org/wiki/HotSpot

JIT Compilation - Wikipedia
https://en.wikipedia.org/wiki/Just-in-time_compilation

Tracing JIT - Wikipedia
https://en.wikipedia.org/wiki/Tracing_just-in-time_compilation

Recursion & Tail Call Optimization

Stack Based Programming