Camil Demetrescu edited osr-llvm.tex  over 8 years ago

Commit id: 00b8a6a1cf560c88d803583b48b7b7bd696c2469

deletions | additions      

       

In this section we discuss our implementation of the approach described in \mysection\ref{se:overview} in \tinyvm, a proof-of-concept virtual machine we developed as a playground to exercise our OSR techniques. TinyVM is based on LLVM's MCJIT compiler and supports interactive invocation of LLVM IR functions either generated at run-time or loaded from disk. The main design goal behind TinyVM is the creation of an interactive environment for IR manipulation and JIT-compilation of functions: for instance, it allows the user to insert OSR points in loaded functions, run optimization passes on them or display their CFGs, repeatedly invoke a function for a specified amount of times and so on. TinyVM supports dynamic library loading and linking, and comes with a helper component for MCJIT that simplifies tasks such as handling multiple IR modules, symbol resolution in presence of multiple versions of a function, and tracking native code and other machine-level generated object such as Stackmaps.  \subsection{Example}  To explain how \tinyvm\ implements in LLVM the OSR approach of \mysection\ref{se:overview}, we consider the simple example of \myfigure\ref{fi:isord-example}. Function {\tt isord} checks whether an array of numbers is sorted according to some ordering specified by a comparator. The scenario we explore is profile-driven optimization, where we dynamically divert control to a faster version if the number of iterations exceeds a certain threshold.   \paragraph{OSR Instrumentation.}  We use deferred compilation by instrumenting {\tt isord} in \tinyvm\ with an open OSR at the beginning of the loop body, as shown in \myfigure\ref{fig:isordfrom}. Portions added to the original code by OSR istrumentation are highlighted in grey.  %The figure illustrates how the original {\tt isord} code is instrumented by \tinyvm, highlighting in grey the added portions.   A new basic block is placed at the beginning of the loop body, incrementing the hotness counter {\tt p.osr} and jumping to an OSR-firing block if the counter reaches the threshold (1000 iterations in this example). The OSR block contains a tail call to the target generation stub, which receives as parameters the four live variables at the OSR point ({\tt v}, {\tt n}, {\tt i}, {\tt c}). Notice that maintaining the SSA form requires adjusting phi nodes. The stub (see \myfigure[...]) calls a code generator that: 1) builds an optimized version of {\tt isord} by inlining the comparator, and 2) uses it to create the continuation function {\tt isordto} shown in \myfigure\ref{fig:isordascto}. Notice that instrumentation replaces the function entry point, removes dead code, replaces live variables with the function parameters, and fixes phi nodes accordingly. Additions resulting from the IR instrumentation are in grey, while removals are struck-through.  \paragraph{x86-64 Lowering.}  [...]  \subsection{Instrumentation API.}  [...]  %To place an open OSR, \tinyvm\ requires the following pieces of information: 1) a pointer to the   %LLVMContext& Context, OSRLibrary::OpenOSRInfo& info,  % OSRLibrary::OSRCond& cond, Value* profDataVal, OSRLibrary::DestFunGenerator destFunGenerator,  % std::vector *valuesToTransfer, OSRLibrary::OSRPointConfig &config  %Function* src, BasicBlock* src_bb, std::string* F1NewName, OSRLibrary::OSRCond &cond, int branchTakenProb  %OSR instrumentation produces function {\tt isordfrom} obtained from {\tt isord} by including   %includes extra code (in grey) that increments a counter ,   %The generated stub calls a function inliner that generates a version of {\tt isord} where the comparator's body, pointed to by live variable {\tt c_osr}, is inlined in the loop body. The optimized version {\tt isordto} is shown in \myfigure\ref{fig:isordascto}.  \ifdefined\noauthorea  \begin{figure}[t]  \begin{center} 

\end{figure}  \fi  \subsection{Example}  To explain how \tinyvm\ implements in LLVM the OSR approach of \mysection\ref{se:overview}, we consider the simple example of \myfigure\ref{fi:isord-example}. Function {\tt isord} checks whether an array of numbers is sorted according to some ordering specified by a comparator. The scenario we explore is profile-driven optimization, where we dynamically divert control to a faster version if the number of iterations exceeds a certain threshold.  \ifdefined\noauthorea  \begin{figure}[t]  \begin{center} 

\end{figure}  \fi  \paragraph{OSR Instrumentation.}  We use deferred compilation by instrumenting {\tt isord} in \tinyvm\ with an open OSR at the beginning of the loop body, as shown in \myfigure\ref{fig:isordfrom}. Portions added to the original code by OSR istrumentation are highlighted in grey.  %The figure illustrates how the original {\tt isord} code is instrumented by \tinyvm, highlighting in grey the added portions.   A new basic block is placed at the beginning of the loop body, incrementing the hotness counter {\tt p.osr} and jumping to an OSR-firing block if the counter reaches the threshold (1000 iterations in this example). The OSR block contains a tail call to the target generation stub, which receives as parameters the four live variables at the OSR point ({\tt v}, {\tt n}, {\tt i}, {\tt c}). Notice that maintaining the SSA form requires adjusting phi nodes. The stub (see \myfigure[...]) calls a code generator that: 1) builds an optimized version of {\tt isord} by inlining the comparator, and 2) uses it to create the continuation function {\tt isordto} shown in \myfigure\ref{fig:isordascto}. The stub terminates with a tail call to {\tt isordto}. Notice that OSR instrumentation replaces the function entry point, removes dead code, replaces live variables with the function parameters, and fixes phi nodes accordingly. Additions resulting from the IR instrumentation are in grey, while removals are struck-through.  \ifdefined\noauthorea  \begin{figure}[t]  \begin{center} 

\end{figure}  \fi  \paragraph{x86-64 Lowering.}  [...]  \subsection{Instrumentation API.}  To support the IR instrumentation tasks of \mysection\section{}, \tinyvm\ provides a number of abstractions for VM builders:  \begin{itemize}  \item [...]  \end{itemize}  %To place an open OSR, \tinyvm\ requires the following pieces of information: 1) a pointer to the   %LLVMContext& Context, OSRLibrary::OpenOSRInfo& info,  % OSRLibrary::OSRCond& cond, Value* profDataVal, OSRLibrary::DestFunGenerator destFunGenerator,  % std::vector *valuesToTransfer, OSRLibrary::OSRPointConfig &config  %Function* src, BasicBlock* src_bb, std::string* F1NewName, OSRLibrary::OSRCond &cond, int branchTakenProb  %OSR instrumentation produces function {\tt isordfrom} obtained from {\tt isord} by including   %includes extra code (in grey) that increments a counter ,   %The generated stub calls a function inliner that generates a version of {\tt isord} where the comparator's body, pointed to by live variable {\tt c_osr}, is inlined in the loop body. The optimized version {\tt isordto} is shown in \myfigure\ref{fig:isordascto}.  %\subsection{Resolved OSR Points}  %\subsection{Open OSR Points}