TRACED: Execution-aware Pre-training for Source Code

We introduce TRACED, an execution-aware pre-training strategy for source code. Specifically, we pre-train code language models with a combination of source code, executable inputs, and corresponding execution traces. We fine-tune and evaluate TRACED on three downstream tasks: static execution estimation, clone retrieval, and vulnerability detection.

• TRACED relatively improves the statically pre-trained code models by 12.4% for complete execution path prediction and by 25.2% for runtime variable value predictions.
• TRACED also significantly outperforms statically pre-trained models in clone retrieval and vulnerability detection across four public benchmarks.