Skip to content

L3‐LOC User's Guide

Jump to bottom
Aditya P. Gurajada edited this page May 3, 2024 · 14 revisions

What is L3 and LOC?

L3 is a standalone high-performance mmap()'ed based logging library that can be used with C/C++ programs

LOC is an independent Line-Of-Code tracking library also designed for C/C++ programs.

L3 is integrated with LOC to generate log-entries capturing the exact line-of-code where the log-entry was generated.

Simple C-APIs are provided that can be used in user-applications. Python scripts are provided to generate required source / header files and to dump L3-log dumps.

Sample Outputs

A sample output from a unit-test in this repo for L3-logging is shown below.

---- Run L3 unit-tests: ----
./build/release/bin/unit/l3_dump.py-test
Generated fast log-entries to log-file: /tmp/l3.c-fast-unit-test.dat
Generated slow log-entries to log-file: /tmp/l3.c-small-unit-test.dat

Executing the unit-test binary, ./build/release/bin/unit/l3_dump.py-test, generates two L3-log files. At run-time, very compact logging is performed, each log-entry consuming exactly 32-bytes. Currently, we support C-printf()-style messages, with 2 64-bit arguments.

Dumping one of the files unpacks the message logged, substituting the argument-values, to produce this output:

$ python3 l3_dump.py --log-file /tmp/l3.c-small-unit-test.dat --binary ./build/release/bin/unit/l3_dump.py-test

tid=103141 'Simple-log-msg-Args(arg1=1, arg2=2)'
tid=103141 'Simple-log-msg-Args(arg3=3, arg4=4)'
tid=103141 'Potential memory overwrite (addr=0xdeadbabe, size=1024)'
tid=103141 'Invalid buffer handle (addr=0xbeefabcd), lockrec=0x0'
Unpacked nentries=4 log-entries.

The workflow is similar when L3-LOC-logging is performed.

The application has to be re-built with the L3_LOC_ENABLED={1,2} environment variable, to generate the LOC-ID at compile-time, which gets logged as part of the L3-logging call. (Certain Makefile-build steps, discussed further below, have to be added to your project to include the LOC-specific files as part of the build.)

Dumping the L3-log-file produces this output, where the LOC-ID is expanded to the source-location where the logging was performed:

$ python3 l3_dump.py --log-file /tmp/l3.c-small-test.dat --binary ./build/release/bin/use-cases/single-file-C-program

tid=103253 single-file-C-program/test-main.c:76  'Simple-log-msg-Args(arg1=1, arg2=2)'
tid=103253 single-file-C-program/test-main.c:77  'Potential memory overwrite (addr=0xdeadbabe, size=1024)'
tid=103253 single-file-C-program/test-main.c:78  'Invalid buffer handle (addr=0xbeefabcd), refcount=0'
tid=103253 single-file-C-program/test-main.c:79  'Fast-logging msg1=10, addr=0xdeadbeef'
tid=103253 single-file-C-program/test-main.c:80  'Fast-logging msg2=20, addr=0xbeefbabe'
Unpacked nentries=5 log-entries.

The dumping utility, l3_dump.py unpacks the unique LOC-ID field to produce the source-code information, as seen above.

Typical Use-cases

There are several use-cases where L3 logging, either independently or enhanced with LOC-tracking, can be used. Some examples are:

  • Instrumentation of high-performance computing applications to track & debug race conditions
  • Troubleshooting of concurrent programming systems with light-weight logging
  • Logging of resource allocation / acquisition events (e.g., memory allocation, lock acquistion and release etc.)
  • Timing metrics, e.g., of RPCs, inter-process communication, round-trip of client/server communication etc.

Interfaces

Following is an evolving set of interfaces for these packages.

L3 APIs

A small set of interfaces are provided:

  • int l3_init(const char *mmap_file_path)
  • void l3_log_simple(const char *msg, const uint64_t arg1, const uint64_t arg2)
  • void l3_log_fast(const char *msg, const uint64_t arg1, const uint64_t arg2)

There are plans to enhance these as follows:

  • Provide l3_deinint(), to disable logging and flush the mmap()-ed file
  • Provide l3_stop() and l3_resume() to pause and restart L3-logging
  • Simplify logging interfaces to l3_log() and l3_log_fast()

LOC APIs

The interfaces for use in your programs and project Makefiles are:

  • __LOC__ - Generates a unique 4-byte LOC-ID (code-location ID).

    • Use it in a manner similar to __FILE__ or __LINE__ macros, except that typically you would store the generated LOC-ID in some structure, to record the code-location.

  • LOC_FILE(loc_id), LOC_LINE(loc_id), LOC_FUNC(loc_id) - Interfaces to unpack the LOC-ID to its constituent parts.

    • Typically, this unpacking will be done as part of instrumentation / diagnostics. Some examples where this kind of unpacking can be done is to detect, say, resource-leaks, or as part of exception processing, or deadlock chain detection.

  • LineOfCode/loc/gen_loc_files.py - Python script to generate required loc.h and loc.c files, defining the above-specified LOC-interfaces

  • LineOfCode/include/loc.h and LineOfCode/src/loc.c - Provide interfaces for LOC-ELF encoding, without the need for additional Python-generated files

How to use the L3 and LOC packages in your application

Clone this wiki locally