Rewriting the Rules: The First Windows Kernel Driver Built in Nim

The Windows kernel is the heart of the operating system, a realm where precision and stability are paramount. Traditionally dominated by C and C++, kernel development is not for the faint of heart—making my decision to create the first-ever Windows kernel driver in Nim an exciting and ambitious challenge (for my holidays).

Goal: "Scripting" in Kernel-land

Nim offers a syntax reminiscent of a blend between JavaScript and Python, packed with powerful features. It boasts a highly customizable garbage collector (or the option to go without one entirely), compile-time evaluation of custom functions, and the ability to compile directly to C ;)

Most important: Making it compile!

Nim includes a tool called vccexe.exe (admittedly a peculiar name) that can compile Windows programs out of the box. However, for this project, I wanted to replicate the exact compiler and linker flags from my Hello World Windows kernel driver. Fortunately, Nim allows us to customize this easily by modifying the nim/config/nim.cfg file to specify cl.exe, link.exe, and the necessary flags:

vcc.exe = "cl.exe"
vcc.linkerexe = "link.exe"
vcc.options.always = ".."
vcc.options.linker= ".."        

Now if we run "vcvarsall.bat" inside our visual studio folder, it should setup our building environment and we are good to go.

However we need to disable some features which aren't available inside the kernel, so we adjust our local "configs.nim":

--os:standalone
--stdout:off
--noMain
--tlsEmulation:off
--mm:arc
--threads:off
--cc:vcc
--overflowChecks:off
--define:useMalloc        

We’re stripping out many features, but we’re keeping the garbage collector with Automatic Reference Counting (ARC) for its strong performance in hard real-time systems. Additionally, we specify useMalloc to retain essential utilities like string handling and related functions.

At this point, you’ll likely encounter errors indicating that malloc, free, and similar functions are missing. This is expected, as we don’t have access to the kernel32 library in kernel-land. Fortunately, we can easily replace these functions to work within the kernel environment:

proc ExAllocatePoolWithTag(PoolType: int32, NumberOfBytes: uint64, Tag: uint32): pointer 
    {.importc }
  
proc ExFreePoolWithTag(P: pointer, Tag: uint32) 
    {.importc }

proc RtlCopyMemory(Destination: pointer, Source: pointer, Length: uint64) 
    {.importc }

proc RtlZeroMemory(Destination: pointer, Length: uint64) 
    {.importc }

const 
    # NonPagedPoolNx is recommended on newer systems
    # (or use NonPagedPool if on older Windows versions)
    PoolType = 0x0  # Typically NonPagedPool = 0; NonPagedPoolNx = 0x200
    PoolTag  = 0x41414141'u  # Use a custom four-byte tag (e.g., 'AAAA')

type
  const_pointer {.importc: "const void *".} = pointer

  # Replace standard CRT memory functions
proc malloc*(size: csize_t): pointer {.exportc.} =
    var actualSize = size
    if actualSize == 0:
      # Allocation of 0 bytes is undefined, allocate at least 1 byte
      actualSize = 1
    result = ExAllocatePoolWithTag(PoolType, uint64(actualSize), uint32(PoolTag))
    # result will be nil on failure, which is consistent with malloc

proc free*(p: pointer) {.exportc.} =
    if p != nil:
      ExFreePoolWithTag(p, uint32(PoolTag))

proc calloc*(num: csize_t, size: csize_t): pointer {.exportc.} =
    let total = num * size
    result = malloc(total)
    if result != nil:
      RtlZeroMemory(result, uint64(total))

proc realloc*(p: pointer, new_size: csize_t): pointer {.exportc.} =
    # realloc semantics: allocate new block, copy old data, free old block
    let newPtr = malloc(new_size)
    if p != nil and newPtr != nil:
      # copy only min(old_size, new_size), but we don't know old_size here
      # For simplicity, assume caller ensures correct usage.
      # If needed, keep track of allocated sizes somewhere.
      RtlCopyMemory(newPtr, p, uint64(new_size))
      free(p)
    result = newPtr

  # Other functions that are meaningless in kernel mode can remain stubs.
proc fflush(stream: File): cint {.exportc.} =
    # No stdio in kernel; just return success.
    return 0.cint

proc exit(): pointer {.exportc.} =
    # Kernel code doesn't exit like user-mode apps
    discard
    return nil

proc fwrite(buf: const_pointer, size: csize_t, count: csize_t, stream: File): csize_t {.exportc.} =
    # No file I/O in kernel mode like this; just a stub.
    return 0.csize_t        

Testing the Waters: Does It Work?

Great. That was easy. Now for the fun part—let’s see if it actually works!

# Override echo function as well
proc echo(msg: string): void =
  var print = msg & "\n"
  discard DbgPrintEx(0,0,print)

proc Main(ctx: pointer) {.stdcall, exportc.} =
    echo("Main thread started")
    var test = @["Hello", "World"]
    for i in 0 ..< test.len:
      echo(test[i])
    echo(test.map(proc(x: string): string = x & " Nim").join(", "))

proc DriverEntry(driverObject: pointer, registryPath: pointer): int {.exportc.} =
    var handle: pointer
    if PsCreateSystemThread(addr handle, 0, nil, nil, nil, Main, nil) == 0:
      echo("Thread created successfully")
    discard ZwClose(handle)
    return 0        

And this should be our output:

Wrapping Up: Nim in Kernel-Land

Perfect. Everything is working as expected—no BSODs, no unexpected crashes. This project demonstrates that Nim, with its flexible features and performance capabilities, can hold its own even in the demanding environment of kernel development.

I hope this article inspires others to experiment with Nim or even embark on their own unconventional projects in kernel-land.