System Call

Written by: Iron Hulk Published: Jan 20, 2025 Reading time: Iron Hulk
Back to Blogs

بسم الله الرحمن الرحيم


What Is a System Call?

A system call is the mechanism by which a user-mode application requests a service from the operating system’s kernel. Whenever you open a file, allocate memory, create or terminate a process, or perform any I/O operation, your application invokes a system call. At a high level, the process goes like this: the application executes a special instruction that triggers a controlled switch from user mode to kernel mode. The kernel then validates the request, performs the requested operation on behalf of the application, and returns control (and any results) back to user space. System calls constitute a protected boundary: without them, user-mode code couldn’t directly manipulate hardware or critical OS data structures, preserving system stability and security.


Why care? Syscalls are the only legal way to touch hardware/memory. Malware abuses them. Security tools spy on them.

What Is a Systemcall Service Number?

A system call service number (SSN) is basically an integer code that acts as a shortcut for the operating system to figure out which kernel function to run when an app makes a request. Think of it like a phone extension so when you dial a number, it routes you to the right department (in this case, a specific kernel service). This number is used in direct system calls to tell the kernel exactly what operation to perform, without passing around function names, which keeps things fast and efficient.


Windows (SSDT Index)

The System Service Dispatch Table (SSDT) holds pointers to every kernel routine exposed via Nt/Zw calls. When you invoke a syscall stub in ntdll.dll, it loads its corresponding service number into the EAX (x86) or RAX (x64) register and executes the syscall (or sysenter) instruction. The kernel uses that value as an index into the SSDT to dispatch to the correct function. For example:

  • NtCreateFile 0x55
  • NtOpenProcess 0x23
  • NtAllocateVirtualMemory 0x18
  • NtCreateTimer 0xCB

Linux (SYSCALL Table Index)

System calls are defined in a syscall table, often in arch/x86/entry/syscalls/syscall_64.tbl or similar. User code places the syscall number in the RAX register, arguments in RDI, RSI, etc., and then executes syscall. The kernel reads RAX, looks up the handler in its internal table, and jumps there. For instance, on x86_64:

  • read() 0
  • write() 1
  • open() 2
  • execve() 59

Note: Service numbers vary by Windows version and build, never hard-code; always query dynamically. Try Hell's Gate, FreshyCalls or SysWhispers3.

Direct/Indirect System Calls


Direct Syscalls

A direct system call is when an application or malware directly invokes a kernel function without going through any intermediate layers or standard libraries. This means the code is essentially talking straight to the operating system's kernel using low-level mechanisms. For example, on Windows, it's similar but often involves functions from the Native API, such as NtCreateFile or ZwWriteFile, which are part of the System Service Dispatch Table (SSDT).


In practice:

When a direct system call is made, the CPU switches from user mode to kernel mode via a special instruction like int 0x80 on older x86 systems or syscall on modern AMD64. The kernel then handles the request directly based on the syscall number and parameters passed. This bypasses any user-space wrappers, giving you raw access to kernel services.


Advantages:

Offers high performance and evasion potential by bypassing user-mode hooks in EDR tools, making it ideal for stealthy code execution in pen tests. Direct calls reduce detection chances in initial access phases.


Disadvantages:

Highly platform-specific, with syscall numbers changing across OS updates, leading to crashes if not handled dynamically. Also, kernel-level monitoring can flag them as suspicious, increasing detection risk.

Indirect Syscalls

An indirect system call, on the other hand, involves invoking a kernel service through higher-level abstractions, like standard libraries or APIs, rather than directly addressing the kernel. Here, the application uses functions from user-space libraries like fopen from libc on Linux or CreateFileW from the Windows API to make the call. These libraries act as intermediaries, translating your request into the actual kernel syscall behind the scenes.


In practice:

When you call an indirect system call, the library handles the details such as mapping the function to the correct syscall number, managing parameters, and dealing with any OS-specific quirks before the request is passed to the kernel. For example, calling fopen in C doesn't directly use the syscall instruction; instead, it goes through libc, which then invokes the appropriate kernel function like sys_open on Linux. This adds a layer of indirection, making the code more readable and portable but also introducing potential overhead.


Advantages:

Easier to implement and more portable, allowing code to run across different OSes without major changes. Can blend with legitimate traffic for better stealth in less-monitored environments.


Disadvantages:

Adds performance overhead and is vulnerable to API hooking, making it less reliable for evasion in advanced EDR setups.


Why Direct Syscalls Are Watched Most By AV/EDR

Behavioral and Anomaly Detection

EDR tools use syscall monitoring to build behavioral baselines. For instance, tools like CrowdStrike or Microsoft Defender analyze syscall sequences to spot deviations from normal patterns. If a process suddenly makes unusual syscalls such as rapid file reads/writes, it could flag ransomware or exploit attempts. This is more effective than monitoring less critical areas like user-mode APIs alone.

Universal Gateway to the Kernel

Every sensitive operation such as file I/O, memory allocation, process/thread control, DLL loading, ultimately happens via a syscall. By monitoring these low-level entry points, AV/EDR solutions gain visibility over all critical system activity, regardless of which library or shim the attacker uses.

Evasion and Stealth Challenges

Malware often tries to evade detection by manipulating syscalls like, using direct syscalls to bypass API hooks. As a result, AV/EDR has evolved to focus heavily on syscall-level interception through kernel hooks, ETW (Event Tracing for Windows), or eBPF (on Linux) to cover both direct and indirect calls. This makes syscalls a "must-watch" area, as they're harder for attackers to obfuscate completely.

High-Value, High-Risk Targets

Certain syscall families like process creation, memory manipulation, module loading are disproportionately used in exploits and code-injection attacks. AV/EDR tools prioritize these “high-risk” indices in the Service Dispatch Table to optimize performance while minimizing blind spots.


In short:
In short, syscalls are watched most because they're a concentrated point of risk, defenders get maximum insight with minimal false positives, while attackers see them as a key to success. This makes understanding syscall monitoring essential for both red-team evasion and blue-team hardening.

Cybersecurity Takeaways

Offensive Angle: If you're testing defenses, leveraging direct calls can help evade initial detection since they're less commonly monitored. For example, in a red-team exercise, using direct syscalls might bypass API-based hooks, giving you an edge in stealthy persistence. However, if EDR is tuned for kernel-level monitoring, it could backfire and alert defenders faster.

Defensive Angle: As a blue-teamer, focusing your monitoring on indirect calls provides the best bang for your buck, covering 80-90% of potential threats with less overhead. But always layer in direct call detection for high-confidence alerts.


Evasion Techniques

Using Direct Syscall

Bypasses user-mode hooks by issuing the syscall/sysenter instruction directly or calling the Nt* stub in ntdll.dll by index. No Win32/API wrapper, no IAT entry, AV/EDR misses most of these calls.

Hell’s Gate & SysWhispers

Dynamically resolve and invoke syscalls by patching in-memory stubs that map to kernel service numbers, avoiding static import of ntdll.dll functions. This thwarts signature-based detection of known syscall trampolines.

Syscall Proxying and Obfuscation

Redirect syscall calls through custom proxies or wrappers to mask their origin. On Linux, you might hook into the syscall table with a user-defined function that randomizes parameters or adds noise, confusing behavioral analyzers. In Windows, techniques like syscall proxying can involve injecting code that mimics legitimate processes, making it tougher for ETW logging to pin down malicious activity.

Syscall Randomization and Dynamic Loading

Instead of sticking to static syscall numbers, randomize or load them at runtime to avoid predictable patterns that EDR tools might flag. For example, in Windows, you could write code that scans the PEB (Process Environment Block) or uses libraries like HalosGate to dynamically fetch and invoke syscall indices. On Linux, leverage tools or custom scripts to alter syscall arguments or use seccomp filters to mask calls, making it harder for behavioral analyzers to build a profile of your activity. This technique helps evade machine learning-based detection by introducing variability, perfect for red-team exercises where you want to simulate adaptive threats.


Real-World Case Studies

SolarWinds Supply Chain Attack (2020):

In this high-profile incident, attackers manipulated syscalls to maintain persistence and evade detection. They used direct syscall invocations to interact with kernel objects without triggering user-mode hooks, allowing them to move laterally and steal data for months. Takeaway for pen testers, this shows how dynamic syscall handling can bypass even mature EDR systems, try replicating this in your tests to expose similar weaknesses.

Cobalt Strike HalosGate & HellsGate syscaller:

Cobalt Strike’s HalosGate & HellsGate syscaller technique dynamically builds syscall stubs at runtime, pulling the SSDT index from KiServiceTable. This allowed red-team operators to call syscalls by number without ever importing native API functions, evading both user-mode and basic kernel-mode hooks.

WannaCry Ransomware (2017):

While not syscall-specific, WannaCry leveraged indirect syscalls for file encryption, but advanced variants learned from it. Modern ransomware now often mixes direct and indirect calls to confuse monitors. For example, using NtCreateFile directly for speed while avoiding API logs. Lesson: Evasion isn't just about speed; it's about adaptability, test how your syscall-based tools hold up against behavioral analytics in a lab.

Discord Loader (2021)

A loader distributed via Discord abused Linux’s io_uring interface to download and write payloads in a single batched syscall submission. Defenders monitoring only individual read/write syscalls missed the combined operation, delaying detection.


Project


SSN Enumeration
            

// -------------------------------------
// Project Name: Syscall-SSN Enumerator |
// By: Iron Hulk                        |
// -------------------------------------

#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winternl.h>

#include <iostream>
#include <iomanip>
#include <sstream>
#include <string>
#include <vector>


typedef LONG(NTAPI* RtlGetVersionPtr)(PRTL_OSVERSIONINFOEXW);
struct ExportInfo 
{
    std::string name;
    DWORD       ssn;
    DWORD       syscallOffset;
};


// ------------------------------------------------------------
// SSN Enumeration
// ------------------------------------------------------------
std::vector<ExportInfo> GetNtExportsSSN() {
    HMODULE hNtdll = LoadLibraryW(L"ntdll.dll");
    if (!hNtdll) throw std::runtime_error("LoadLibrary failed");

    auto dos = (IMAGE_DOS_HEADER*)hNtdll;
    auto nt = (IMAGE_NT_HEADERS*)((BYTE*)hNtdll + dos->e_lfanew);
    auto exports = (IMAGE_EXPORT_DIRECTORY*)((BYTE*)hNtdll +
        nt->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT].VirtualAddress);

    auto names = (DWORD*)((BYTE*)hNtdll + exports->AddressOfNames);
    auto funcs = (DWORD*)((BYTE*)hNtdll + exports->AddressOfFunctions);
    auto ords = (WORD*)((BYTE*)hNtdll + exports->AddressOfNameOrdinals);

    std::vector<ExportInfo> list;
    for (DWORD i = 0; i < exports->NumberOfNames; i++) {
        const char* fnName = (const char*)hNtdll + names[i];
        DWORD       rva = funcs[ords[i]];
        BYTE* stub = (BYTE*)hNtdll + rva;

        // mov r10,rcx; mov eax,imm32
        if (stub[0] == 0x4C && stub[1] == 0x8B &&
            stub[2] == 0xD1 && stub[3] == 0xB8)
        {
            DWORD ssn = *(DWORD*)(stub + 4);

            DWORD syscallOffset = 0xFFFFFFFF;

            // Scan first 32 bytes for syscall (0F 05)
            for (DWORD j = 0; j < 32; j++) {
                if (stub[j] == 0x0F && stub[j + 1] == 0x05) {
                    syscallOffset = j;
                    break;
                }
            }

            if (syscallOffset != 0xFFFFFFFF) {
                list.push_back({ fnName, ssn, syscallOffset });
            }
        }

    }
    return list;
}

// ------------------------------------------------------------
// Registry Helpers
// ------------------------------------------------------------
static bool RegGetStringView(
    HKEY root,
    const wchar_t* subKey,
    const wchar_t* valueName,
    REGSAM viewFlag,
    std::wstring& out
) {
    out.clear();

    HKEY hKey = nullptr;
    if (RegOpenKeyExW(root, subKey, 0, KEY_QUERY_VALUE | viewFlag, &hKey) != ERROR_SUCCESS)
        return false;

    DWORD type = 0;
    DWORD cb = 0;

    LONG st = RegQueryValueExW(hKey, valueName, nullptr, &type, nullptr, &cb);
    if (st != ERROR_SUCCESS || type != REG_SZ || cb < sizeof(wchar_t)) {
        RegCloseKey(hKey);
        return false;
    }

    std::wstring buf(cb / sizeof(wchar_t), L'\0');
    st = RegQueryValueExW(hKey, valueName, nullptr, &type, reinterpret_cast<LPBYTE>(&buf[0]), &cb);
    RegCloseKey(hKey);

    if (st != ERROR_SUCCESS || type != REG_SZ)
        return false;

    if (!buf.empty() && buf.back() == L'\0') buf.pop_back();
    out = buf;
    return true;
}

static bool RegGetDWORDView(
    HKEY root,
    const wchar_t* subKey,
    const wchar_t* valueName,
    REGSAM viewFlag,
    DWORD& out
) {
    out = 0;

    HKEY hKey = nullptr;
    if (RegOpenKeyExW(root, subKey, 0, KEY_QUERY_VALUE | viewFlag, &hKey) != ERROR_SUCCESS)
        return false;

    DWORD type = 0;
    DWORD cb = sizeof(DWORD);
    DWORD data = 0;

    LONG st = RegQueryValueExW(hKey, valueName, nullptr, &type, reinterpret_cast<LPBYTE>(&data), &cb);
    RegCloseKey(hKey);

    if (st != ERROR_SUCCESS || type != REG_DWORD)
        return false;

    out = data;
    return true;
}

// ------------------------------------------------------------
// WOW64 / Architecture Helpers
// ------------------------------------------------------------
static bool IsProcessWow64() {
    BOOL wow = FALSE;
    typedef BOOL(WINAPI* IsWow64Process_t)(HANDLE, PBOOL);

    auto p = reinterpret_cast<IsWow64Process_t>(
        GetProcAddress(GetModuleHandleW(L"kernel32.dll"), "IsWow64Process")
        );

    return (p && p(GetCurrentProcess(), &wow)) ? (wow == TRUE) : false;
}

static void PrintArchitecture() {
    // Process architecture (compile-time)
#if defined(_WIN64)
    const char* procArch = "x64";
#else
    const char* procArch = "x86";
#endif

    // OS architecture (runtime best-effort)
    std::string osArch = "unknown";

    typedef BOOL(WINAPI* IsWow64Process2_t)(HANDLE, USHORT*, USHORT*);
    auto p2 = reinterpret_cast<IsWow64Process2_t>(
        GetProcAddress(GetModuleHandleW(L"kernel32.dll"), "IsWow64Process2")
        );

    if (p2) {
        USHORT pm = 0, nm = 0;
        if (p2(GetCurrentProcess(), &pm, &nm)) {
            switch (nm) {
            case IMAGE_FILE_MACHINE_AMD64: osArch = "x64"; break;
            case IMAGE_FILE_MACHINE_ARM64: osArch = "arm64"; break;
            case IMAGE_FILE_MACHINE_I386:  osArch = "x86"; break;
            default: osArch = "unknown"; break;
            }

#if !defined(_WIN64)
            if (pm != IMAGE_FILE_MACHINE_UNKNOWN && osArch == "x64") {
                osArch = "x64 (WOW64)";
            }
#endif
        }
    }
    else {
#if defined(_WIN64)
        osArch = "x64";
#else
        osArch = IsProcessWow64() ? "x64 (WOW64)" : "x86";
#endif
    }
    std::cout << "Process architecture: " << procArch << "\n"
        << "OS architecture:      " << osArch << "\n";
}

// ------------------------------------------------------------
// Windows Marketing / Build Info
// ------------------------------------------------------------
struct WinMarketingInfo {
    std::wstring productName;     // e.g. "Windows 11 Pro"
    std::wstring displayVersion;  // e.g. "22H2" / "24H2" / "25H2"
    DWORD currentBuild = 0;       // e.g. 19045 / 22631 / 26100...
    DWORD ubr = 0;                // revision
};

static WinMarketingInfo GetWindowsMarketingInfo() {
    constexpr const wchar_t* kSubKey = L"SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion";

    // Choose native registry view when on 64-bit OS
    REGSAM view =
#if defined(_WIN64)
        KEY_WOW64_64KEY;
#else
        (IsProcessWow64() ? KEY_WOW64_64KEY : KEY_WOW64_32KEY);
#endif

    WinMarketingInfo info;

    RegGetStringView(HKEY_LOCAL_MACHINE, kSubKey, L"ProductName", view, info.productName);
    RegGetStringView(HKEY_LOCAL_MACHINE, kSubKey, L"DisplayVersion", view, info.displayVersion);

    std::wstring buildStr;
    if (RegGetStringView(HKEY_LOCAL_MACHINE, kSubKey, L"CurrentBuildNumber", view, buildStr) ||
        RegGetStringView(HKEY_LOCAL_MACHINE, kSubKey, L"CurrentBuild", view, buildStr)) {
        info.currentBuild = static_cast<DWORD>(_wtoi(buildStr.c_str()));
    }

    RegGetDWORDView(HKEY_LOCAL_MACHINE, kSubKey, L"UBR", view, info.ubr);
    return info;
}

// ------------------------------------------------------------
// OS Version (RtlGetVersion)
// ------------------------------------------------------------
static RTL_OSVERSIONINFOEXW GetOsVersion() {
    RTL_OSVERSIONINFOEXW osvi = {};
    osvi.dwOSVersionInfoSize = sizeof(osvi);

    HMODULE hNtDll = GetModuleHandleW(L"ntdll.dll");
    if (!hNtDll) hNtDll = LoadLibraryW(L"ntdll.dll");
    if (!hNtDll) return osvi;

    auto rtlGetVersion = reinterpret_cast<RtlGetVersionPtr>(
        GetProcAddress(hNtDll, "RtlGetVersion")
        );

    if (rtlGetVersion) rtlGetVersion(&osvi);
    return osvi;
}

static std::wstring GetWindowsFamily(const RTL_OSVERSIONINFOEXW& osvi) {
    if (osvi.dwMajorVersion == 10) {
        return (osvi.dwBuildNumber >= 22000) ? L"Windows 11" : L"Windows 10";
    }
    if (osvi.dwMajorVersion == 6 && osvi.dwMinorVersion == 3) return L"Windows 8.1";
    if (osvi.dwMajorVersion == 6 && osvi.dwMinorVersion == 2) return L"Windows 8";
    if (osvi.dwMajorVersion == 6 && osvi.dwMinorVersion == 1) return L"Windows 7";
    return L"Windows (unknown)";
}

// ------------------------------------------------------------
// main
// ------------------------------------------------------------
int main() {
    // --- OS / Build identification ---
    const auto osvi = GetOsVersion();
    const auto mk = GetWindowsMarketingInfo();

    const std::wstring family = GetWindowsFamily(osvi);

    // Keep ProductName if present, but correct Win11 family if build indicates Win11
    std::wstring marketing = mk.productName.empty() ? family : mk.productName;
    if (osvi.dwBuildNumber >= 22000 && marketing.find(L"Windows 11") == std::wstring::npos) marketing = L"Windows 11";
    const std::wstring hv = mk.displayVersion.empty() ? L"(unknown)" : mk.displayVersion;
    const DWORD buildBase = mk.currentBuild ? mk.currentBuild : static_cast<DWORD>(osvi.dwBuildNumber);
    const DWORD ubr = mk.ubr;
    std::wcout << L"OS Family:    " << family << L"\n"
        << L"OS Version:   " << osvi.dwMajorVersion << L"." << osvi.dwMinorVersion
        << L" (Build " << osvi.dwBuildNumber << L")\n"
        << L"OS Marketing: " << marketing << L" " << hv << L"\n"
        << L"OS Build:     " << buildBase
        << (ubr ? (L"." + std::to_wstring(ubr)) : L"")
        << L"\n\n";

    // --- Architecture (printed once) ---
    PrintArchitecture();

    // --- SSN enumeration output ---
    std::cout << "\nSyscall SSNs:\n\n";

    try {
        const auto exports = GetNtExportsSSN();

        std::cout << std::left << std::setw(60) << "Syscall"
            << " | " << std::right << std::setw(7) << "Dec"
            << " | " << std::right << std::setw(7) << "Hex"
            << " | " << std::right << std::setw(8) << "Offset"
            << "\n";
        std::cout << std::string(60, '-') << "-+-"
            << std::string(7, '-') << "-+-"
            << std::string(7, '-') << "-+-"
            << std::string(8, '-') << "\n";
        for (const auto& e : exports) 
        {
            std::ostringstream hexssn;
            hexssn << "0x" << std::hex << std::uppercase << e.ssn;
            std::ostringstream hexoff;
            hexoff << "0x" << std::hex << std::uppercase << e.syscallOffset;
            std::cout << std::left << std::setw(60) << e.name
                << " | " << std::right << std::setw(7) << std::dec << e.ssn
                << " | " << std::right << std::setw(7) << hexssn.str()
                << " | " << std::right << std::setw(8) << hexoff.str()
                << std::dec << "\n";
        }
    }
    catch (const std::exception& ex) {
        std::cerr << "Error: " << ex.what() << "\n";
        return 1;
    }
    return 0;
}