Windows 10: New Anti-Debug OutputDebugStringW

Prior to Windows 10, OutputDebugStringW was only a dummy implementation. The function converted the input Unicode string to a simple Ansi string and calls the Ansi version of the function OutputDebugStringA internally. Now with Windows 10, Microsoft implemented the real Unicode function for OutputDebugString. Therefore, in the past it was enough to handle OutputDebugStringA for Anti-Anti-Debug purpose, now we have to defeat another Anti-Debug possibility.

The magic behind OutputDebugString is to raise an exception. This exception should be handled by a debugger and most debugger swallow this exception, so we can easily detect that the exception was swallowed by a debugger.

The simplified Ansi version of the function with implemented Anti-Debug:

void __stdcall _OutputDebugStringA(LPCSTR lpOutputString)
{
	ULONG_PTR args[2];
	args[0] = (ULONG_PTR)strlen(lpOutputString) + 1;
	args[1] = (ULONG_PTR)lpOutputString;

	__try
	{
		RaiseException(0x40010006, 0, 2, args);//DBG_PRINTEXCEPTION_C
		ShowMessageBox("DBG_PRINTEXCEPTION_C -> Debugger detected");
	}
	__except(EXCEPTION_EXECUTE_HANDLER)
	{
		ShowMessageBox("DBG_PRINTEXCEPTION_C -> Debugger NOT detected");
	}
}

RaiseException is used to create an exception for the debugger. There are 2 special arguments for the debugger. The buffer and the length of the string. The exception code is 0x40010006 which is DBG_PRINTEXCEPTION_C.

Now the new simplified Unicode implementation for Windows 10 looks like this:

void __stdcall _OutputDebugStringW(LPCWSTR lpOutputString)
{
	char outputDebugStringBuffer[1000] = {0};
	WideCharToMultiByte(CP_ACP, 0, lpOutputString, -1, outputDebugStringBuffer, sizeof(outputDebugStringBuffer), 0, 0);

	ULONG_PTR args[4];

	//unicode
	args[0] = (ULONG_PTR)wcslen(lpOutputString) + 1;
	args[1] = (ULONG_PTR)lpOutputString;

	//ansi for compatibility
	args[2] = (ULONG_PTR)wcslen(lpOutputString) + 1;
	args[3] = (ULONG_PTR)outputDebugStringBuffer;

	__try
	{
		RaiseException(0x4001000A, 0, 4, args);//DBG_PRINTEXCEPTION_WIDE_C
		ShowMessageBox("DBG_PRINTEXCEPTION_WIDE_C -> Debugger detected");
	}
	__except(EXCEPTION_EXECUTE_HANDLER)
	{
		ShowMessageBox("DBG_PRINTEXCEPTION_WIDE_C -> Debugger NOT detected");
	}
}

Windows creates a new exception with RaiseException: DBG_PRINTEXCEPTION_WIDE_C (0x4001000A). This new exception code needs 4 parameters. For backwards compatibility it also requests the Ansi version of the string.