Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 1f311da336 | |||
| 35033603d7 |
30
README.md
30
README.md
@ -4,9 +4,9 @@ by LoRd_MuldeR <<mulder2@gmx>> | <http://muldersoft.com/>
|
||||
Introduction
|
||||
============
|
||||
|
||||
**TimedExec** is a small utility for *benchmarking* command-line programs. It will *execute* the specified program with the specified command-line arguments and then *measure* the time that it takes for the execution to complete. In order to obtain *accurate* results, all measurements are implemented via *high-resolution* performance timers. And, since program execution times unavoidably are subject to certain variations (e.g. due to environmental noise), each test will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of "warm-up" passes can be performed *prior to* the first metering pass. The warm-up passes prevent caching effects from interfering with the execution times.
|
||||
**TimedExec** is a small utility for *benchmarking* programs. It will *execute* the specified program with the specified command-line arguments and *measure* the time that it takes for the execution to complete. Because the execution time of a program unavoidably is subject to certain variations (e.g. due to environmental noise), each measurement will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of “warm-up” passes can be performed *prior to* the metering passes. The warm-up passes prevent caching effects from interfering with the measurement.
|
||||
|
||||
TimedExec will then compute the ***mean*** execution time as well as the ***median*** execution time of all metering passes. It will also record the *fastest* and *slowest* execution time that has been measured. Furthermore, TimedExec computes the *standard error* in order to determine [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf) from the benchmarking results. These are the *ranges* which contain the program's “real” average execution time (expected value), *with very high probability*. All results will be saved to a log file.
|
||||
Once all metering passes have been completed, TimedExec will compute the ***mean*** execution time as well as the ***median*** execution time of the program. It will also record the *fastest* and *slowest* execution time that has been observed. Furthermore, TimedExec computes the *standard error*, in order to determine [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf) from the benchmarking results.
|
||||
|
||||
|
||||
Usage Instructions
|
||||
@ -18,7 +18,7 @@ Usage Instructions
|
||||
|
||||
```
|
||||
===============================================================================
|
||||
Timed Exec - Benchmarking Utility, Version 1.05
|
||||
Timed Exec - Benchmarking Utility, Version 1.06
|
||||
Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
@ -33,7 +33,9 @@ Influential environment variables:
|
||||
TIMED_EXEC_PASSES - Number of execution passes (default: 5)
|
||||
TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: 1)
|
||||
TIMED_EXEC_LOGFILE - Log-File Name (default: "TimedExec.log")
|
||||
TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks
|
||||
TIMED_EXEC_NO_CHECKS - Do *not* check the process exit codes
|
||||
TIMED_EXEC_NO_PRIORITY - Do *not* adjust the process priorities
|
||||
TIMED_EXEC_NO_PERIOD - Do *not* adjust system timer period
|
||||
TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements
|
||||
```
|
||||
|
||||
@ -56,7 +58,6 @@ In the following example we use *TimedExec* to benchmark the program **`ping.exe
|
||||
TimedExec.exe C:\Windows\System32\ping.exe -n 12 www.google.com
|
||||
```
|
||||
|
||||
|
||||
Results
|
||||
=======
|
||||
|
||||
@ -81,11 +82,16 @@ Active Clock Type : WALLCLOCK (0)
|
||||
Interpretation
|
||||
--------------
|
||||
|
||||
When comparing measurement results, the ***mean*** (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the ***median*** execution times often is the better choice. That is because the *median* of a data sample is much more robust against outliers.
|
||||
When comparing measurement results, the [***mean***](https://en.wikipedia.org/wiki/Arithmetic_mean) (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the [***median***](https://en.wikipedia.org/wiki/Median) of the execution times often is the better choice. That is because the *median* of a data sample is much more robust against outliers.
|
||||
|
||||
Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes – which is **not** possible in practice. In this situation, we can have a look at the ***confidence intervals***. These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one (higher confidence means broader interval, and vice versa).
|
||||
Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes – which is **not** possible in practice. In this situation, we can have a look at the [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf). These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one. Higher confidence means broader interval, and vice versa.
|
||||
|
||||
Simply put, as long as the confidence intervals of program A and program B *overlap* (at least partially), we **must not** conclude that either of these programs runs faster (or slower) in the average case. ***No*** conclusion can be drawn in that case!
|
||||
Simply put, as long as the confidence intervals of the runtime of program “A” and the runtime of program “B” *overlap*, we **must not** conclude that either of these programs runs faster (or slower). In fact, **no** real conclusion can be drawn in that case!
|
||||
|
||||
Limitations
|
||||
-----------
|
||||
|
||||
This tools measures the runtime of *processes*. Because creating a process has a certain overhead, and because the system timer has a limited precision – usually in the range of a few milliseconds, but can be worse – this tool is **not** suitable for benchmarking programs or functions with *very short* runtime! The process to be measured should run *at least* for a couple of seconds, in order to get useful benchmark results. If you need to benchmark functions with *very short* runtime, it is recommended to use [*high-precision timers*](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter) directly inside your program code, rather than launching separate processes.
|
||||
|
||||
|
||||
Sources
|
||||
@ -93,10 +99,10 @@ Sources
|
||||
|
||||
The *TimedExec* source codes are managed by [**Git**](http://git-scm.com/doc) and are available from one of the official mirrors:
|
||||
|
||||
* <tt>https://github.com/lordmulder/TimedExec.git</tt> ([Browse](https://github.com/lordmulder/TimedExec))
|
||||
* <tt>https://bitbucket.org/muldersoft/timedexec.git</tt> ([Browse](https://bitbucket.org/muldersoft/timedexec))
|
||||
* <tt>https://gitlab.com/timedexec/timedexec.git</tt> ([Browse](https://gitlab.com/timedexec/timedexec))
|
||||
|
||||
* `https://github.com/lordmulder/TimedExec.git` ([Browse](https://github.com/lordmulder/TimedExec))
|
||||
* `https://bitbucket.org/muldersoft/timedexec.git` ([Browse](https://bitbucket.org/muldersoft/timedexec))
|
||||
* `https://gitlab.com/timedexec/timedexec.git` ([Browse](https://gitlab.com/timedexec/timedexec))
|
||||
* `https://punkindrublic.mooo.com:3000/Muldersoft/TimedExec.git` ([Browse](https://punkindrublic.mooo.com:3000/Muldersoft/TimedExec))
|
||||
|
||||
License
|
||||
=======
|
||||
|
||||
@ -115,7 +115,7 @@
|
||||
<Link>
|
||||
<SubSystem>Console</SubSystem>
|
||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
|
||||
@ -130,7 +130,7 @@
|
||||
<Link>
|
||||
<SubSystem>Console</SubSystem>
|
||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
|
||||
@ -159,7 +159,7 @@
|
||||
<EnableCOMDATFolding>true</EnableCOMDATFolding>
|
||||
<OptimizeReferences>true</OptimizeReferences>
|
||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
@ -190,7 +190,7 @@
|
||||
<OptimizeReferences>true</OptimizeReferences>
|
||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||
<AdditionalDependencies>Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
|
||||
|
||||
@ -116,7 +116,7 @@
|
||||
<Link>
|
||||
<SubSystem>Console</SubSystem>
|
||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
|
||||
@ -130,7 +130,7 @@
|
||||
<Link>
|
||||
<SubSystem>Console</SubSystem>
|
||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
|
||||
@ -159,7 +159,7 @@
|
||||
<EnableCOMDATFolding>true</EnableCOMDATFolding>
|
||||
<OptimizeReferences>true</OptimizeReferences>
|
||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||
<MinimumRequiredVersion>5.1</MinimumRequiredVersion>
|
||||
</Link>
|
||||
@ -191,7 +191,7 @@
|
||||
<OptimizeReferences>true</OptimizeReferences>
|
||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||
<AdditionalDependencies>Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||
<MinimumRequiredVersion>5.2</MinimumRequiredVersion>
|
||||
</Link>
|
||||
</ItemDefinitionGroup>
|
||||
|
||||
@ -34,7 +34,12 @@
|
||||
#define WIN32_LEAN_AND_MEAN 1
|
||||
#include <Windows.h>
|
||||
#include <ShellAPI.h>
|
||||
#include <Shlwapi.h>
|
||||
|
||||
#if _WIN32_WINNT >= 0x0603
|
||||
# include <timeapi.h>
|
||||
#else
|
||||
# include <MMSystem.h>
|
||||
#endif
|
||||
|
||||
#define DEFAULT_EXEC_LOOPS 5
|
||||
#define DEFAULT_WARMUP_LOOPS 1
|
||||
@ -42,9 +47,9 @@
|
||||
#define ENABLE_ENV_VARS true
|
||||
|
||||
#ifdef _UNICODE
|
||||
#define tstring std::wstring
|
||||
# define tstring std::wstring
|
||||
#else
|
||||
#define tstring std::string
|
||||
# define tstring std::string
|
||||
#endif
|
||||
|
||||
static HANDLE g_hAbortEvent = NULL;
|
||||
@ -88,10 +93,49 @@ static const char *printClockType(const clock_type_t clock_type)
|
||||
return "N/A";
|
||||
}
|
||||
|
||||
static bool setTimerResolution(UINT& period)
|
||||
{
|
||||
TIMECAPS timecaps;
|
||||
if (timeGetDevCaps(&timecaps, sizeof(TIMECAPS)) == MMSYSERR_NOERROR)
|
||||
{
|
||||
if (timeBeginPeriod(timecaps.wPeriodMin) == MMSYSERR_NOERROR)
|
||||
{
|
||||
period = timecaps.wPeriodMin;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
period = MAXUINT;
|
||||
return false;
|
||||
}
|
||||
|
||||
static void resetTimerResolution(const UINT period)
|
||||
{
|
||||
if (period != MAXUINT)
|
||||
{
|
||||
timeEndPeriod(period);
|
||||
}
|
||||
}
|
||||
|
||||
static tstring trim(const std::vector<_TCHAR> &buffer)
|
||||
{
|
||||
std::vector<_TCHAR>::const_iterator left;
|
||||
for (left = buffer.cbegin(); (left != buffer.cend()) && (*left <= 0x20); ++left);
|
||||
if (left != buffer.cend())
|
||||
{
|
||||
std::vector<_TCHAR>::const_reverse_iterator right;
|
||||
for (right = buffer.crbegin(); (right != buffer.crend()) && (*right <= 0x20); ++right);
|
||||
if (right != buffer.crend())
|
||||
{
|
||||
return tstring(left, right.base());
|
||||
}
|
||||
}
|
||||
return tstring();
|
||||
}
|
||||
|
||||
static bool getEnvVariable(const _TCHAR *const name, tstring &value)
|
||||
{
|
||||
std::vector<TCHAR> buffer;
|
||||
|
||||
std::vector<_TCHAR> buffer(MAX_PATH);
|
||||
value.clear();
|
||||
for (int i = 0; i < 3; ++i)
|
||||
{
|
||||
const DWORD result = GetEnvironmentVariable(name, buffer.data(), (DWORD)buffer.size());
|
||||
@ -99,21 +143,28 @@ static bool getEnvVariable(const _TCHAR *const name, tstring &value)
|
||||
{
|
||||
break; /*failed*/
|
||||
}
|
||||
if (result > buffer.size())
|
||||
else if (result > buffer.size())
|
||||
{
|
||||
buffer.resize(result); /*adjust buffer*/
|
||||
buffer.resize(result);
|
||||
}
|
||||
else if (result < buffer.size())
|
||||
{
|
||||
value = tstring(buffer.data());
|
||||
value = trim(buffer);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
value.clear();
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool parseFlag(const tstring &value)
|
||||
{
|
||||
if ((!_tcsicmp(value.c_str(), _T("yes"))) || (!_tcsicmp(value.c_str(), _T("true"))))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
return (_tcstol(value.c_str(), NULL, 10) > 0L);
|
||||
}
|
||||
|
||||
static ULONGLONG fileTimeToU64(const PFILETIME fileTime)
|
||||
{
|
||||
ULARGE_INTEGER temp;
|
||||
@ -170,8 +221,7 @@ static bool checkBinary(const tstring &filePath)
|
||||
|
||||
static tstring getFullPath(const _TCHAR *const fileName)
|
||||
{
|
||||
std::vector<TCHAR> buffer;
|
||||
|
||||
std::vector<TCHAR> buffer(MAX_PATH);
|
||||
for (int i = 0; i < 3; ++i)
|
||||
{
|
||||
const DWORD result = GetFullPathName(fileName, (DWORD)buffer.size(), buffer.data(), NULL);
|
||||
@ -179,32 +229,33 @@ static tstring getFullPath(const _TCHAR *const fileName)
|
||||
{
|
||||
break; /*failed*/
|
||||
}
|
||||
if (result > buffer.size())
|
||||
else if (result > buffer.size())
|
||||
{
|
||||
buffer.resize(result); /*adjust buffer*/
|
||||
buffer.resize(result);
|
||||
}
|
||||
else if (result < buffer.size())
|
||||
{
|
||||
return tstring(buffer.data());
|
||||
return tstring(buffer.begin(), buffer.end());
|
||||
}
|
||||
}
|
||||
|
||||
return tstring(fileName);
|
||||
}
|
||||
|
||||
static tstring getFileNameOnly(const tstring &filePath)
|
||||
{
|
||||
TCHAR *buffer = _tcsdup(filePath.c_str());
|
||||
if (!buffer)
|
||||
for (tstring::const_reverse_iterator iter = filePath.crbegin(); iter != filePath.crend(); ++iter)
|
||||
{
|
||||
if ((*iter == _T('/')) || (*iter == _T('\\')))
|
||||
{
|
||||
tstring::const_iterator offset = iter.base();
|
||||
if (offset != filePath.cend())
|
||||
{
|
||||
return tstring(offset, filePath.cend());
|
||||
}
|
||||
return tstring();
|
||||
}
|
||||
|
||||
PathStripPath(buffer);
|
||||
|
||||
const tstring result(buffer);
|
||||
free(buffer);
|
||||
return result;
|
||||
}
|
||||
return filePath;
|
||||
}
|
||||
|
||||
static double computeMedian(std::vector<double> &data)
|
||||
@ -224,6 +275,25 @@ static double computeMedian(std::vector<double> &data)
|
||||
: data[center];
|
||||
}
|
||||
|
||||
static void appendStr(tstring &commandLine, const tstring &token)
|
||||
{
|
||||
for (tstring::const_iterator iter = token.cbegin(); iter != token.cend(); ++iter)
|
||||
{
|
||||
if (*iter)
|
||||
{
|
||||
if (*iter == _T('"'))
|
||||
{
|
||||
commandLine += _T('\\');
|
||||
}
|
||||
commandLine += *iter;
|
||||
}
|
||||
else
|
||||
{
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
||||
{
|
||||
commandLine.clear();
|
||||
@ -240,7 +310,6 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
||||
for(int i = 1; i < nArgs; i++)
|
||||
{
|
||||
tstring token;
|
||||
|
||||
if (i > 1)
|
||||
{
|
||||
token = tstring(szArglist[i]);
|
||||
@ -249,17 +318,16 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
||||
else
|
||||
{
|
||||
token = getFullPath(szArglist[i]);
|
||||
programFile += token;
|
||||
programFile = token;
|
||||
}
|
||||
|
||||
if (token.find(' ') == tstring::npos)
|
||||
if (token.find(_T(' ')) == tstring::npos)
|
||||
{
|
||||
commandLine += token;
|
||||
appendStr(commandLine, token);
|
||||
}
|
||||
else
|
||||
{
|
||||
commandLine += _T('"');
|
||||
commandLine += token;
|
||||
appendStr(commandLine, token);
|
||||
commandLine += _T('"');
|
||||
}
|
||||
}
|
||||
@ -268,14 +336,14 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
||||
return nArgs;
|
||||
}
|
||||
|
||||
static bool createProcess(const tstring &commandLine, HANDLE &hThrd, HANDLE &hProc, const bool suspended = false)
|
||||
static bool createProcess(const tstring &commandLine, HANDLE &hThrd, HANDLE &hProc, const bool highPriority = false)
|
||||
{
|
||||
STARTUPINFO startInfo;
|
||||
SecureZeroMemory(&startInfo, sizeof(STARTUPINFO));
|
||||
PROCESS_INFORMATION processInfo;
|
||||
SecureZeroMemory(&processInfo, sizeof(PROCESS_INFORMATION));
|
||||
|
||||
if (!CreateProcess(NULL, (LPTSTR)commandLine.c_str(), NULL, NULL, false, HIGH_PRIORITY_CLASS | (suspended ? CREATE_SUSPENDED : 0), NULL, NULL, &startInfo, &processInfo))
|
||||
if (!CreateProcess(NULL, (LPTSTR)commandLine.c_str(), NULL, NULL, false, highPriority ? HIGH_PRIORITY_CLASS : 0U, NULL, NULL, &startInfo, &processInfo))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@ -308,8 +376,7 @@ static int getProcessExitCode(const HANDLE &hProc)
|
||||
{
|
||||
return *reinterpret_cast<int*>(&exitCode);
|
||||
}
|
||||
|
||||
return 0;
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void abortedHandlerRoutine(const HANDLE &hProc)
|
||||
@ -351,8 +418,11 @@ static LONG WINAPI crashHandlerRoutine(struct _EXCEPTION_POINTERS *ExceptionInfo
|
||||
// MAIN FUNCTION
|
||||
// =============================================================================================================
|
||||
|
||||
#define PROCESS_FAILED() do { CloseHandle(hThrd); CloseHandle(hProc); goto cleanup; } while (0)
|
||||
|
||||
static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
{
|
||||
int exitCode = EXIT_FAILURE;
|
||||
std::ios initFmt(NULL);
|
||||
initFmt.copyfmt(std::cerr);
|
||||
|
||||
@ -379,7 +449,9 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
std::cerr << " TIMED_EXEC_PASSES - Number of execution passes (default: " << DEFAULT_EXEC_LOOPS << ")" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: " << DEFAULT_WARMUP_LOOPS << ")" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_LOGFILE - Log-File Name (default: \"" << DEFAULT_LOGFILE << "\")" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_NO_CHECKS - Do *not* check the process exit codes" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_NO_PRIORITY - Do *not* adjust the process priorities" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_NO_PERIOD - Do *not* adjust system timer period" << std::endl;
|
||||
std::cerr << " TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
}
|
||||
@ -401,9 +473,9 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
/* ---------------------------------------------------------- */
|
||||
|
||||
tstring logFile(getFullPath(_T(DEFAULT_LOGFILE)));
|
||||
int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS;
|
||||
bool checkExitCodes = true;
|
||||
clock_type_t clock_type = CLOCK_WALLCLOCK;
|
||||
int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS;
|
||||
bool checkExitCodes = true, adjustPriority = true, adjustPeriod = true;
|
||||
|
||||
if (ENABLE_ENV_VARS)
|
||||
{
|
||||
@ -422,14 +494,21 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
}
|
||||
if (getEnvVariable(_T("TIMED_EXEC_NO_CHECKS"), temp))
|
||||
{
|
||||
checkExitCodes = (_tstoi(temp.c_str()) == 0);
|
||||
checkExitCodes = (!parseFlag(temp));
|
||||
}
|
||||
if (getEnvVariable(_T("TIMED_EXEC_NO_PRIORITY"), temp))
|
||||
{
|
||||
adjustPriority = (!parseFlag(temp));
|
||||
}
|
||||
if (getEnvVariable(_T("TIMED_EXEC_NO_PERIOD"), temp))
|
||||
{
|
||||
adjustPeriod = (!parseFlag(temp));
|
||||
}
|
||||
if (getEnvVariable(_T("TIMED_EXEC_CLOCK_TYPE"), temp))
|
||||
{
|
||||
if (!parseClockType(temp, clock_type))
|
||||
{
|
||||
_ftprintf(stderr, _T("Specified clock type \"%s\" is unsupported.\nPlease see the documentation for a list of supported clock types!\n\n"), temp.c_str());
|
||||
return EXIT_FAILURE;
|
||||
_ftprintf(stderr, _T("WARNING: Specified clock type \"%s\" is unsupported. Using default clock type!\n\n"), temp.c_str());
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -442,9 +521,22 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
_ftprintf(stderr, _T("Log File:\n%s\n\n"), logFile.c_str());
|
||||
std::cerr << "Warm-Up / Metering passes: " << maxWarmUpPasses << "x / " << maxPasses << 'x' << std::endl;
|
||||
|
||||
if (adjustPriority)
|
||||
{
|
||||
if (!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS))
|
||||
{
|
||||
std::cerr << "\nWARNING: Failed to set process priroity class!" << std::endl;
|
||||
std::cerr << "\nWARNING: Failed to adjust process priroity class!" << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
UINT timerPeriod = MAXUINT;
|
||||
if (adjustPeriod)
|
||||
{
|
||||
if (!setTimerResolution(timerPeriod))
|
||||
{
|
||||
std::cerr << "\nWARNING: Failed to adjust timer period!" << std::endl;
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
std::vector<double> stats_samples(maxPasses, 0.0);
|
||||
@ -466,29 +558,29 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
|
||||
HANDLE hThrd, hProc;
|
||||
|
||||
if (!createProcess(commandLine, hThrd, hProc))
|
||||
if (!createProcess(commandLine, hThrd, hProc, adjustPriority))
|
||||
{
|
||||
std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!waitForProcess(hProc))
|
||||
{
|
||||
std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
PROCESS_FAILED();
|
||||
}
|
||||
|
||||
if (g_aborted)
|
||||
{
|
||||
abortedHandlerRoutine(hProc);
|
||||
return EXIT_FAILURE;
|
||||
PROCESS_FAILED();
|
||||
}
|
||||
|
||||
const int exitCode = getProcessExitCode(hProc);
|
||||
if (checkExitCodes && (exitCode != 0))
|
||||
{
|
||||
std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
PROCESS_FAILED();
|
||||
}
|
||||
else
|
||||
{
|
||||
@ -511,36 +603,29 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
|
||||
HANDLE hThrd, hProc;
|
||||
|
||||
if (!createProcess(commandLine, hThrd, hProc, true))
|
||||
if (!createProcess(commandLine, hThrd, hProc, adjustPriority))
|
||||
{
|
||||
std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
}
|
||||
|
||||
if (ResumeThread(hThrd) == ((DWORD) -1))
|
||||
{
|
||||
std::cerr << "\n\nSYSTEM ERROR: Failed to resume child process!\n" << std::endl;
|
||||
TerminateProcess(hProc, UINT(-1));
|
||||
return EXIT_FAILURE;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
if (!waitForProcess(hProc))
|
||||
{
|
||||
std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
PROCESS_FAILED();
|
||||
}
|
||||
|
||||
if (g_aborted)
|
||||
{
|
||||
abortedHandlerRoutine(hProc);
|
||||
return EXIT_FAILURE;
|
||||
PROCESS_FAILED();
|
||||
}
|
||||
|
||||
const int exitCode = getProcessExitCode(hProc);
|
||||
if (checkExitCodes && (exitCode != 0))
|
||||
{
|
||||
std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl;
|
||||
return EXIT_FAILURE;
|
||||
PROCESS_FAILED();
|
||||
}
|
||||
else
|
||||
{
|
||||
@ -635,7 +720,11 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
||||
/* Goodbye! */
|
||||
/* ---------------------------------------------------------- */
|
||||
|
||||
return EXIT_SUCCESS;
|
||||
exitCode = EXIT_SUCCESS;
|
||||
|
||||
cleanup:
|
||||
resetTimerResolution(timerPeriod);
|
||||
return exitCode;
|
||||
}
|
||||
|
||||
// =============================================================================================================
|
||||
|
||||
@ -24,6 +24,6 @@
|
||||
|
||||
#define VERSION_MAJOR 1
|
||||
#define VERSION_MINOR_HI 0
|
||||
#define VERSION_MINOR_LO 5
|
||||
#define VERSION_MINOR_LO 6
|
||||
|
||||
#endif
|
||||
|
||||
Loading…
Reference in New Issue
Block a user