Compare commits
2 Commits
Author | SHA1 | Date | |
---|---|---|---|
1f311da336 | |||
35033603d7 |
30
README.md
30
README.md
@ -4,9 +4,9 @@ by LoRd_MuldeR <<mulder2@gmx>> | <http://muldersoft.com/>
|
|||||||
Introduction
|
Introduction
|
||||||
============
|
============
|
||||||
|
|
||||||
**TimedExec** is a small utility for *benchmarking* command-line programs. It will *execute* the specified program with the specified command-line arguments and then *measure* the time that it takes for the execution to complete. In order to obtain *accurate* results, all measurements are implemented via *high-resolution* performance timers. And, since program execution times unavoidably are subject to certain variations (e.g. due to environmental noise), each test will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of "warm-up" passes can be performed *prior to* the first metering pass. The warm-up passes prevent caching effects from interfering with the execution times.
|
**TimedExec** is a small utility for *benchmarking* programs. It will *execute* the specified program with the specified command-line arguments and *measure* the time that it takes for the execution to complete. Because the execution time of a program unavoidably is subject to certain variations (e.g. due to environmental noise), each measurement will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of “warm-up” passes can be performed *prior to* the metering passes. The warm-up passes prevent caching effects from interfering with the measurement.
|
||||||
|
|
||||||
TimedExec will then compute the ***mean*** execution time as well as the ***median*** execution time of all metering passes. It will also record the *fastest* and *slowest* execution time that has been measured. Furthermore, TimedExec computes the *standard error* in order to determine [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf) from the benchmarking results. These are the *ranges* which contain the program's “real” average execution time (expected value), *with very high probability*. All results will be saved to a log file.
|
Once all metering passes have been completed, TimedExec will compute the ***mean*** execution time as well as the ***median*** execution time of the program. It will also record the *fastest* and *slowest* execution time that has been observed. Furthermore, TimedExec computes the *standard error*, in order to determine [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf) from the benchmarking results.
|
||||||
|
|
||||||
|
|
||||||
Usage Instructions
|
Usage Instructions
|
||||||
@ -18,7 +18,7 @@ Usage Instructions
|
|||||||
|
|
||||||
```
|
```
|
||||||
===============================================================================
|
===============================================================================
|
||||||
Timed Exec - Benchmarking Utility, Version 1.05
|
Timed Exec - Benchmarking Utility, Version 1.06
|
||||||
Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
|
Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
|
||||||
|
|
||||||
This program is free software: you can redistribute it and/or modify
|
This program is free software: you can redistribute it and/or modify
|
||||||
@ -33,7 +33,9 @@ Influential environment variables:
|
|||||||
TIMED_EXEC_PASSES - Number of execution passes (default: 5)
|
TIMED_EXEC_PASSES - Number of execution passes (default: 5)
|
||||||
TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: 1)
|
TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: 1)
|
||||||
TIMED_EXEC_LOGFILE - Log-File Name (default: "TimedExec.log")
|
TIMED_EXEC_LOGFILE - Log-File Name (default: "TimedExec.log")
|
||||||
TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks
|
TIMED_EXEC_NO_CHECKS - Do *not* check the process exit codes
|
||||||
|
TIMED_EXEC_NO_PRIORITY - Do *not* adjust the process priorities
|
||||||
|
TIMED_EXEC_NO_PERIOD - Do *not* adjust system timer period
|
||||||
TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements
|
TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -56,7 +58,6 @@ In the following example we use *TimedExec* to benchmark the program **`ping.exe
|
|||||||
TimedExec.exe C:\Windows\System32\ping.exe -n 12 www.google.com
|
TimedExec.exe C:\Windows\System32\ping.exe -n 12 www.google.com
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
Results
|
Results
|
||||||
=======
|
=======
|
||||||
|
|
||||||
@ -81,11 +82,16 @@ Active Clock Type : WALLCLOCK (0)
|
|||||||
Interpretation
|
Interpretation
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
When comparing measurement results, the ***mean*** (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the ***median*** execution times often is the better choice. That is because the *median* of a data sample is much more robust against outliers.
|
When comparing measurement results, the [***mean***](https://en.wikipedia.org/wiki/Arithmetic_mean) (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the [***median***](https://en.wikipedia.org/wiki/Median) of the execution times often is the better choice. That is because the *median* of a data sample is much more robust against outliers.
|
||||||
|
|
||||||
Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes – which is **not** possible in practice. In this situation, we can have a look at the ***confidence intervals***. These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one (higher confidence means broader interval, and vice versa).
|
Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes – which is **not** possible in practice. In this situation, we can have a look at the [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf). These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one. Higher confidence means broader interval, and vice versa.
|
||||||
|
|
||||||
Simply put, as long as the confidence intervals of program A and program B *overlap* (at least partially), we **must not** conclude that either of these programs runs faster (or slower) in the average case. ***No*** conclusion can be drawn in that case!
|
Simply put, as long as the confidence intervals of the runtime of program “A” and the runtime of program “B” *overlap*, we **must not** conclude that either of these programs runs faster (or slower). In fact, **no** real conclusion can be drawn in that case!
|
||||||
|
|
||||||
|
Limitations
|
||||||
|
-----------
|
||||||
|
|
||||||
|
This tools measures the runtime of *processes*. Because creating a process has a certain overhead, and because the system timer has a limited precision – usually in the range of a few milliseconds, but can be worse – this tool is **not** suitable for benchmarking programs or functions with *very short* runtime! The process to be measured should run *at least* for a couple of seconds, in order to get useful benchmark results. If you need to benchmark functions with *very short* runtime, it is recommended to use [*high-precision timers*](https://learn.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter) directly inside your program code, rather than launching separate processes.
|
||||||
|
|
||||||
|
|
||||||
Sources
|
Sources
|
||||||
@ -93,10 +99,10 @@ Sources
|
|||||||
|
|
||||||
The *TimedExec* source codes are managed by [**Git**](http://git-scm.com/doc) and are available from one of the official mirrors:
|
The *TimedExec* source codes are managed by [**Git**](http://git-scm.com/doc) and are available from one of the official mirrors:
|
||||||
|
|
||||||
* <tt>https://github.com/lordmulder/TimedExec.git</tt> ([Browse](https://github.com/lordmulder/TimedExec))
|
* `https://github.com/lordmulder/TimedExec.git` ([Browse](https://github.com/lordmulder/TimedExec))
|
||||||
* <tt>https://bitbucket.org/muldersoft/timedexec.git</tt> ([Browse](https://bitbucket.org/muldersoft/timedexec))
|
* `https://bitbucket.org/muldersoft/timedexec.git` ([Browse](https://bitbucket.org/muldersoft/timedexec))
|
||||||
* <tt>https://gitlab.com/timedexec/timedexec.git</tt> ([Browse](https://gitlab.com/timedexec/timedexec))
|
* `https://gitlab.com/timedexec/timedexec.git` ([Browse](https://gitlab.com/timedexec/timedexec))
|
||||||
|
* `https://punkindrublic.mooo.com:3000/Muldersoft/TimedExec.git` ([Browse](https://punkindrublic.mooo.com:3000/Muldersoft/TimedExec))
|
||||||
|
|
||||||
License
|
License
|
||||||
=======
|
=======
|
||||||
|
@ -115,7 +115,7 @@
|
|||||||
<Link>
|
<Link>
|
||||||
<SubSystem>Console</SubSystem>
|
<SubSystem>Console</SubSystem>
|
||||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
|
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
|
||||||
@ -130,7 +130,7 @@
|
|||||||
<Link>
|
<Link>
|
||||||
<SubSystem>Console</SubSystem>
|
<SubSystem>Console</SubSystem>
|
||||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
|
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
|
||||||
@ -159,7 +159,7 @@
|
|||||||
<EnableCOMDATFolding>true</EnableCOMDATFolding>
|
<EnableCOMDATFolding>true</EnableCOMDATFolding>
|
||||||
<OptimizeReferences>true</OptimizeReferences>
|
<OptimizeReferences>true</OptimizeReferences>
|
||||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
@ -190,7 +190,7 @@
|
|||||||
<OptimizeReferences>true</OptimizeReferences>
|
<OptimizeReferences>true</OptimizeReferences>
|
||||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||||
<AdditionalDependencies>Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
|
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
|
||||||
|
@ -116,7 +116,7 @@
|
|||||||
<Link>
|
<Link>
|
||||||
<SubSystem>Console</SubSystem>
|
<SubSystem>Console</SubSystem>
|
||||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
|
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
|
||||||
@ -130,7 +130,7 @@
|
|||||||
<Link>
|
<Link>
|
||||||
<SubSystem>Console</SubSystem>
|
<SubSystem>Console</SubSystem>
|
||||||
<GenerateDebugInformation>true</GenerateDebugInformation>
|
<GenerateDebugInformation>true</GenerateDebugInformation>
|
||||||
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
|
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
|
||||||
@ -159,7 +159,7 @@
|
|||||||
<EnableCOMDATFolding>true</EnableCOMDATFolding>
|
<EnableCOMDATFolding>true</EnableCOMDATFolding>
|
||||||
<OptimizeReferences>true</OptimizeReferences>
|
<OptimizeReferences>true</OptimizeReferences>
|
||||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||||
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||||
<MinimumRequiredVersion>5.1</MinimumRequiredVersion>
|
<MinimumRequiredVersion>5.1</MinimumRequiredVersion>
|
||||||
</Link>
|
</Link>
|
||||||
@ -191,7 +191,7 @@
|
|||||||
<OptimizeReferences>true</OptimizeReferences>
|
<OptimizeReferences>true</OptimizeReferences>
|
||||||
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
|
||||||
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
|
||||||
<AdditionalDependencies>Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
<AdditionalDependencies>Shell32.lib;Shlwapi.lib;Winmm.lib;%(AdditionalDependencies)</AdditionalDependencies>
|
||||||
<MinimumRequiredVersion>5.2</MinimumRequiredVersion>
|
<MinimumRequiredVersion>5.2</MinimumRequiredVersion>
|
||||||
</Link>
|
</Link>
|
||||||
</ItemDefinitionGroup>
|
</ItemDefinitionGroup>
|
||||||
|
@ -34,7 +34,12 @@
|
|||||||
#define WIN32_LEAN_AND_MEAN 1
|
#define WIN32_LEAN_AND_MEAN 1
|
||||||
#include <Windows.h>
|
#include <Windows.h>
|
||||||
#include <ShellAPI.h>
|
#include <ShellAPI.h>
|
||||||
#include <Shlwapi.h>
|
|
||||||
|
#if _WIN32_WINNT >= 0x0603
|
||||||
|
# include <timeapi.h>
|
||||||
|
#else
|
||||||
|
# include <MMSystem.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
#define DEFAULT_EXEC_LOOPS 5
|
#define DEFAULT_EXEC_LOOPS 5
|
||||||
#define DEFAULT_WARMUP_LOOPS 1
|
#define DEFAULT_WARMUP_LOOPS 1
|
||||||
@ -42,9 +47,9 @@
|
|||||||
#define ENABLE_ENV_VARS true
|
#define ENABLE_ENV_VARS true
|
||||||
|
|
||||||
#ifdef _UNICODE
|
#ifdef _UNICODE
|
||||||
#define tstring std::wstring
|
# define tstring std::wstring
|
||||||
#else
|
#else
|
||||||
#define tstring std::string
|
# define tstring std::string
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
static HANDLE g_hAbortEvent = NULL;
|
static HANDLE g_hAbortEvent = NULL;
|
||||||
@ -88,10 +93,49 @@ static const char *printClockType(const clock_type_t clock_type)
|
|||||||
return "N/A";
|
return "N/A";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static bool setTimerResolution(UINT& period)
|
||||||
|
{
|
||||||
|
TIMECAPS timecaps;
|
||||||
|
if (timeGetDevCaps(&timecaps, sizeof(TIMECAPS)) == MMSYSERR_NOERROR)
|
||||||
|
{
|
||||||
|
if (timeBeginPeriod(timecaps.wPeriodMin) == MMSYSERR_NOERROR)
|
||||||
|
{
|
||||||
|
period = timecaps.wPeriodMin;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
period = MAXUINT;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void resetTimerResolution(const UINT period)
|
||||||
|
{
|
||||||
|
if (period != MAXUINT)
|
||||||
|
{
|
||||||
|
timeEndPeriod(period);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static tstring trim(const std::vector<_TCHAR> &buffer)
|
||||||
|
{
|
||||||
|
std::vector<_TCHAR>::const_iterator left;
|
||||||
|
for (left = buffer.cbegin(); (left != buffer.cend()) && (*left <= 0x20); ++left);
|
||||||
|
if (left != buffer.cend())
|
||||||
|
{
|
||||||
|
std::vector<_TCHAR>::const_reverse_iterator right;
|
||||||
|
for (right = buffer.crbegin(); (right != buffer.crend()) && (*right <= 0x20); ++right);
|
||||||
|
if (right != buffer.crend())
|
||||||
|
{
|
||||||
|
return tstring(left, right.base());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return tstring();
|
||||||
|
}
|
||||||
|
|
||||||
static bool getEnvVariable(const _TCHAR *const name, tstring &value)
|
static bool getEnvVariable(const _TCHAR *const name, tstring &value)
|
||||||
{
|
{
|
||||||
std::vector<TCHAR> buffer;
|
std::vector<_TCHAR> buffer(MAX_PATH);
|
||||||
|
value.clear();
|
||||||
for (int i = 0; i < 3; ++i)
|
for (int i = 0; i < 3; ++i)
|
||||||
{
|
{
|
||||||
const DWORD result = GetEnvironmentVariable(name, buffer.data(), (DWORD)buffer.size());
|
const DWORD result = GetEnvironmentVariable(name, buffer.data(), (DWORD)buffer.size());
|
||||||
@ -99,21 +143,28 @@ static bool getEnvVariable(const _TCHAR *const name, tstring &value)
|
|||||||
{
|
{
|
||||||
break; /*failed*/
|
break; /*failed*/
|
||||||
}
|
}
|
||||||
if (result > buffer.size())
|
else if (result > buffer.size())
|
||||||
{
|
{
|
||||||
buffer.resize(result); /*adjust buffer*/
|
buffer.resize(result);
|
||||||
}
|
}
|
||||||
else if (result < buffer.size())
|
else if (result < buffer.size())
|
||||||
{
|
{
|
||||||
value = tstring(buffer.data());
|
value = trim(buffer);
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
value.clear();
|
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static bool parseFlag(const tstring &value)
|
||||||
|
{
|
||||||
|
if ((!_tcsicmp(value.c_str(), _T("yes"))) || (!_tcsicmp(value.c_str(), _T("true"))))
|
||||||
|
{
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return (_tcstol(value.c_str(), NULL, 10) > 0L);
|
||||||
|
}
|
||||||
|
|
||||||
static ULONGLONG fileTimeToU64(const PFILETIME fileTime)
|
static ULONGLONG fileTimeToU64(const PFILETIME fileTime)
|
||||||
{
|
{
|
||||||
ULARGE_INTEGER temp;
|
ULARGE_INTEGER temp;
|
||||||
@ -170,8 +221,7 @@ static bool checkBinary(const tstring &filePath)
|
|||||||
|
|
||||||
static tstring getFullPath(const _TCHAR *const fileName)
|
static tstring getFullPath(const _TCHAR *const fileName)
|
||||||
{
|
{
|
||||||
std::vector<TCHAR> buffer;
|
std::vector<TCHAR> buffer(MAX_PATH);
|
||||||
|
|
||||||
for (int i = 0; i < 3; ++i)
|
for (int i = 0; i < 3; ++i)
|
||||||
{
|
{
|
||||||
const DWORD result = GetFullPathName(fileName, (DWORD)buffer.size(), buffer.data(), NULL);
|
const DWORD result = GetFullPathName(fileName, (DWORD)buffer.size(), buffer.data(), NULL);
|
||||||
@ -179,32 +229,33 @@ static tstring getFullPath(const _TCHAR *const fileName)
|
|||||||
{
|
{
|
||||||
break; /*failed*/
|
break; /*failed*/
|
||||||
}
|
}
|
||||||
if (result > buffer.size())
|
else if (result > buffer.size())
|
||||||
{
|
{
|
||||||
buffer.resize(result); /*adjust buffer*/
|
buffer.resize(result);
|
||||||
}
|
}
|
||||||
else if (result < buffer.size())
|
else if (result < buffer.size())
|
||||||
{
|
{
|
||||||
return tstring(buffer.data());
|
return tstring(buffer.begin(), buffer.end());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return tstring(fileName);
|
return tstring(fileName);
|
||||||
}
|
}
|
||||||
|
|
||||||
static tstring getFileNameOnly(const tstring &filePath)
|
static tstring getFileNameOnly(const tstring &filePath)
|
||||||
{
|
{
|
||||||
TCHAR *buffer = _tcsdup(filePath.c_str());
|
for (tstring::const_reverse_iterator iter = filePath.crbegin(); iter != filePath.crend(); ++iter)
|
||||||
if (!buffer)
|
|
||||||
{
|
{
|
||||||
|
if ((*iter == _T('/')) || (*iter == _T('\\')))
|
||||||
|
{
|
||||||
|
tstring::const_iterator offset = iter.base();
|
||||||
|
if (offset != filePath.cend())
|
||||||
|
{
|
||||||
|
return tstring(offset, filePath.cend());
|
||||||
|
}
|
||||||
return tstring();
|
return tstring();
|
||||||
}
|
}
|
||||||
|
}
|
||||||
PathStripPath(buffer);
|
return filePath;
|
||||||
|
|
||||||
const tstring result(buffer);
|
|
||||||
free(buffer);
|
|
||||||
return result;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
static double computeMedian(std::vector<double> &data)
|
static double computeMedian(std::vector<double> &data)
|
||||||
@ -224,6 +275,25 @@ static double computeMedian(std::vector<double> &data)
|
|||||||
: data[center];
|
: data[center];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static void appendStr(tstring &commandLine, const tstring &token)
|
||||||
|
{
|
||||||
|
for (tstring::const_iterator iter = token.cbegin(); iter != token.cend(); ++iter)
|
||||||
|
{
|
||||||
|
if (*iter)
|
||||||
|
{
|
||||||
|
if (*iter == _T('"'))
|
||||||
|
{
|
||||||
|
commandLine += _T('\\');
|
||||||
|
}
|
||||||
|
commandLine += *iter;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
||||||
{
|
{
|
||||||
commandLine.clear();
|
commandLine.clear();
|
||||||
@ -240,7 +310,6 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
|||||||
for(int i = 1; i < nArgs; i++)
|
for(int i = 1; i < nArgs; i++)
|
||||||
{
|
{
|
||||||
tstring token;
|
tstring token;
|
||||||
|
|
||||||
if (i > 1)
|
if (i > 1)
|
||||||
{
|
{
|
||||||
token = tstring(szArglist[i]);
|
token = tstring(szArglist[i]);
|
||||||
@ -249,17 +318,16 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
|||||||
else
|
else
|
||||||
{
|
{
|
||||||
token = getFullPath(szArglist[i]);
|
token = getFullPath(szArglist[i]);
|
||||||
programFile += token;
|
programFile = token;
|
||||||
}
|
}
|
||||||
|
if (token.find(_T(' ')) == tstring::npos)
|
||||||
if (token.find(' ') == tstring::npos)
|
|
||||||
{
|
{
|
||||||
commandLine += token;
|
appendStr(commandLine, token);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
commandLine += _T('"');
|
commandLine += _T('"');
|
||||||
commandLine += token;
|
appendStr(commandLine, token);
|
||||||
commandLine += _T('"');
|
commandLine += _T('"');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -268,14 +336,14 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile)
|
|||||||
return nArgs;
|
return nArgs;
|
||||||
}
|
}
|
||||||
|
|
||||||
static bool createProcess(const tstring &commandLine, HANDLE &hThrd, HANDLE &hProc, const bool suspended = false)
|
static bool createProcess(const tstring &commandLine, HANDLE &hThrd, HANDLE &hProc, const bool highPriority = false)
|
||||||
{
|
{
|
||||||
STARTUPINFO startInfo;
|
STARTUPINFO startInfo;
|
||||||
SecureZeroMemory(&startInfo, sizeof(STARTUPINFO));
|
SecureZeroMemory(&startInfo, sizeof(STARTUPINFO));
|
||||||
PROCESS_INFORMATION processInfo;
|
PROCESS_INFORMATION processInfo;
|
||||||
SecureZeroMemory(&processInfo, sizeof(PROCESS_INFORMATION));
|
SecureZeroMemory(&processInfo, sizeof(PROCESS_INFORMATION));
|
||||||
|
|
||||||
if (!CreateProcess(NULL, (LPTSTR)commandLine.c_str(), NULL, NULL, false, HIGH_PRIORITY_CLASS | (suspended ? CREATE_SUSPENDED : 0), NULL, NULL, &startInfo, &processInfo))
|
if (!CreateProcess(NULL, (LPTSTR)commandLine.c_str(), NULL, NULL, false, highPriority ? HIGH_PRIORITY_CLASS : 0U, NULL, NULL, &startInfo, &processInfo))
|
||||||
{
|
{
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
@ -308,8 +376,7 @@ static int getProcessExitCode(const HANDLE &hProc)
|
|||||||
{
|
{
|
||||||
return *reinterpret_cast<int*>(&exitCode);
|
return *reinterpret_cast<int*>(&exitCode);
|
||||||
}
|
}
|
||||||
|
return -1;
|
||||||
return 0;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
static void abortedHandlerRoutine(const HANDLE &hProc)
|
static void abortedHandlerRoutine(const HANDLE &hProc)
|
||||||
@ -351,8 +418,11 @@ static LONG WINAPI crashHandlerRoutine(struct _EXCEPTION_POINTERS *ExceptionInfo
|
|||||||
// MAIN FUNCTION
|
// MAIN FUNCTION
|
||||||
// =============================================================================================================
|
// =============================================================================================================
|
||||||
|
|
||||||
|
#define PROCESS_FAILED() do { CloseHandle(hThrd); CloseHandle(hProc); goto cleanup; } while (0)
|
||||||
|
|
||||||
static int timedExecMain(int argc, _TCHAR* argv[])
|
static int timedExecMain(int argc, _TCHAR* argv[])
|
||||||
{
|
{
|
||||||
|
int exitCode = EXIT_FAILURE;
|
||||||
std::ios initFmt(NULL);
|
std::ios initFmt(NULL);
|
||||||
initFmt.copyfmt(std::cerr);
|
initFmt.copyfmt(std::cerr);
|
||||||
|
|
||||||
@ -379,7 +449,9 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
std::cerr << " TIMED_EXEC_PASSES - Number of execution passes (default: " << DEFAULT_EXEC_LOOPS << ")" << std::endl;
|
std::cerr << " TIMED_EXEC_PASSES - Number of execution passes (default: " << DEFAULT_EXEC_LOOPS << ")" << std::endl;
|
||||||
std::cerr << " TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: " << DEFAULT_WARMUP_LOOPS << ")" << std::endl;
|
std::cerr << " TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: " << DEFAULT_WARMUP_LOOPS << ")" << std::endl;
|
||||||
std::cerr << " TIMED_EXEC_LOGFILE - Log-File Name (default: \"" << DEFAULT_LOGFILE << "\")" << std::endl;
|
std::cerr << " TIMED_EXEC_LOGFILE - Log-File Name (default: \"" << DEFAULT_LOGFILE << "\")" << std::endl;
|
||||||
std::cerr << " TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks" << std::endl;
|
std::cerr << " TIMED_EXEC_NO_CHECKS - Do *not* check the process exit codes" << std::endl;
|
||||||
|
std::cerr << " TIMED_EXEC_NO_PRIORITY - Do *not* adjust the process priorities" << std::endl;
|
||||||
|
std::cerr << " TIMED_EXEC_NO_PERIOD - Do *not* adjust system timer period" << std::endl;
|
||||||
std::cerr << " TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements\n" << std::endl;
|
std::cerr << " TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
return EXIT_FAILURE;
|
||||||
}
|
}
|
||||||
@ -401,9 +473,9 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
/* ---------------------------------------------------------- */
|
/* ---------------------------------------------------------- */
|
||||||
|
|
||||||
tstring logFile(getFullPath(_T(DEFAULT_LOGFILE)));
|
tstring logFile(getFullPath(_T(DEFAULT_LOGFILE)));
|
||||||
int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS;
|
|
||||||
bool checkExitCodes = true;
|
|
||||||
clock_type_t clock_type = CLOCK_WALLCLOCK;
|
clock_type_t clock_type = CLOCK_WALLCLOCK;
|
||||||
|
int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS;
|
||||||
|
bool checkExitCodes = true, adjustPriority = true, adjustPeriod = true;
|
||||||
|
|
||||||
if (ENABLE_ENV_VARS)
|
if (ENABLE_ENV_VARS)
|
||||||
{
|
{
|
||||||
@ -422,14 +494,21 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
}
|
}
|
||||||
if (getEnvVariable(_T("TIMED_EXEC_NO_CHECKS"), temp))
|
if (getEnvVariable(_T("TIMED_EXEC_NO_CHECKS"), temp))
|
||||||
{
|
{
|
||||||
checkExitCodes = (_tstoi(temp.c_str()) == 0);
|
checkExitCodes = (!parseFlag(temp));
|
||||||
|
}
|
||||||
|
if (getEnvVariable(_T("TIMED_EXEC_NO_PRIORITY"), temp))
|
||||||
|
{
|
||||||
|
adjustPriority = (!parseFlag(temp));
|
||||||
|
}
|
||||||
|
if (getEnvVariable(_T("TIMED_EXEC_NO_PERIOD"), temp))
|
||||||
|
{
|
||||||
|
adjustPeriod = (!parseFlag(temp));
|
||||||
}
|
}
|
||||||
if (getEnvVariable(_T("TIMED_EXEC_CLOCK_TYPE"), temp))
|
if (getEnvVariable(_T("TIMED_EXEC_CLOCK_TYPE"), temp))
|
||||||
{
|
{
|
||||||
if (!parseClockType(temp, clock_type))
|
if (!parseClockType(temp, clock_type))
|
||||||
{
|
{
|
||||||
_ftprintf(stderr, _T("Specified clock type \"%s\" is unsupported.\nPlease see the documentation for a list of supported clock types!\n\n"), temp.c_str());
|
_ftprintf(stderr, _T("WARNING: Specified clock type \"%s\" is unsupported. Using default clock type!\n\n"), temp.c_str());
|
||||||
return EXIT_FAILURE;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -442,9 +521,22 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
_ftprintf(stderr, _T("Log File:\n%s\n\n"), logFile.c_str());
|
_ftprintf(stderr, _T("Log File:\n%s\n\n"), logFile.c_str());
|
||||||
std::cerr << "Warm-Up / Metering passes: " << maxWarmUpPasses << "x / " << maxPasses << 'x' << std::endl;
|
std::cerr << "Warm-Up / Metering passes: " << maxWarmUpPasses << "x / " << maxPasses << 'x' << std::endl;
|
||||||
|
|
||||||
|
if (adjustPriority)
|
||||||
|
{
|
||||||
if (!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS))
|
if (!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS))
|
||||||
{
|
{
|
||||||
std::cerr << "\nWARNING: Failed to set process priroity class!" << std::endl;
|
std::cerr << "\nWARNING: Failed to adjust process priroity class!" << std::endl;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
UINT timerPeriod = MAXUINT;
|
||||||
|
if (adjustPeriod)
|
||||||
|
{
|
||||||
|
if (!setTimerResolution(timerPeriod))
|
||||||
|
{
|
||||||
|
std::cerr << "\nWARNING: Failed to adjust timer period!" << std::endl;
|
||||||
|
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
std::vector<double> stats_samples(maxPasses, 0.0);
|
std::vector<double> stats_samples(maxPasses, 0.0);
|
||||||
@ -466,29 +558,29 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
|
|
||||||
HANDLE hThrd, hProc;
|
HANDLE hThrd, hProc;
|
||||||
|
|
||||||
if (!createProcess(commandLine, hThrd, hProc))
|
if (!createProcess(commandLine, hThrd, hProc, adjustPriority))
|
||||||
{
|
{
|
||||||
std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl;
|
std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
goto cleanup;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!waitForProcess(hProc))
|
if (!waitForProcess(hProc))
|
||||||
{
|
{
|
||||||
std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl;
|
std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
PROCESS_FAILED();
|
||||||
}
|
}
|
||||||
|
|
||||||
if (g_aborted)
|
if (g_aborted)
|
||||||
{
|
{
|
||||||
abortedHandlerRoutine(hProc);
|
abortedHandlerRoutine(hProc);
|
||||||
return EXIT_FAILURE;
|
PROCESS_FAILED();
|
||||||
}
|
}
|
||||||
|
|
||||||
const int exitCode = getProcessExitCode(hProc);
|
const int exitCode = getProcessExitCode(hProc);
|
||||||
if (checkExitCodes && (exitCode != 0))
|
if (checkExitCodes && (exitCode != 0))
|
||||||
{
|
{
|
||||||
std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl;
|
std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
PROCESS_FAILED();
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
@ -511,36 +603,29 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
|
|
||||||
HANDLE hThrd, hProc;
|
HANDLE hThrd, hProc;
|
||||||
|
|
||||||
if (!createProcess(commandLine, hThrd, hProc, true))
|
if (!createProcess(commandLine, hThrd, hProc, adjustPriority))
|
||||||
{
|
{
|
||||||
std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl;
|
std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
goto cleanup;
|
||||||
}
|
|
||||||
|
|
||||||
if (ResumeThread(hThrd) == ((DWORD) -1))
|
|
||||||
{
|
|
||||||
std::cerr << "\n\nSYSTEM ERROR: Failed to resume child process!\n" << std::endl;
|
|
||||||
TerminateProcess(hProc, UINT(-1));
|
|
||||||
return EXIT_FAILURE;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!waitForProcess(hProc))
|
if (!waitForProcess(hProc))
|
||||||
{
|
{
|
||||||
std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl;
|
std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
PROCESS_FAILED();
|
||||||
}
|
}
|
||||||
|
|
||||||
if (g_aborted)
|
if (g_aborted)
|
||||||
{
|
{
|
||||||
abortedHandlerRoutine(hProc);
|
abortedHandlerRoutine(hProc);
|
||||||
return EXIT_FAILURE;
|
PROCESS_FAILED();
|
||||||
}
|
}
|
||||||
|
|
||||||
const int exitCode = getProcessExitCode(hProc);
|
const int exitCode = getProcessExitCode(hProc);
|
||||||
if (checkExitCodes && (exitCode != 0))
|
if (checkExitCodes && (exitCode != 0))
|
||||||
{
|
{
|
||||||
std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl;
|
std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl;
|
||||||
return EXIT_FAILURE;
|
PROCESS_FAILED();
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
@ -635,7 +720,11 @@ static int timedExecMain(int argc, _TCHAR* argv[])
|
|||||||
/* Goodbye! */
|
/* Goodbye! */
|
||||||
/* ---------------------------------------------------------- */
|
/* ---------------------------------------------------------- */
|
||||||
|
|
||||||
return EXIT_SUCCESS;
|
exitCode = EXIT_SUCCESS;
|
||||||
|
|
||||||
|
cleanup:
|
||||||
|
resetTimerResolution(timerPeriod);
|
||||||
|
return exitCode;
|
||||||
}
|
}
|
||||||
|
|
||||||
// =============================================================================================================
|
// =============================================================================================================
|
||||||
|
@ -24,6 +24,6 @@
|
|||||||
|
|
||||||
#define VERSION_MAJOR 1
|
#define VERSION_MAJOR 1
|
||||||
#define VERSION_MINOR_HI 0
|
#define VERSION_MINOR_HI 0
|
||||||
#define VERSION_MINOR_LO 5
|
#define VERSION_MINOR_LO 6
|
||||||
|
|
||||||
#endif
|
#endif
|
||||||
|
Loading…
Reference in New Issue
Block a user