Muldersoft/TimedExec
Muldersoft/TimedExec
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity

Use GetProcessTimes() function for time measurements + added option to measure CPU time instead of wall-clock time.

Browse Source
This commit is contained in:
LoRd_MuldeR 2023-04-12 23:09:39 +02:00
parent 5464c19013
commit 0398acf4b1
11 changed files with 311 additions and 84 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@ -1,13 +1,12 @@
% ![](img/timedexec/banner.jpg) ![TimedExec](img/timedexec/banner.jpg)
TimedExec &ndash; README by LoRd_MuldeR &lt;<mulder2@gmx>&gt; | <http://muldersoft.com/>
% by LoRd_MuldeR &lt;<mulder2@gmx>&gt; | <http://muldersoft.com/>
Introduction Introduction
============ ============
**TimedExec** is a small utility for *benchmarking* command-line programs. It will *execute* the specified program with the specified command-line arguments and then *measure* the time that it takes for the execution to complete. In order to obtain *accurate* results, all measurements are implemented via *high-resolution* performance timers. And, since program execution times unavoidably are subject to certain variations (e.g. due to environmental noise), each test will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of "warm-up" passes can be performed *prior to* the first metering pass. The warm-up passes prevent caching effects from interfering with the execution times. **TimedExec** is a small utility for *benchmarking* command-line programs. It will *execute* the specified program with the specified command-line arguments and then *measure* the time that it takes for the execution to complete. In order to obtain *accurate* results, all measurements are implemented via *high-resolution* performance timers. And, since program execution times unavoidably are subject to certain variations (e.g. due to environmental noise), each test will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of "warm-up" passes can be performed *prior to* the first metering pass. The warm-up passes prevent caching effects from interfering with the execution times.
TimedExec will then compute the ***mean*** execution time as well as the ***median*** execution time of all metering passes. It will also record the *fastest* and *slowest* execution time that has been measured. Furthermore, TimedExec computes the *standard error* in order to determine ***confidence intervals*** from the benchmarking results^[[Konfidenzintervalle so einfach wie möglich erklärt](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf)]. These are the *ranges* which contain the program's “real” average execution time (expected value), *with very high probability*. All results will be saved to a log file. TimedExec will then compute the ***mean*** execution time as well as the ***median*** execution time of all metering passes. It will also record the *fastest* and *slowest* execution time that has been measured. Furthermore, TimedExec computes the *standard error* in order to determine [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf) from the benchmarking results. These are the *ranges* which contain the program's “real” average execution time (expected value), *with very high probability*. All results will be saved to a log file.
Usage Instructions Usage Instructions
@ -19,8 +18,8 @@ Usage Instructions
``` ```
=============================================================================== ===============================================================================
Timed Exec - Benchmarking Utility, Version 1.03 Timed Exec - Benchmarking Utility, Version 1.05
Copyright (c) 2018 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved. Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License <http://www.gnu.org/>. it under the terms of the GNU General Public License <http://www.gnu.org/>.
@ -35,8 +34,19 @@ Influential environment variables:
TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: 1) TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: 1)
TIMED_EXEC_LOGFILE - Log-File Name (default: "TimedExec.log") TIMED_EXEC_LOGFILE - Log-File Name (default: "TimedExec.log")
TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks
TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements
``` ```
Clock Types
-----------
*TimedExec* supports the following clock types:
- `WALLCLOCK` &ndash; the wall-clock time, also known as *elapsed real time* (default)
- `CPU_TOTAL` &ndash; total CPU time, i.e. sum of CPU time spent in "user" *and* "kernel" modes
- `CPU_USER` &ndash; CPU time spent in "user" mode only
- `CPU_KERNEL` &ndash; CPU time spent in "kernel" mode only
Usage Example Usage Example
------------- -------------
@ -64,13 +74,14 @@ Median Execution Time : 20.522 seconds
Standard Deviation : 7.697 seconds Standard Deviation : 7.697 seconds
Standard Error : 3.848 seconds Standard Error : 3.848 seconds
Fastest / Slowest Pass : 19.739 / 37.916 seconds Fastest / Slowest Pass : 19.739 / 37.916 seconds
Active Clock Type : WALLCLOCK (0)
=============================================================================== ===============================================================================
``` ```
Interpretation Interpretation
-------------- --------------
When comparing measurement results, the ***mean*** (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the ***median*** execution times usually is the preferable choice. That is because the *median* of a data sample is much more robust against outliers. When comparing measurement results, the ***mean*** (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the ***median*** execution times often is the better choice. That is because the *median* of a data sample is much more robust against outliers.
Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes &ndash; which is **not** possible in practice. In this situation, we can have a look at the ***confidence intervals***. These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one (higher confidence means broader interval, and vice versa). Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes &ndash; which is **not** possible in practice. In this situation, we can have a look at the ***confidence intervals***. These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one (higher confidence means broader interval, and vice versa).
@ -94,7 +105,7 @@ TimedExec is released under the terms of the [GNU General Public License](http:/
``` ```
Timed Exec - Command-Line Benchmarking Utility Timed Exec - Command-Line Benchmarking Utility
Copyright (c) 2018 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved. Copyright (c) 2018-2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
This program is free software; you can redistribute it and/or This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License modify it under the terms of the GNU General Public License

9
TimedExec_VS2010.vcxproj
View File

@ -21,6 +21,15 @@
<ItemGroup> <ItemGroup>
<ClCompile Include="src\TimedExec.cpp" /> <ClCompile Include="src\TimedExec.cpp" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<ResourceCompile Include="res\version.rc" />
</ItemGroup>
<ItemGroup>
<Manifest Include="res\compatibility.manifest" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="src\Version.h" />
</ItemGroup>
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{7BB6FBD8-4531-4CBB-B9EE-D440ACD7BCEC}</ProjectGuid> <ProjectGuid>{7BB6FBD8-4531-4CBB-B9EE-D440ACD7BCEC}</ProjectGuid>
<Keyword>Win32Proj</Keyword> <Keyword>Win32Proj</Keyword>

15
TimedExec_VS2010.vcxproj.filters
View File

@ -19,4 +19,19 @@
<Filter>Source Files</Filter> <Filter>Source Files</Filter>
</ClCompile> </ClCompile>
</ItemGroup> </ItemGroup>
<ItemGroup>
<ResourceCompile Include="res\version.rc">
<Filter>Resource Files</Filter>
</ResourceCompile>
</ItemGroup>
<ItemGroup>
<Manifest Include="res\compatibility.manifest">
<Filter>Resource Files</Filter>
</Manifest>
</ItemGroup>
<ItemGroup>
<ClInclude Include="src\Version.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project> </Project>

20
TimedExec_VS2017.vcxproj
View File

@ -21,38 +21,48 @@
<ItemGroup> <ItemGroup>
<ClCompile Include="src\TimedExec.cpp" /> <ClCompile Include="src\TimedExec.cpp" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<ClInclude Include="src\Version.h" />
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="res\version.rc" />
</ItemGroup>
<ItemGroup>
<Manifest Include="res\compatibility.manifest" />
</ItemGroup>
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{7BB6FBD8-4531-4CBB-B9EE-D440ACD7BCEC}</ProjectGuid> <ProjectGuid>{7BB6FBD8-4531-4CBB-B9EE-D440ACD7BCEC}</ProjectGuid>
<Keyword>Win32Proj</Keyword> <Keyword>Win32Proj</Keyword>
<RootNamespace>TimedExec</RootNamespace> <RootNamespace>TimedExec</RootNamespace>
<ProjectName>TimedExec</ProjectName> <ProjectName>TimedExec</ProjectName>
<WindowsTargetPlatformVersion>10.0.19041.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries> <UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet> <CharacterSet>Unicode</CharacterSet>
<PlatformToolset>v141_xp</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries> <UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet> <CharacterSet>Unicode</CharacterSet>
<PlatformToolset>v141_xp</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries> <UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization> <WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet> <CharacterSet>Unicode</CharacterSet>
<PlatformToolset>v141_xp</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries> <UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization> <WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet> <CharacterSet>Unicode</CharacterSet>
<PlatformToolset>v141_xp</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings"> <ImportGroup Label="ExtensionSettings">
@ -151,6 +161,7 @@
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration> <LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
<AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies> <AdditionalDependencies>$(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers> <ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
<MinimumRequiredVersion>5.1</MinimumRequiredVersion>
</Link> </Link>
</ItemDefinitionGroup> </ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'"> <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
@ -181,6 +192,7 @@
<LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration> <LinkTimeCodeGeneration>UseLinkTimeCodeGeneration</LinkTimeCodeGeneration>
<ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers> <ImageHasSafeExceptionHandlers>false</ImageHasSafeExceptionHandlers>
<AdditionalDependencies>Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies> <AdditionalDependencies>Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies)</AdditionalDependencies>
<MinimumRequiredVersion>5.2</MinimumRequiredVersion>
</Link> </Link>
</ItemDefinitionGroup> </ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />

15
TimedExec_VS2017.vcxproj.filters
View File

@ -19,4 +19,19 @@
<Filter>Source Files</Filter> <Filter>Source Files</Filter>
</ClCompile> </ClCompile>
</ItemGroup> </ItemGroup>
<ItemGroup>
<ClInclude Include="src\Version.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="res\version.rc">
<Filter>Resource Files</Filter>
</ResourceCompile>
</ItemGroup>
<ItemGroup>
<Manifest Include="res\compatibility.manifest">
<Filter>Resource Files</Filter>
</Manifest>
</ItemGroup>
</Project> </Project>

BIN
res/appicon.ico Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.7 KiB

17
res/compatibility.manifest Normal file
View File

@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
<application>
<!-- Windows Vista -->
<supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>
<!-- Windows 7 -->
<supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
<!-- Windows 8 -->
<supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
<!-- Windows 8.1 -->
<supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>
<!-- Windows 10 -->
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/>
</application>
</compatibility>
</assembly>

68
res/version.rc Normal file
View File

@ -0,0 +1,68 @@
/////////////////////////////////////////////////////////////////////////////
//
// Microsoft Visual C++ generated resource script.
//
#define APSTUDIO_READONLY_SYMBOLS
#include "WinResrc.h"
#undef APSTUDIO_READONLY_SYMBOLS
#include "../src/Version.h"
#define ___FULL_VERSION_STR___(X) #X
#define __FULL_VERSION_STR__(X,Y,Z) ___FULL_VERSION_STR___(X.Y##Z)
#define _FULL_VERSION_STR_(X,Y,Z) __FULL_VERSION_STR__(X,Y,Z)
#define FULL_VERSION_STR _FULL_VERSION_STR_(VERSION_MAJOR,VERSION_MINOR_HI,VERSION_MINOR_LO)
/////////////////////////////////////////////////////////////////////////////
//
// Neutral resources
//
#ifdef _WIN32
LANGUAGE LANG_NEUTRAL, SUBLANG_NEUTRAL
#pragma code_page(1252)
#endif //_WIN32
/////////////////////////////////////////////////////////////////////////////
//
// Icon
//
101 ICON "appicon.ico"
/////////////////////////////////////////////////////////////////////////////
//
// Version
//
VS_VERSION_INFO VERSIONINFO
FILEVERSION VERSION_MAJOR, VERSION_MINOR_HI, VERSION_MINOR_LO, 0
PRODUCTVERSION VERSION_MAJOR, VERSION_MINOR_HI, VERSION_MINOR_LO, 0
FILEFLAGSMASK 0x17L
#ifdef _DEBUG
FILEFLAGS 0x3L
#else
FILEFLAGS 0x2L
#endif
FILEOS 0x40004L
FILETYPE 0x1L
FILESUBTYPE 0x0L
BEGIN
BLOCK "StringFileInfo"
BEGIN
BLOCK "000004b0"
BEGIN
VALUE "ProductName", "TimedExec"
VALUE "FileDescription", "TimedExec"
VALUE "ProductVersion", FULL_VERSION_STR
VALUE "FileVersion", FULL_VERSION_STR
VALUE "InternalName", "TimedExec"
VALUE "OriginalFilename", "TimedExec.exe"
VALUE "LegalCopyright", "Created by LoRd_MuldeR <MuldeR2@GMX.de>"
VALUE "CompanyName", "Muldersoft"
VALUE "LegalTrademarks", "Muldersoft"
VALUE "Comments", "Released under the GNU General Public License"
END
END
BLOCK "VarFileInfo"
BEGIN
VALUE "Translation", 0x0, 1200
END
END

115
src/TimedExec.cpp
View File

@ -1,6 +1,6 @@
////////////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////////////
// Timed Exec - Command-Line Benchmarking Utility // Timed Exec - Command-Line Benchmarking Utility
// Copyright (c) 2018 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved. // Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
// //
// This program is free software; you can redistribute it and/or // This program is free software; you can redistribute it and/or
// modify it under the terms of the GNU General Public License // modify it under the terms of the GNU General Public License
@ -19,6 +19,8 @@
// http://www.gnu.org/licenses/gpl-2.0.txt // http://www.gnu.org/licenses/gpl-2.0.txt
////////////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////////////
#include "Version.h"
#include <cstdlib> #include <cstdlib>
#include <vector> #include <vector>
#include <algorithm> #include <algorithm>
@ -34,9 +36,6 @@
#include <ShellAPI.h> #include <ShellAPI.h>
#include <Shlwapi.h> #include <Shlwapi.h>
#define VERSION_MAJOR 1
#define VERSION_MINOR 4
#define DEFAULT_EXEC_LOOPS 5 #define DEFAULT_EXEC_LOOPS 5
#define DEFAULT_WARMUP_LOOPS 1 #define DEFAULT_WARMUP_LOOPS 1
#define DEFAULT_LOGFILE "TimedExec.log" #define DEFAULT_LOGFILE "TimedExec.log"
@ -51,10 +50,44 @@
static HANDLE g_hAbortEvent = NULL; static HANDLE g_hAbortEvent = NULL;
static volatile bool g_aborted = false; static volatile bool g_aborted = false;
/* clock for time measurement */
typedef enum
{
CLOCK_WALLCLOCK,
CLOCK_CPU_TOTAL,
CLOCK_CPU_USER,
CLOCK_CPU_KERNEL
}
clock_type_t;
// ============================================================================================================= // =============================================================================================================
// INTERNAL FUNCTIONS // INTERNAL FUNCTIONS
// ============================================================================================================= // =============================================================================================================
#define _PRINT_CLOCK_TYPE(X) case CLOCK_##X: return #X;
#define _PARSE_CLOCK_TYPE(X) if (!_tcsicmp(name.c_str(), _T(#X))) { clock_type = (CLOCK_##X); return true; }
static bool parseClockType(const tstring &name, clock_type_t &clock_type)
{
_PARSE_CLOCK_TYPE(WALLCLOCK)
_PARSE_CLOCK_TYPE(CPU_TOTAL)
_PARSE_CLOCK_TYPE(CPU_USER)
_PARSE_CLOCK_TYPE(CPU_KERNEL)
return false;
}
static const char *printClockType(const clock_type_t clock_type)
{
switch (clock_type)
{
_PRINT_CLOCK_TYPE(WALLCLOCK)
_PRINT_CLOCK_TYPE(CPU_TOTAL)
_PRINT_CLOCK_TYPE(CPU_USER)
_PRINT_CLOCK_TYPE(CPU_KERNEL)
}
return "N/A";
}
static bool getEnvVariable(const _TCHAR *const name, tstring &value) static bool getEnvVariable(const _TCHAR *const name, tstring &value)
{ {
std::vector<TCHAR> buffer; std::vector<TCHAR> buffer;
@ -81,26 +114,42 @@ static bool getEnvVariable(const _TCHAR *const name, tstring &value)
return false; return false;
} }
static LONGLONG getCurrentTime(void) static ULONGLONG fileTimeToU64(const PFILETIME fileTime)
{ {
LARGE_INTEGER timeValue; ULARGE_INTEGER temp;
if(!QueryPerformanceCounter(&timeValue)) temp.HighPart = fileTime->dwHighDateTime;
{ temp.LowPart = fileTime->dwLowDateTime;
std::cerr << "\n\nSYSTEM ERROR: Failed to query performance counter!\n" << std::endl; return temp.QuadPart;
_exit(EXIT_FAILURE);
}
return timeValue.QuadPart;
} }
static LONGLONG getTimerFrequency(void) static ULONGLONG getTimeElapsed(const ULONGLONG timeStart, const ULONGLONG timeExit)
{ {
LARGE_INTEGER timeValue; return (timeExit > timeStart) ? (timeExit - timeStart) : 0ULL;
if(!QueryPerformanceFrequency(&timeValue))
{
std::cerr << "\n\nSYSTEM ERROR: Failed to query performance counter frequency!\n" << std::endl;
_exit(EXIT_FAILURE);
} }
return timeValue.QuadPart;
static double getProcessTime(const HANDLE hProc, const clock_type_t clock_type)
{
FILETIME timeStart, timeExit, timeKernel, timeUser;
ULONGLONG result = 0ULL;
if (GetProcessTimes(hProc, &timeStart, &timeExit, &timeKernel, &timeUser))
{
switch (clock_type)
{
case CLOCK_WALLCLOCK:
result = getTimeElapsed(fileTimeToU64(&timeStart), fileTimeToU64(&timeExit));
break;
case CLOCK_CPU_USER:
result = fileTimeToU64(&timeUser);
break;
case CLOCK_CPU_KERNEL:
result = fileTimeToU64(&timeKernel);
break;
case CLOCK_CPU_TOTAL:
result = fileTimeToU64(&timeKernel) + fileTimeToU64(&timeUser);
break;
}
}
return static_cast<double>(result) / 10000000.0;
} }
static long long getCurrentFileSize(FILE *const filePtr) static long long getCurrentFileSize(FILE *const filePtr)
@ -308,8 +357,8 @@ static int timedExecMain(int argc, _TCHAR* argv[])
initFmt.copyfmt(std::cerr); initFmt.copyfmt(std::cerr);
std::cerr << "\n===============================================================================" << std::endl; std::cerr << "\n===============================================================================" << std::endl;
std::cerr << "Timed Exec - Benchmarking Utility, Version " << VERSION_MAJOR << '.' << std::setfill('0') << std::setw(2) << VERSION_MINOR << " [" __DATE__ "]" << std::endl; std::cerr << "Timed Exec - Benchmarking Utility, Version " << VERSION_MAJOR << '.' << std::setfill('0') << std::setw(2) << (10 * VERSION_MINOR_HI) + VERSION_MINOR_LO << " [" __DATE__ "]" << std::endl;
std::cerr << "Copyright (c) 2018 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.\n" << std::endl; std::cerr << "Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.\n" << std::endl;
std::cerr << "This program is free software: you can redistribute it and/or modify" << std::endl; std::cerr << "This program is free software: you can redistribute it and/or modify" << std::endl;
std::cerr << "it under the terms of the GNU General Public License <http://www.gnu.org/>." << std::endl; std::cerr << "it under the terms of the GNU General Public License <http://www.gnu.org/>." << std::endl;
std::cerr << "Note that this program is distributed with ABSOLUTELY NO WARRANTY." << std::endl; std::cerr << "Note that this program is distributed with ABSOLUTELY NO WARRANTY." << std::endl;
@ -330,7 +379,8 @@ static int timedExecMain(int argc, _TCHAR* argv[])
std::cerr << " TIMED_EXEC_PASSES - Number of execution passes (default: " << DEFAULT_EXEC_LOOPS << ")" << std::endl; std::cerr << " TIMED_EXEC_PASSES - Number of execution passes (default: " << DEFAULT_EXEC_LOOPS << ")" << std::endl;
std::cerr << " TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: " << DEFAULT_WARMUP_LOOPS << ")" << std::endl; std::cerr << " TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: " << DEFAULT_WARMUP_LOOPS << ")" << std::endl;
std::cerr << " TIMED_EXEC_LOGFILE - Log-File Name (default: \"" << DEFAULT_LOGFILE << "\")" << std::endl; std::cerr << " TIMED_EXEC_LOGFILE - Log-File Name (default: \"" << DEFAULT_LOGFILE << "\")" << std::endl;
std::cerr << " TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks\n" << std::endl; std::cerr << " TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks" << std::endl;
std::cerr << " TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements\n" << std::endl;
return EXIT_FAILURE; return EXIT_FAILURE;
} }
@ -353,6 +403,7 @@ static int timedExecMain(int argc, _TCHAR* argv[])
tstring logFile(getFullPath(_T(DEFAULT_LOGFILE))); tstring logFile(getFullPath(_T(DEFAULT_LOGFILE)));
int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS; int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS;
bool checkExitCodes = true; bool checkExitCodes = true;
clock_type_t clock_type = CLOCK_WALLCLOCK;
if (ENABLE_ENV_VARS) if (ENABLE_ENV_VARS)
{ {
@ -373,6 +424,14 @@ static int timedExecMain(int argc, _TCHAR* argv[])
{ {
checkExitCodes = (_tstoi(temp.c_str()) == 0); checkExitCodes = (_tstoi(temp.c_str()) == 0);
} }
if (getEnvVariable(_T("TIMED_EXEC_CLOCK_TYPE"), temp))
{
if (!parseClockType(temp, clock_type))
{
_ftprintf(stderr, _T("Specified clock type \"%s\" is unsupported.\nPlease see the documentation for a list of supported clock types!\n\n"), temp.c_str());
return EXIT_FAILURE;
}
}
} }
/* ---------------------------------------------------------- */ /* ---------------------------------------------------------- */
@ -383,14 +442,10 @@ static int timedExecMain(int argc, _TCHAR* argv[])
_ftprintf(stderr, _T("Log File:\n%s\n\n"), logFile.c_str()); _ftprintf(stderr, _T("Log File:\n%s\n\n"), logFile.c_str());
std::cerr << "Warm-Up / Metering passes: " << maxWarmUpPasses << "x / " << maxPasses << 'x' << std::endl; std::cerr << "Warm-Up / Metering passes: " << maxWarmUpPasses << "x / " << maxPasses << 'x' << std::endl;
const LONGLONG timerFrequency = getTimerFrequency();
if(!SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS))
{
if (!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS)) if (!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS))
{ {
std::cerr << "\nWARNING: Failed to set process priroity class!" << std::endl; std::cerr << "\nWARNING: Failed to set process priroity class!" << std::endl;
} }
}
std::vector<double> stats_samples(maxPasses, 0.0); std::vector<double> stats_samples(maxPasses, 0.0);
@ -462,8 +517,6 @@ static int timedExecMain(int argc, _TCHAR* argv[])
return EXIT_FAILURE; return EXIT_FAILURE;
} }
const LONGLONG timeStart = getCurrentTime();
if (ResumeThread(hThrd) == ((DWORD) -1)) if (ResumeThread(hThrd) == ((DWORD) -1))
{ {
std::cerr << "\n\nSYSTEM ERROR: Failed to resume child process!\n" << std::endl; std::cerr << "\n\nSYSTEM ERROR: Failed to resume child process!\n" << std::endl;
@ -477,9 +530,6 @@ static int timedExecMain(int argc, _TCHAR* argv[])
return EXIT_FAILURE; return EXIT_FAILURE;
} }
const LONGLONG timeFinish = getCurrentTime();
const double execTime = static_cast<double>(timeFinish - timeStart) / static_cast<double>(timerFrequency);
if (g_aborted) if (g_aborted)
{ {
abortedHandlerRoutine(hProc); abortedHandlerRoutine(hProc);
@ -497,6 +547,8 @@ static int timedExecMain(int argc, _TCHAR* argv[])
std::cerr << "\n>> Process terminated with exit code " << exitCode << '.' << std::endl; std::cerr << "\n>> Process terminated with exit code " << exitCode << '.' << std::endl;
} }
const double execTime = getProcessTime(hProc, clock_type);
std::cerr << std::setprecision(3) << std::fixed; std::cerr << std::setprecision(3) << std::fixed;
std::cerr << ">> Execution took " << execTime << " seconds." << std::endl; std::cerr << ">> Execution took " << execTime << " seconds." << std::endl;
std::cerr.copyfmt(initFmt); std::cerr.copyfmt(initFmt);
@ -548,6 +600,7 @@ static int timedExecMain(int argc, _TCHAR* argv[])
std::cerr << "Standard Deviation : " << standardDeviation << " seconds" << std::endl; std::cerr << "Standard Deviation : " << standardDeviation << " seconds" << std::endl;
std::cerr << "Standard Error : " << standardError << " seconds" << std::endl; std::cerr << "Standard Error : " << standardError << " seconds" << std::endl;
std::cerr << "Fastest / Slowest Pass : " << stats_fastest << " / " << stats_slowest << " seconds" << std::endl; std::cerr << "Fastest / Slowest Pass : " << stats_fastest << " / " << stats_slowest << " seconds" << std::endl;
std::cerr << "Active Clock Type : " << printClockType(clock_type) << " (" << clock_type << ')' << std::endl;
std::cerr << "===============================================================================\n" << std::endl; std::cerr << "===============================================================================\n" << std::endl;
std::cerr.copyfmt(initFmt); std::cerr.copyfmt(initFmt);

29
src/Version.h Normal file
View File

@ -0,0 +1,29 @@
//////////////////////////////////////////////////////////////////////////////////
// Timed Exec - Command-Line Benchmarking Utility
// Copyright (c) 2023 LoRd_MuldeR <mulder2@gmx.de>. Some rights reserved.
//
// This program is free software; you can redistribute it and/or
// modify it under the terms of the GNU General Public License
// as published by the Free Software Foundation; either version 2
// of the License, or (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program; if not, write to the Free Software
// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
//
// http://www.gnu.org/licenses/gpl-2.0.txt
//////////////////////////////////////////////////////////////////////////////////
#ifndef _INC_TIMED_EXEC_VERSION_H
#define _INC_TIMED_EXEC_VERSION_H
#define VERSION_MAJOR 1
#define VERSION_MINOR_HI 0
#define VERSION_MINOR_LO 5
#endif

10
z_build.bat
View File

@ -108,13 +108,12 @@ echo BEGIN PACKAGING [Release]
echo --------------------------------------------------------------------- echo ---------------------------------------------------------------------
mkdir "%PACK_PATH%" mkdir "%PACK_PATH%"
mkdir "%PACK_PATH%\x64"
copy "%~dp0\bin\Win32\Release\*.exe" "%PACK_PATH%" copy "%~dp0\bin\Win32\Release\TimedExec.exe" "%PACK_PATH%\TimedExec-x86.exe"
copy "%~dp0\bin\x64\.\Release\*.exe" "%PACK_PATH%\x64" copy "%~dp0\bin\x64\.\Release\TimedExec.exe" "%PACK_PATH%\TimedExec-x64.exe"
copy "%~dp0\LICENSE.html" "%PACK_PATH%" copy "%~dp0\LICENSE.html" "%PACK_PATH%"
"%PDOC_PATH%\pandoc.exe" --from markdown_github+pandoc_title_block+header_attributes+implicit_figures+inline_notes --to html5 --toc -N --standalone -H "%~dp0\..\Prerequisites\Pandoc\css\github-pandoc.inc" "README.md" | "%JAVA_HOME%\bin\java.exe" -jar "%~dp0\..\Prerequisites\HTMLCompressor\bin\htmlcompressor-1.5.3.jar" --compress-css -o "%PACK_PATH%\README.html" "%PDOC_PATH%\pandoc.exe" --from markdown_github+pandoc_title_block+header_attributes+implicit_figures+inline_notes --to html5 --toc -N --metadata title="TimedExec" --standalone -H "%~dp0\..\Prerequisites\Pandoc\css\github-pandoc.inc" "README.md" | "%JAVA_HOME%\bin\java.exe" -jar "%~dp0\..\Prerequisites\HTMLCompressor\bin\htmlcompressor-1.5.3.jar" --compress-css -o "%PACK_PATH%\README.html"
mkdir "%PACK_PATH%\img" mkdir "%PACK_PATH%\img"
mkdir "%PACK_PATH%\img\timedexec" mkdir "%PACK_PATH%\img\timedexec"
@ -125,13 +124,12 @@ REM ///////////////////////////////////////////////////////////////////////////
REM // Compress REM // Compress
REM /////////////////////////////////////////////////////////////////////////// REM ///////////////////////////////////////////////////////////////////////////
"%UPX3_PATH%\upx.exe" --best "%PACK_PATH%\*.exe" "%UPX3_PATH%\upx.exe" --best "%PACK_PATH%\*.exe"
"%UPX3_PATH%\upx.exe" --best "%PACK_PATH%\x64\*.exe"
REM /////////////////////////////////////////////////////////////////////////// REM ///////////////////////////////////////////////////////////////////////////
REM // Create version tag REM // Create version tag
REM /////////////////////////////////////////////////////////////////////////// REM ///////////////////////////////////////////////////////////////////////////
echo TimedExec> "%PACK_PATH%\BUILD_TAG" echo TimedExec> "%PACK_PATH%\BUILD_TAG"
echo Copyright (C) 2018 LoRd_MuldeR ^<MuldeR2@GMX.de^>>> "%PACK_PATH%\BUILD_TAG" echo Copyright (C) 2023 LoRd_MuldeR ^<MuldeR2@GMX.de^>>> "%PACK_PATH%\BUILD_TAG"
echo.>> "%PACK_PATH%\BUILD_TAG" echo.>> "%PACK_PATH%\BUILD_TAG"
echo Built on %ISO_DATE%, at %ISO_TIME%>> "%PACK_PATH%\BUILD_TAG" echo Built on %ISO_DATE%, at %ISO_TIME%>> "%PACK_PATH%\BUILD_TAG"
echo.>> "%PACK_PATH%\BUILD_TAG" echo.>> "%PACK_PATH%\BUILD_TAG"