From 0398acf4b118c0a56b9383c0d46b7e561d8cb021 Mon Sep 17 00:00:00 2001 From: LoRd_MuldeR Date: Wed, 12 Apr 2023 23:09:39 +0200 Subject: [PATCH] Use GetProcessTimes() function for time measurements + added option to measure CPU time instead of wall-clock time. --- README.md | 27 +++-- TimedExec_VS2010.vcxproj | 9 ++ TimedExec_VS2010.vcxproj.filters | 15 +++ TimedExec_VS2017.vcxproj | 20 +++- TimedExec_VS2017.vcxproj.filters | 15 +++ res/appicon.ico | Bin 0 -> 7886 bytes res/compatibility.manifest | 17 +++ res/version.rc | 68 ++++++++++++ src/TimedExec.cpp | 183 ++++++++++++++++++++----------- src/Version.h | 29 +++++ z_build.bat | 12 +- 11 files changed, 311 insertions(+), 84 deletions(-) create mode 100644 res/appicon.ico create mode 100644 res/compatibility.manifest create mode 100644 res/version.rc create mode 100644 src/Version.h diff --git a/README.md b/README.md index b2f1c9e..45b665b 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,12 @@ -% ![](img/timedexec/banner.jpg) -TimedExec – README -% by LoRd_MuldeR <> | +![TimedExec](img/timedexec/banner.jpg) +by LoRd_MuldeR <> | Introduction ============ **TimedExec** is a small utility for *benchmarking* command-line programs. It will *execute* the specified program with the specified command-line arguments and then *measure* the time that it takes for the execution to complete. In order to obtain *accurate* results, all measurements are implemented via *high-resolution* performance timers. And, since program execution times unavoidably are subject to certain variations (e.g. due to environmental noise), each test will be repeated *multiple* times. The number of metering passes can be configured as desired. Optionally, a number of "warm-up" passes can be performed *prior to* the first metering pass. The warm-up passes prevent caching effects from interfering with the execution times. -TimedExec will then compute the ***mean*** execution time as well as the ***median*** execution time of all metering passes. It will also record the *fastest* and *slowest* execution time that has been measured. Furthermore, TimedExec computes the *standard error* in order to determine ***confidence intervals*** from the benchmarking results^[[Konfidenzintervalle so einfach wie möglich erklärt](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf)]. These are the *ranges* which contain the program's “real” average execution time (expected value), *with very high probability*. All results will be saved to a log file. +TimedExec will then compute the ***mean*** execution time as well as the ***median*** execution time of all metering passes. It will also record the *fastest* and *slowest* execution time that has been measured. Furthermore, TimedExec computes the *standard error* in order to determine [***confidence intervals***](http://www.uni-siegen.de/phil/sozialwissenschaften/soziologie/mitarbeiter/ludwig-mayerhofer/statistik/statistik_downloads/konfidenzintervalle.pdf) from the benchmarking results. These are the *ranges* which contain the program's “real” average execution time (expected value), *with very high probability*. All results will be saved to a log file. Usage Instructions @@ -19,8 +18,8 @@ Usage Instructions ``` =============================================================================== -Timed Exec - Benchmarking Utility, Version 1.03 -Copyright (c) 2018 LoRd_MuldeR . Some rights reserved. +Timed Exec - Benchmarking Utility, Version 1.05 +Copyright (c) 2023 LoRd_MuldeR . Some rights reserved. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License . @@ -35,8 +34,19 @@ Influential environment variables: TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: 1) TIMED_EXEC_LOGFILE - Log-File Name (default: "TimedExec.log") TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks + TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements ``` +Clock Types +----------- + +*TimedExec* supports the following clock types: + +- `WALLCLOCK` – the wall-clock time, also known as *elapsed real time* (default) +- `CPU_TOTAL` – total CPU time, i.e. sum of CPU time spent in "user" *and* "kernel" modes +- `CPU_USER` – CPU time spent in "user" mode only +- `CPU_KERNEL` – CPU time spent in "kernel" mode only + Usage Example ------------- @@ -64,13 +74,14 @@ Median Execution Time : 20.522 seconds Standard Deviation : 7.697 seconds Standard Error : 3.848 seconds Fastest / Slowest Pass : 19.739 / 37.916 seconds +Active Clock Type : WALLCLOCK (0) =============================================================================== ``` Interpretation -------------- -When comparing measurement results, the ***mean*** (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the ***median*** execution times usually is the preferable choice. That is because the *median* of a data sample is much more robust against outliers. +When comparing measurement results, the ***mean*** (average) execution time may seem like the most obvious choice. However, it has to be noted that the *mean* of a data sample is highly sensitive to “outliers” and therefore can be misleading! This is especially true, when there exists a lot of variation in the data sample. Consequently, comparing the ***median*** execution times often is the better choice. That is because the *median* of a data sample is much more robust against outliers. Furthermore, it is important to keep in mind that the *mean* (or *median*) execution time computed from a limited number of metering passes only yields an ***estimate*** of the program's “real” average execution time (expected value). The “real” value can only be determined accurately from an *infitinte* number of metering passes – which is **not** possible in practice. In this situation, we can have a look at the ***confidence intervals***. These intervals contain the “real” value, *with very high probability*. The most commonly used *confidence interval* is the “95%” one (higher confidence means broader interval, and vice versa). @@ -94,7 +105,7 @@ TimedExec is released under the terms of the [GNU General Public License](http:/ ``` Timed Exec - Command-Line Benchmarking Utility -Copyright (c) 2018 LoRd_MuldeR . Some rights reserved. +Copyright (c) 2018-2023 LoRd_MuldeR . Some rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License diff --git a/TimedExec_VS2010.vcxproj b/TimedExec_VS2010.vcxproj index cb6faba..f795cb4 100644 --- a/TimedExec_VS2010.vcxproj +++ b/TimedExec_VS2010.vcxproj @@ -21,6 +21,15 @@ + + + + + + + + + {7BB6FBD8-4531-4CBB-B9EE-D440ACD7BCEC} Win32Proj diff --git a/TimedExec_VS2010.vcxproj.filters b/TimedExec_VS2010.vcxproj.filters index 3735910..e869e35 100644 --- a/TimedExec_VS2010.vcxproj.filters +++ b/TimedExec_VS2010.vcxproj.filters @@ -19,4 +19,19 @@ Source Files + + + Resource Files + + + + + Resource Files + + + + + Header Files + + \ No newline at end of file diff --git a/TimedExec_VS2017.vcxproj b/TimedExec_VS2017.vcxproj index 0008056..24a60b8 100644 --- a/TimedExec_VS2017.vcxproj +++ b/TimedExec_VS2017.vcxproj @@ -21,38 +21,48 @@ + + + + + + + + + {7BB6FBD8-4531-4CBB-B9EE-D440ACD7BCEC} Win32Proj TimedExec TimedExec + 10.0.19041.0 Application true Unicode - v141_xp + v141 Application true Unicode - v141_xp + v141 Application false true Unicode - v141_xp + v141 Application false true Unicode - v141_xp + v141 @@ -151,6 +161,7 @@ UseLinkTimeCodeGeneration $(SolutionDir)\etc\EncodePointer\lib\EncodePointer.lib;Shell32.lib;Shlwapi.lib;%(AdditionalDependencies) false + 5.1 @@ -181,6 +192,7 @@ UseLinkTimeCodeGeneration false Shell32.lib;;Shlwapi.lib;%(AdditionalDependencies) + 5.2 diff --git a/TimedExec_VS2017.vcxproj.filters b/TimedExec_VS2017.vcxproj.filters index 3735910..3fae12b 100644 --- a/TimedExec_VS2017.vcxproj.filters +++ b/TimedExec_VS2017.vcxproj.filters @@ -19,4 +19,19 @@ Source Files + + + Header Files + + + + + Resource Files + + + + + Resource Files + + \ No newline at end of file diff --git a/res/appicon.ico b/res/appicon.ico new file mode 100644 index 0000000000000000000000000000000000000000..5676849e9d9e0c9fae783952b28eb5776f4f79c2 GIT binary patch literal 7886 zcmcgx30PD|wr=8l1~nPUWU@|PLX0scCNY_8&uAi=Xf%q^7~{T#8I3G4ib~uC7i3X3 zK@>zq5D}DB_9%_xG)?TXoKUo)LH2PqO%mm67&M zJ)aZmp8)@k41+L!81M`*3itpp0Y2*oZsT{{XH3HGNkm1?6k|K--;sg!#`Ag;nE35C zbDcL@uPfNKb357X*wzQ!#_#6N{l)=(zf7?Elsa=AsUsgU>@@kkG8llyqX6q|-0GG8 z4Blh6lU(;ZlkZ_~I(5>Y&YV8g2i(T*4(xZK1Geia!FD{Ua~!GZf;F}0xAdjw+fONvRMXr!*LPU4c5mL`Gg z+@@$!P-)pUk|&&@m~9`DBGZL#N35jVk*nxV)GAWt9iiA?*8}ghr%)z2?A=YreUDH~ zbR=bGWl&>7J+tXkE?&GyjQ-_zX%02Xh#?rk^PbxpPfXYuV z29{8H&>~;~HRc?q_}!CvPxj`|=ldD!AJ_OFba$ojbD@-)a*;YZw3M2fO3BH|;KzJA zC0(;jT<`s#+gLs>5jJmAUFH$GdSVWhoSfGU|G89hVm3A89@F#ZwbmE58>3vD9Vs;U zG{wb4)9qW;5-sMY^ef8uoh8-l^8zW(ZX(aU zH*c1~$b8kxZAT7yQdC43B_+m5bh^5_z*mPkcTs0&Cuz0XUqwe}m$U|vHgD10D*<%b za|-2nO#`M-?d9W?X!|kqev%pXD^FqFa{^DEkj`0UMLBgrzgn#ZXeSLr^M6AN{qPDf z|6g*0Nq%5FrMpa^+Pol2+W8^vuysfWU$$xe%hMjN2c&gBb2^Zk6bkCVTs3WN)II<- z?QNumJRKUy2k850G+N>nh_rc=?&gQl#XV!Gt}u)ew!cSbBVx(j&7J4ZcSry7v?umm zOjHEdi8pTm@ul|?Do_wAUcy0wMW>Ta~Rw~)}O=t{AM zo|{TEden=wb<1W+US6v<)(aLOa^o1PD~YGb^`oVCzP`Sc5FbZA-d+#o@9pV9mr|4I z&_Q>JKcBlc%)LdWlF+7B(fv*(bc!L}r9VJz?LtpJ3_vwQ3 zEQ(k+QqskJ=!4I68qV|aqdpJg|I}gd&&$0`2XF=j&M~jMTGc`-rIOSwO6t@oDZ4;T zvX_Z+id$fecnyfWy}jwR*IuJY*nS<(Nn0EAputn4LtRT2mUe?Y)$i3fgL7{%W;? z>h3ks2aCI?zCnS#u7FMy)YPn?=4O^pNm|&+veFXzc>H*}T2$CGmgQ4Rw3sH-W**?r zwyuA9WZ$0M=278cbU7=N?Dy;@K4UGIdpqX--a5k4Wwx73nlKpHt;F} zZ!u7yeb^sOxSN@0N=gbXS+WH7LnF~(T1=B^gDKx}{R`V-zKdRfdF%#@xJUK%4*_Iqf*vw5GAJh}hi=}uPCwafkZ7^*g7du}Jot|9T_k@4 z4eUEkSh0MWJ?>snfPKj8wR+V`Vq47npLJe;ua553)k@C>$ie8{#(mT+Dyo9~JeFzX z=U&0NJ5me!{qG!>gXjIxUif$RyKHIa_HA-U`#qT_VFSy{N+~Bhn=Dp;54(I;qT7u5 z>DOK^mlH$(pWiX>2=v7~q_O2?W#sSYi~mb|-}TH_U!HKyF1m{ z$syU(!!3tpC@Z;2jhNGR*tmbJSWY1!XXt8ik+j!xbMuJ7`;OZ}LW0Q*eYkHU_5zO$ z01Zd`y(7VlecFM_V0V}Obfc>97fRYHs7U(R*2v`ICOQjo+oGrocZxLH@^% z?gNwld#yj;-v0;t!1riTDK)UwDOk%^FArBfFMsLj^vH2P-{pb+$1es2_`@Fu_}NqE zHh#x_(EnlQyEngq>Vx@yfGul>uhZIdR}5sU1{(WEwGHOEunuuoj0e$&?VCS@AAE?N zz(h)sy#!?s7l^yGSU12x2@cggNWl{@00#XQRD+IfcMQilfR}o(6K#}YGKi|dRSAs7 z3#2k+?*RHd#R}v!{>ZY~YLUU)uaDrIz%&1tH|hUHeddfAgFcxs!4Mew>4cAmP5yM; zu+Jy{_t2>mKQerO?C3||cyajWi%ed2H~RCiSHGYC+j&@7Sq?Hbe(7;TL&Lv8zPABR z?{I9y;5xS%FETN9z8PlGwQ}0X;J*wX_UhJE(|>heI_$GIuwJ)z$l$?)U;nQcURXMJ z&g>(0JGS4!`6lNcaGl$}o-@a1&Fr_5>Jn_I{^w1!dEwt%}JU_^Z*-SGO!5_b)tbHf|U)a^x#x7k)c04Dm=8?n9BCnJ(_${e@auRH7KR77$!V zo5;$zLz%Lc-r-2V#OC91wo!~ly`IIFd5*1O~L}|c0 zDh*tK=eMHj+)64uy+*8=_x>~{zF_{hgN9(AxHvm0&V~ewg8W>;_dQ}mp-?D9Pi)xx z$#v1CyD!l%kG2&BM@^}~Z?-7#H5FB{c9iJ4z(35zOtP`lzWBoMo3H1rh3%%qgjmV9 zwp zD)X>KGsg|>*1Sw+YC3b+8Y>HV__+|t=aAwj`0)DpiDL<^2Ce-`Mk{lQ8e9T^T^KU^UHBHG#zpI{%t?|`f- zO25)37H`qfw5cW{Fd#sv)oRfO9hD@(rwZLFiX!aUHxp5j5n_Y&TKS@{OdilTojG&p zx*va#v+fh)7YOXA5+A|#R-o6YAf@_)i^Jq z?bK1aTrLvgVkG(tXMfg<{tQFpH!dY#NTQ^Kc!9V^D3#5Szey;Yn}p>a^B(4j+~I_%|{hFDQ}AMzCJ$7ZC<#hsd4qM_UO7CrtqKSe!w zu*?Yw38Jj5R2)9!#c`%M;pcnN!^P==>|cL9XSk8k@F_1F8(VI$UUwEbJxz9I2Eo@O zoMXw4*89>m&{si3?0n3@6@7WknV`UH=R!}9)yvM$Cyb2V+ibHj2e$q0o?Uin z9&Rqh=c6LDnd#|t;+UT}6>w4%78X!l?Ojn+RD?LGNN~NdAfMmSv7^2s1N#Q{y2j0A zziDX5nLa+0&4%?;kqhiP=;4;f`KZG3Yo+Qs#2y@fFu0Bwp(Xx&bYW<4kO$5TzI;5qz^^qp2d1VCVCj+wX1W&Gi9}^w*%EPJledFQoYH;%SF>B;ebg-kY`|S7j z(PQ7`xxppMi$5%`i4L=Gbf$z~31dT>o2!=UnAm z#zrgGF8yB{hV}==+DzP-Of(r9FZ?-YHE_ebxo^^@{WH;mBCA>^2}1I2wO{U8ykJg!o#una7X+c--RPQvs6gl}Zp6+wE=bq(#mu zCtwW~oL)oW{vO0*x<|`oGp0|I#&hi5s#bT$<;^NuvsXuRzn%sC-K2KhNvZDh=w4+u zg@D&A_(b~gGryefz~3}DucyNOqi$&;>wOx0zfjUGK_ zWAx*H{K3jJKIS~-Udh4TrJw`;t)RoX8NOS(^8;>fuEjpxXst+2+31Z$oI&hiF(;B%~$UY2|iur + + + + + + + + + + + + + + + + diff --git a/res/version.rc b/res/version.rc new file mode 100644 index 0000000..69ade53 --- /dev/null +++ b/res/version.rc @@ -0,0 +1,68 @@ +///////////////////////////////////////////////////////////////////////////// +// +// Microsoft Visual C++ generated resource script. +// +#define APSTUDIO_READONLY_SYMBOLS +#include "WinResrc.h" +#undef APSTUDIO_READONLY_SYMBOLS + +#include "../src/Version.h" + +#define ___FULL_VERSION_STR___(X) #X +#define __FULL_VERSION_STR__(X,Y,Z) ___FULL_VERSION_STR___(X.Y##Z) +#define _FULL_VERSION_STR_(X,Y,Z) __FULL_VERSION_STR__(X,Y,Z) +#define FULL_VERSION_STR _FULL_VERSION_STR_(VERSION_MAJOR,VERSION_MINOR_HI,VERSION_MINOR_LO) + +///////////////////////////////////////////////////////////////////////////// +// +// Neutral resources +// +#ifdef _WIN32 +LANGUAGE LANG_NEUTRAL, SUBLANG_NEUTRAL +#pragma code_page(1252) +#endif //_WIN32 + +///////////////////////////////////////////////////////////////////////////// +// +// Icon +// +101 ICON "appicon.ico" + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// +VS_VERSION_INFO VERSIONINFO + FILEVERSION VERSION_MAJOR, VERSION_MINOR_HI, VERSION_MINOR_LO, 0 + PRODUCTVERSION VERSION_MAJOR, VERSION_MINOR_HI, VERSION_MINOR_LO, 0 + FILEFLAGSMASK 0x17L +#ifdef _DEBUG + FILEFLAGS 0x3L +#else + FILEFLAGS 0x2L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "000004b0" + BEGIN + VALUE "ProductName", "TimedExec" + VALUE "FileDescription", "TimedExec" + VALUE "ProductVersion", FULL_VERSION_STR + VALUE "FileVersion", FULL_VERSION_STR + VALUE "InternalName", "TimedExec" + VALUE "OriginalFilename", "TimedExec.exe" + VALUE "LegalCopyright", "Created by LoRd_MuldeR " + VALUE "CompanyName", "Muldersoft" + VALUE "LegalTrademarks", "Muldersoft" + VALUE "Comments", "Released under the GNU General Public License" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x0, 1200 + END +END diff --git a/src/TimedExec.cpp b/src/TimedExec.cpp index 6df4c4c..92cabd9 100644 --- a/src/TimedExec.cpp +++ b/src/TimedExec.cpp @@ -1,6 +1,6 @@ ////////////////////////////////////////////////////////////////////////////////// // Timed Exec - Command-Line Benchmarking Utility -// Copyright (c) 2018 LoRd_MuldeR . Some rights reserved. +// Copyright (c) 2023 LoRd_MuldeR . Some rights reserved. // // This program is free software; you can redistribute it and/or // modify it under the terms of the GNU General Public License @@ -19,6 +19,8 @@ // http://www.gnu.org/licenses/gpl-2.0.txt ////////////////////////////////////////////////////////////////////////////////// +#include "Version.h" + #include #include #include @@ -34,9 +36,6 @@ #include #include -#define VERSION_MAJOR 1 -#define VERSION_MINOR 4 - #define DEFAULT_EXEC_LOOPS 5 #define DEFAULT_WARMUP_LOOPS 1 #define DEFAULT_LOGFILE "TimedExec.log" @@ -51,10 +50,44 @@ static HANDLE g_hAbortEvent = NULL; static volatile bool g_aborted = false; +/* clock for time measurement */ +typedef enum +{ + CLOCK_WALLCLOCK, + CLOCK_CPU_TOTAL, + CLOCK_CPU_USER, + CLOCK_CPU_KERNEL +} +clock_type_t; + // ============================================================================================================= // INTERNAL FUNCTIONS // ============================================================================================================= +#define _PRINT_CLOCK_TYPE(X) case CLOCK_##X: return #X; +#define _PARSE_CLOCK_TYPE(X) if (!_tcsicmp(name.c_str(), _T(#X))) { clock_type = (CLOCK_##X); return true; } + +static bool parseClockType(const tstring &name, clock_type_t &clock_type) +{ + _PARSE_CLOCK_TYPE(WALLCLOCK) + _PARSE_CLOCK_TYPE(CPU_TOTAL) + _PARSE_CLOCK_TYPE(CPU_USER) + _PARSE_CLOCK_TYPE(CPU_KERNEL) + return false; +} + +static const char *printClockType(const clock_type_t clock_type) +{ + switch (clock_type) + { + _PRINT_CLOCK_TYPE(WALLCLOCK) + _PRINT_CLOCK_TYPE(CPU_TOTAL) + _PRINT_CLOCK_TYPE(CPU_USER) + _PRINT_CLOCK_TYPE(CPU_KERNEL) + } + return "N/A"; +} + static bool getEnvVariable(const _TCHAR *const name, tstring &value) { std::vector buffer; @@ -81,32 +114,48 @@ static bool getEnvVariable(const _TCHAR *const name, tstring &value) return false; } -static LONGLONG getCurrentTime(void) +static ULONGLONG fileTimeToU64(const PFILETIME fileTime) { - LARGE_INTEGER timeValue; - if(!QueryPerformanceCounter(&timeValue)) - { - std::cerr << "\n\nSYSTEM ERROR: Failed to query performance counter!\n" << std::endl; - _exit(EXIT_FAILURE); - } - return timeValue.QuadPart; + ULARGE_INTEGER temp; + temp.HighPart = fileTime->dwHighDateTime; + temp.LowPart = fileTime->dwLowDateTime; + return temp.QuadPart; } -static LONGLONG getTimerFrequency(void) +static ULONGLONG getTimeElapsed(const ULONGLONG timeStart, const ULONGLONG timeExit) { - LARGE_INTEGER timeValue; - if(!QueryPerformanceFrequency(&timeValue)) + return (timeExit > timeStart) ? (timeExit - timeStart) : 0ULL; +} + +static double getProcessTime(const HANDLE hProc, const clock_type_t clock_type) +{ + FILETIME timeStart, timeExit, timeKernel, timeUser; + ULONGLONG result = 0ULL; + if (GetProcessTimes(hProc, &timeStart, &timeExit, &timeKernel, &timeUser)) { - std::cerr << "\n\nSYSTEM ERROR: Failed to query performance counter frequency!\n" << std::endl; - _exit(EXIT_FAILURE); + switch (clock_type) + { + case CLOCK_WALLCLOCK: + result = getTimeElapsed(fileTimeToU64(&timeStart), fileTimeToU64(&timeExit)); + break; + case CLOCK_CPU_USER: + result = fileTimeToU64(&timeUser); + break; + case CLOCK_CPU_KERNEL: + result = fileTimeToU64(&timeKernel); + break; + case CLOCK_CPU_TOTAL: + result = fileTimeToU64(&timeKernel) + fileTimeToU64(&timeUser); + break; + } } - return timeValue.QuadPart; + return static_cast(result) / 10000000.0; } static long long getCurrentFileSize(FILE *const filePtr) { struct _stati64 stats; - if(_fstati64(_fileno(filePtr), &stats) == 0) + if (_fstati64(_fileno(filePtr), &stats) == 0) { return stats.st_size; } @@ -183,7 +232,7 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile) int nArgs = 0; TCHAR **szArglist = CommandLineToArgvW(GetCommandLineW(), &nArgs); - if((szArglist == NULL) || (nArgs < 2)) + if ((szArglist == NULL) || (nArgs < 2)) { return 0; } @@ -192,7 +241,7 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile) { tstring token; - if(i > 1) + if (i > 1) { token = tstring(szArglist[i]); commandLine += _T(' '); @@ -203,7 +252,7 @@ static int initializeCommandLine(tstring &commandLine, tstring &programFile) programFile += token; } - if(token.find(' ') == tstring::npos) + if (token.find(' ') == tstring::npos) { commandLine += token; } @@ -226,7 +275,7 @@ static bool createProcess(const tstring &commandLine, HANDLE &hThrd, HANDLE &hPr PROCESS_INFORMATION processInfo; SecureZeroMemory(&processInfo, sizeof(PROCESS_INFORMATION)); - if(!CreateProcess(NULL, (LPTSTR)commandLine.c_str(), NULL, NULL, false, HIGH_PRIORITY_CLASS | (suspended ? CREATE_SUSPENDED : 0), NULL, NULL, &startInfo, &processInfo)) + if (!CreateProcess(NULL, (LPTSTR)commandLine.c_str(), NULL, NULL, false, HIGH_PRIORITY_CLASS | (suspended ? CREATE_SUSPENDED : 0), NULL, NULL, &startInfo, &processInfo)) { return false; } @@ -241,11 +290,11 @@ static bool waitForProcess(const HANDLE &hProc) { HANDLE waitHandles[2] = {hProc, g_hAbortEvent}; const DWORD ret = WaitForMultipleObjects(2, &waitHandles[0], FALSE, INFINITE); - if((ret != WAIT_OBJECT_0) && (ret != WAIT_OBJECT_0 + 1)) + if ((ret != WAIT_OBJECT_0) && (ret != WAIT_OBJECT_0 + 1)) { return false; } - if(ret > WAIT_OBJECT_0) + if (ret > WAIT_OBJECT_0) { g_aborted = true; } @@ -255,7 +304,7 @@ static bool waitForProcess(const HANDLE &hProc) static int getProcessExitCode(const HANDLE &hProc) { DWORD exitCode; - if(GetExitCodeProcess(hProc, &exitCode)) + if (GetExitCodeProcess(hProc, &exitCode)) { return *reinterpret_cast(&exitCode); } @@ -308,8 +357,8 @@ static int timedExecMain(int argc, _TCHAR* argv[]) initFmt.copyfmt(std::cerr); std::cerr << "\n===============================================================================" << std::endl; - std::cerr << "Timed Exec - Benchmarking Utility, Version " << VERSION_MAJOR << '.' << std::setfill('0') << std::setw(2) << VERSION_MINOR << " [" __DATE__ "]" << std::endl; - std::cerr << "Copyright (c) 2018 LoRd_MuldeR . Some rights reserved.\n" << std::endl; + std::cerr << "Timed Exec - Benchmarking Utility, Version " << VERSION_MAJOR << '.' << std::setfill('0') << std::setw(2) << (10 * VERSION_MINOR_HI) + VERSION_MINOR_LO << " [" __DATE__ "]" << std::endl; + std::cerr << "Copyright (c) 2023 LoRd_MuldeR . Some rights reserved.\n" << std::endl; std::cerr << "This program is free software: you can redistribute it and/or modify" << std::endl; std::cerr << "it under the terms of the GNU General Public License ." << std::endl; std::cerr << "Note that this program is distributed with ABSOLUTELY NO WARRANTY." << std::endl; @@ -322,7 +371,7 @@ static int timedExecMain(int argc, _TCHAR* argv[]) /* ---------------------------------------------------------- */ tstring commandLine, programFile; - if(initializeCommandLine(commandLine, programFile) < 2) + if (initializeCommandLine(commandLine, programFile) < 2) { std::cerr << "Usage:" << std::endl; std::cerr << " TimedExec.exe [Arguments]\n" << std::endl; @@ -330,17 +379,18 @@ static int timedExecMain(int argc, _TCHAR* argv[]) std::cerr << " TIMED_EXEC_PASSES - Number of execution passes (default: " << DEFAULT_EXEC_LOOPS << ")" << std::endl; std::cerr << " TIMED_EXEC_WARMUP_PASSES - Number of warm-up passes (default: " << DEFAULT_WARMUP_LOOPS << ")" << std::endl; std::cerr << " TIMED_EXEC_LOGFILE - Log-File Name (default: \"" << DEFAULT_LOGFILE << "\")" << std::endl; - std::cerr << " TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks\n" << std::endl; + std::cerr << " TIMED_EXEC_NO_CHECKS - Set this to *disable* exit code checks" << std::endl; + std::cerr << " TIMED_EXEC_CLOCK_TYPE - The type of clock used for measurements\n" << std::endl; return EXIT_FAILURE; } - if(_taccess(programFile.c_str(), 0) != 0) + if (_taccess(programFile.c_str(), 0) != 0) { _ftprintf(stderr, _T("Specified program file could not be found or access denied:\n%s\n\n"), programFile.c_str()); return EXIT_FAILURE; } - if(!checkBinary(programFile)) + if (!checkBinary(programFile)) { _ftprintf(stderr, _T("Specified file does not look like a valid Win32 executable:\n%s\n\n"), programFile.c_str()); return EXIT_FAILURE; @@ -353,26 +403,35 @@ static int timedExecMain(int argc, _TCHAR* argv[]) tstring logFile(getFullPath(_T(DEFAULT_LOGFILE))); int maxPasses = DEFAULT_EXEC_LOOPS, maxWarmUpPasses = DEFAULT_WARMUP_LOOPS; bool checkExitCodes = true; - - if(ENABLE_ENV_VARS) + clock_type_t clock_type = CLOCK_WALLCLOCK; + + if (ENABLE_ENV_VARS) { tstring temp; - if(getEnvVariable(_T("TIMED_EXEC_PASSES"), temp)) + if (getEnvVariable(_T("TIMED_EXEC_PASSES"), temp)) { maxPasses = std::min(SHRT_MAX, std::max(3, _tstoi(temp.c_str()))); } - if(getEnvVariable(_T("TIMED_EXEC_WARMUP_PASSES"), temp)) + if (getEnvVariable(_T("TIMED_EXEC_WARMUP_PASSES"), temp)) { maxWarmUpPasses = std::min(SHRT_MAX, std::max(0, _tstoi(temp.c_str()))); } - if(getEnvVariable(_T("TIMED_EXEC_LOGFILE"), temp)) + if (getEnvVariable(_T("TIMED_EXEC_LOGFILE"), temp)) { logFile = getFullPath(temp.c_str()); } - if(getEnvVariable(_T("TIMED_EXEC_NO_CHECKS"), temp)) + if (getEnvVariable(_T("TIMED_EXEC_NO_CHECKS"), temp)) { checkExitCodes = (_tstoi(temp.c_str()) == 0); } + if (getEnvVariable(_T("TIMED_EXEC_CLOCK_TYPE"), temp)) + { + if (!parseClockType(temp, clock_type)) + { + _ftprintf(stderr, _T("Specified clock type \"%s\" is unsupported.\nPlease see the documentation for a list of supported clock types!\n\n"), temp.c_str()); + return EXIT_FAILURE; + } + } } /* ---------------------------------------------------------- */ @@ -383,13 +442,9 @@ static int timedExecMain(int argc, _TCHAR* argv[]) _ftprintf(stderr, _T("Log File:\n%s\n\n"), logFile.c_str()); std::cerr << "Warm-Up / Metering passes: " << maxWarmUpPasses << "x / " << maxPasses << 'x' << std::endl; - const LONGLONG timerFrequency = getTimerFrequency(); - if(!SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS)) + if (!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS)) { - if(!SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS)) - { - std::cerr << "\nWARNING: Failed to set process priroity class!" << std::endl; - } + std::cerr << "\nWARNING: Failed to set process priroity class!" << std::endl; } std::vector stats_samples(maxPasses, 0.0); @@ -403,7 +458,7 @@ static int timedExecMain(int argc, _TCHAR* argv[]) /* Run Warm-Up Passes */ /* ---------------------------------------------------------- */ - for(int pass = 0; pass < maxWarmUpPasses; pass++) + for (int pass = 0; pass < maxWarmUpPasses; pass++) { std::cerr << "\n===============================================================================" << std::endl; if (maxWarmUpPasses > 1) std::cerr << "WARM-UP PASS " << (pass + 1) << " OF " << maxWarmUpPasses << std::endl; else std::cerr << "WARM-UP PASS" << std::endl; @@ -411,26 +466,26 @@ static int timedExecMain(int argc, _TCHAR* argv[]) HANDLE hThrd, hProc; - if(!createProcess(commandLine, hThrd, hProc)) + if (!createProcess(commandLine, hThrd, hProc)) { std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl; return EXIT_FAILURE; } - if(!waitForProcess(hProc)) + if (!waitForProcess(hProc)) { std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl; return EXIT_FAILURE; } - if(g_aborted) + if (g_aborted) { abortedHandlerRoutine(hProc); return EXIT_FAILURE; } const int exitCode = getProcessExitCode(hProc); - if(checkExitCodes && (exitCode != 0)) + if (checkExitCodes && (exitCode != 0)) { std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl; return EXIT_FAILURE; @@ -456,38 +511,33 @@ static int timedExecMain(int argc, _TCHAR* argv[]) HANDLE hThrd, hProc; - if(!createProcess(commandLine, hThrd, hProc, true)) + if (!createProcess(commandLine, hThrd, hProc, true)) { std::cerr << "\n\nSYSTEM ERROR: Failed to create process!\n" << std::endl; return EXIT_FAILURE; } - const LONGLONG timeStart = getCurrentTime(); - - if(ResumeThread(hThrd) == ((DWORD) -1)) + if (ResumeThread(hThrd) == ((DWORD) -1)) { std::cerr << "\n\nSYSTEM ERROR: Failed to resume child process!\n" << std::endl; TerminateProcess(hProc, UINT(-1)); return EXIT_FAILURE; } - if(!waitForProcess(hProc)) + if (!waitForProcess(hProc)) { std::cerr << "\n\nSYSTEM ERROR: Failed to wait for process termination!\n" << std::endl; return EXIT_FAILURE; } - const LONGLONG timeFinish = getCurrentTime(); - const double execTime = static_cast(timeFinish - timeStart) / static_cast(timerFrequency); - - if(g_aborted) + if (g_aborted) { abortedHandlerRoutine(hProc); return EXIT_FAILURE; } const int exitCode = getProcessExitCode(hProc); - if(checkExitCodes && (exitCode != 0)) + if (checkExitCodes && (exitCode != 0)) { std::cerr << "\n\nPROGRAM ERROR: Abnormal program termination detected! (Exit Code: " << exitCode << ")\n" << std::endl; return EXIT_FAILURE; @@ -497,6 +547,8 @@ static int timedExecMain(int argc, _TCHAR* argv[]) std::cerr << "\n>> Process terminated with exit code " << exitCode << '.' << std::endl; } + const double execTime = getProcessTime(hProc, clock_type); + std::cerr << std::setprecision(3) << std::fixed; std::cerr << ">> Execution took " << execTime << " seconds." << std::endl; std::cerr.copyfmt(initFmt); @@ -508,8 +560,8 @@ static int timedExecMain(int argc, _TCHAR* argv[]) stats_samples[pass] = execTime; // Update slowest/fastest - if(execTime > stats_slowest) stats_slowest = execTime; - if(execTime < stats_fastest) stats_fastest = execTime; + if (execTime > stats_slowest) stats_slowest = execTime; + if (execTime < stats_fastest) stats_fastest = execTime; // Iterative "online" computation of the mean and the variance // See http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm for details! @@ -548,6 +600,7 @@ static int timedExecMain(int argc, _TCHAR* argv[]) std::cerr << "Standard Deviation : " << standardDeviation << " seconds" << std::endl; std::cerr << "Standard Error : " << standardError << " seconds" << std::endl; std::cerr << "Fastest / Slowest Pass : " << stats_fastest << " / " << stats_slowest << " seconds" << std::endl; + std::cerr << "Active Clock Type : " << printClockType(clock_type) << " (" << clock_type << ')' << std::endl; std::cerr << "===============================================================================\n" << std::endl; std::cerr.copyfmt(initFmt); @@ -556,14 +609,14 @@ static int timedExecMain(int argc, _TCHAR* argv[]) /* ---------------------------------------------------------- */ FILE *fLog = NULL; - if(_tfopen_s(&fLog, logFile.c_str(), _T("a+")) == 0) + if (_tfopen_s(&fLog, logFile.c_str(), _T("a+")) == 0) { - if(getCurrentFileSize(fLog) == 0) + if (getCurrentFileSize(fLog) == 0) { _ftprintf_s(fLog, _T("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n"), _T("Program"), _T("Passes"), _T("Mean Time"), _T("Median Time"), _T("90% Confidence Interval"), _T("95% Confidence Interval"), _T("99% Confidence Interval"), _T("Fastest Pass"), _T("Slowest Pass"), _T("Standard Deviation"), _T("Standard Error"), _T("Command Line")); } _ftprintf_s(fLog, _T("%s\t%d\t%f\t%f\t%f\t%f\t%f\t%f\t%f\t%f\t%f\t%s\n"), getFileNameOnly(programFile).c_str(), maxPasses, stats_mean, medianTime, confidenceInterval_90, confidenceInterval_95, confidenceInterval_99, stats_fastest, stats_slowest, standardDeviation, standardError, commandLine.c_str()); - if(ferror(fLog) == 0) + if (ferror(fLog) == 0) { _ftprintf(stderr, _T("Log file updated successfully.\n\n")); } @@ -634,7 +687,7 @@ int _tmain(int argc, _TCHAR* argv[]) _set_invalid_parameter_handler(invalidParameterHandler); #endif // _DEBUG - if(!(g_hAbortEvent = CreateEvent(NULL, TRUE, FALSE, NULL))) + if (!(g_hAbortEvent = CreateEvent(NULL, TRUE, FALSE, NULL))) { std::cerr << "\n\nSYSTEM ERROR: Event object could not be created!\n" << std::endl; return EXIT_FAILURE; diff --git a/src/Version.h b/src/Version.h new file mode 100644 index 0000000..c40dfcd --- /dev/null +++ b/src/Version.h @@ -0,0 +1,29 @@ +////////////////////////////////////////////////////////////////////////////////// +// Timed Exec - Command-Line Benchmarking Utility +// Copyright (c) 2023 LoRd_MuldeR . Some rights reserved. +// +// This program is free software; you can redistribute it and/or +// modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation; either version 2 +// of the License, or (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +// +// http://www.gnu.org/licenses/gpl-2.0.txt +////////////////////////////////////////////////////////////////////////////////// + +#ifndef _INC_TIMED_EXEC_VERSION_H +#define _INC_TIMED_EXEC_VERSION_H + +#define VERSION_MAJOR 1 +#define VERSION_MINOR_HI 0 +#define VERSION_MINOR_LO 5 + +#endif diff --git a/z_build.bat b/z_build.bat index 12c5268..7cb25bc 100644 --- a/z_build.bat +++ b/z_build.bat @@ -108,13 +108,12 @@ echo BEGIN PACKAGING [Release] echo --------------------------------------------------------------------- mkdir "%PACK_PATH%" -mkdir "%PACK_PATH%\x64" -copy "%~dp0\bin\Win32\Release\*.exe" "%PACK_PATH%" -copy "%~dp0\bin\x64\.\Release\*.exe" "%PACK_PATH%\x64" -copy "%~dp0\LICENSE.html" "%PACK_PATH%" +copy "%~dp0\bin\Win32\Release\TimedExec.exe" "%PACK_PATH%\TimedExec-x86.exe" +copy "%~dp0\bin\x64\.\Release\TimedExec.exe" "%PACK_PATH%\TimedExec-x64.exe" +copy "%~dp0\LICENSE.html" "%PACK_PATH%" -"%PDOC_PATH%\pandoc.exe" --from markdown_github+pandoc_title_block+header_attributes+implicit_figures+inline_notes --to html5 --toc -N --standalone -H "%~dp0\..\Prerequisites\Pandoc\css\github-pandoc.inc" "README.md" | "%JAVA_HOME%\bin\java.exe" -jar "%~dp0\..\Prerequisites\HTMLCompressor\bin\htmlcompressor-1.5.3.jar" --compress-css -o "%PACK_PATH%\README.html" +"%PDOC_PATH%\pandoc.exe" --from markdown_github+pandoc_title_block+header_attributes+implicit_figures+inline_notes --to html5 --toc -N --metadata title="TimedExec" --standalone -H "%~dp0\..\Prerequisites\Pandoc\css\github-pandoc.inc" "README.md" | "%JAVA_HOME%\bin\java.exe" -jar "%~dp0\..\Prerequisites\HTMLCompressor\bin\htmlcompressor-1.5.3.jar" --compress-css -o "%PACK_PATH%\README.html" mkdir "%PACK_PATH%\img" mkdir "%PACK_PATH%\img\timedexec" @@ -125,13 +124,12 @@ REM /////////////////////////////////////////////////////////////////////////// REM // Compress REM /////////////////////////////////////////////////////////////////////////// "%UPX3_PATH%\upx.exe" --best "%PACK_PATH%\*.exe" -"%UPX3_PATH%\upx.exe" --best "%PACK_PATH%\x64\*.exe" REM /////////////////////////////////////////////////////////////////////////// REM // Create version tag REM /////////////////////////////////////////////////////////////////////////// echo TimedExec> "%PACK_PATH%\BUILD_TAG" -echo Copyright (C) 2018 LoRd_MuldeR ^>> "%PACK_PATH%\BUILD_TAG" +echo Copyright (C) 2023 LoRd_MuldeR ^>> "%PACK_PATH%\BUILD_TAG" echo.>> "%PACK_PATH%\BUILD_TAG" echo Built on %ISO_DATE%, at %ISO_TIME%>> "%PACK_PATH%\BUILD_TAG" echo.>> "%PACK_PATH%\BUILD_TAG"