Muldersoft/LameXP
Muldersoft/LameXP
1
0
Fork 0
Code Releases 2 Activity
87ebf07930
LameXP/src/Thread_FileAnalyzer_Task.cpp

909 lines
21 KiB
C++
Raw Normal View History

Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
///////////////////////////////////////////////////////////////////////////////
// LameXP - Audio Encoder Front-End
// Copyright (C) 2004-2012 LoRd_MuldeR <MuldeR2@GMX.de>
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License along
// with this program; if not, write to the Free Software Foundation, Inc.,
// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
//
// http://www.gnu.org/licenses/gpl-2.0.txt
///////////////////////////////////////////////////////////////////////////////
#include "Thread_FileAnalyzer_Task.h"
#include "Global.h"
#include "LockedFile.h"
#include "Model_AudioFile.h"
#include "PlaylistImporter.h"
#include <QDir>
#include <QFileInfo>
#include <QProcess>
#include <QDate>
#include <QTime>
#include <QDebug>
#include <QImage>
#include <QReadLocker>
#include <QWriteLocker>
#include <QThread>
#include <math.h>
#include <time.h>
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
#include <assert.h>
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
#define IS_KEY(KEY) (key.compare(KEY, Qt::CaseInsensitive) == 0)
#define IS_SEC(SEC) (key.startsWith((SEC "_"), Qt::CaseInsensitive))
#define FIRST_TOK(STR) (STR.split(" ", QString::SkipEmptyParts).first())
/* static vars */
Now using a QWaitCondition to synchronize the FileAnalyzer threads.
2012-05-05 03:55:27 +02:00
QMutex AnalyzeTask::s_waitMutex;
QWaitCondition AnalyzeTask::s_waitCond;
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
QSet<unsigned int> AnalyzeTask::s_threadIdx_running;
unsigned int AnalyzeTask::s_threadIdx_next = 0;
Now using a separate QSemaphore to limit the number of tasks in the queue of the QThreadPool. Instead of starting exactly as many task as there are threads in the pool and adding a new task only after a running one finished, we will now keep a fixed number of tasks in the queue (currently hardcoded to 32). Since there always will be more tasks left in the queue than there are threads in the pool, we ensure that no thread will ever become idle. Might be slightly faster than the "old" approach.
2012-05-25 02:24:44 +02:00
QSemaphore AnalyzeTask::s_semaphore(0);
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
/* more static vars */
QReadWriteLock AnalyzeTask::s_lock;
unsigned int AnalyzeTask::s_filesAccepted = 0;
unsigned int AnalyzeTask::s_filesRejected = 0;
unsigned int AnalyzeTask::s_filesDenied = 0;
unsigned int AnalyzeTask::s_filesDummyCDDA = 0;
unsigned int AnalyzeTask::s_filesCueSheet = 0;
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
QStringList AnalyzeTask::s_additionalFiles;
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
QSet<QString> AnalyzeTask::s_recentlyAdded;
/*constants*/
const int WAITCOND_TIMEOUT = 2500;
const int MAX_RETRIES = 60000 / WAITCOND_TIMEOUT;
Now using a separate QSemaphore to limit the number of tasks in the queue of the QThreadPool. Instead of starting exactly as many task as there are threads in the pool and adding a new task only after a running one finished, we will now keep a fixed number of tasks in the queue (currently hardcoded to 32). Since there always will be more tasks left in the queue than there are threads in the pool, we ensure that no thread will ever become idle. Might be slightly faster than the "old" approach.
2012-05-25 02:24:44 +02:00
const int MAX_QUEUE_SLOTS = 32;
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
////////////////////////////////////////////////////////////
// Constructor
////////////////////////////////////////////////////////////
AnalyzeTask::AnalyzeTask(const QString &inputFile, const QString &templateFile, volatile bool *abortFlag)
:
m_threadIdx(makeThreadIdx()),
m_inputFile(inputFile),
m_templateFile(templateFile),
m_mediaInfoBin(lamexp_lookup_tool("mediainfo.exe")),
m_avs2wavBin(lamexp_lookup_tool("avs2wav.exe")),
m_abortFlag(abortFlag)
{
if(m_mediaInfoBin.isEmpty() || m_avs2wavBin.isEmpty())
{
qFatal("Invalid path to MediaInfo binary. Tool not initialized properly.");
}
}
AnalyzeTask::~AnalyzeTask(void)
{
Now using a separate QSemaphore to limit the number of tasks in the queue of the QThreadPool. Instead of starting exactly as many task as there are threads in the pool and adding a new task only after a running one finished, we will now keep a fixed number of tasks in the queue (currently hardcoded to 32). Since there always will be more tasks left in the queue than there are threads in the pool, we ensure that no thread will ever become idle. Might be slightly faster than the "old" approach.
2012-05-25 02:24:44 +02:00
s_semaphore.release();
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.lock();
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
s_threadIdx_running.remove(m_threadIdx);
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.unlock();
Now using a QWaitCondition to synchronize the FileAnalyzer threads.
2012-05-05 03:55:27 +02:00
s_waitCond.wakeAll();
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
}
////////////////////////////////////////////////////////////
// Thread Main
////////////////////////////////////////////////////////////
Implemented a method to disables update signals from the FileList model. This will significantly speed-up adding a lot files, but comes with the drawback that updates won't be visible immediately.
2012-05-05 21:56:14 +02:00
void AnalyzeTask::run()
{
try
{
run_ex();
}
catch(...)
{
qWarning("WARNING: Caught an in exception AnalyzeTask thread!");
}
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.lock();
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
s_threadIdx_running.remove(m_threadIdx);
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.unlock();
Implemented a method to disables update signals from the FileList model. This will significantly speed-up adding a lot files, but comes with the drawback that updates won't be visible immediately.
2012-05-05 21:56:14 +02:00
s_waitCond.wakeAll();
}
void AnalyzeTask::run_ex(void)
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
{
int fileType = fileTypeNormal;
QString currentFile = QDir::fromNativeSeparators(m_inputFile);
qDebug("Analyzing: %s", currentFile.toUtf8().constData());
Implemented progress indicator for the working banner.
2012-05-06 04:57:00 +02:00
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
emit fileSelected(QFileInfo(currentFile).fileName());
Implemented progress indicator for the working banner.
2012-05-06 04:57:00 +02:00
emit progressValChanged(m_threadIdx + 1);
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
AudioFileModel file = analyzeFile(currentFile, &fileType);
Implemented progress indicator for the working banner.
2012-05-06 04:57:00 +02:00
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
if(*m_abortFlag)
{
qWarning("Operation cancelled by user!");
return;
}
if(fileType == fileTypeSkip)
{
qWarning("File was recently added, skipping!");
return;
}
if(fileType == fileTypeDenied)
{
QWriteLocker lock(&s_lock);
s_filesDenied++;
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
lock.unlock();
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
qWarning("Cannot access file for reading, skipping!");
return;
}
if(fileType == fileTypeCDDA)
{
QWriteLocker lock(&s_lock);
s_filesDummyCDDA++;
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
lock.unlock();
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
qWarning("Dummy CDDA file detected, skipping!");
return;
}
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
//Handle files with *incomplete* meida info
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
if(file.fileName().isEmpty() || file.formatContainerType().isEmpty() || file.formatAudioType().isEmpty())
{
QStringList fileList;
if(PlaylistImporter::importPlaylist(fileList, currentFile))
{
qDebug("Imported playlist file.");
Slightly improved progress indicator for file analyzer: Additional files from playlists are now taken into account earlier.
2012-05-08 01:12:26 +02:00
QWriteLocker lock(&s_lock);
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
s_additionalFiles << fileList;
}
else if(!QFileInfo(currentFile).suffix().compare("cue", Qt::CaseInsensitive))
{
QWriteLocker lock(&s_lock);
qWarning("Cue Sheet file detected, skipping!");
s_filesCueSheet++;
}
Added support for Opus Audio Codec, based on Opus-Tools v0.1.3 (2012-07-10) by Xiph.org/Mozilla
2012-07-20 23:19:08 +02:00
else if((!QFileInfo(currentFile).suffix().compare("opus", Qt::CaseInsensitive)) || (!QFileInfo(currentFile).suffix().compare("ogg", Qt::CaseInsensitive)))
{
qDebug("Found a potential Opus audio file, investigating...");
QFile opusTest(currentFile);
if(analyzeOpusFile(currentFile, file))
{
qDebug("Accepted Opus file: %s", file.filePath().toUtf8().constData());
QWriteLocker lock(&s_lock);
s_filesAccepted++;
s_recentlyAdded.insert(file.filePath().toLower());
lock.unlock();
waitForPreviousThreads();
emit fileAnalyzed(file);
}
else
{
QWriteLocker lock(&s_lock);
qDebug("Rejected Opus file: %s", file.filePath().toUtf8().constData());
s_filesRejected++;
}
}
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
else if(!QFileInfo(currentFile).suffix().compare("avs", Qt::CaseInsensitive))
{
qDebug("Found a potential Avisynth script, investigating...");
if(analyzeAvisynthFile(currentFile, file))
{
QWriteLocker lock(&s_lock);
s_filesAccepted++;
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
s_recentlyAdded.insert(file.filePath().toLower());
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
lock.unlock();
waitForPreviousThreads();
emit fileAnalyzed(file);
}
else
{
QWriteLocker lock(&s_lock);
qDebug("Rejected Avisynth file: %s", file.filePath().toUtf8().constData());
s_filesRejected++;
}
}
else
{
QWriteLocker lock(&s_lock);
qDebug("Rejected file of unknown type: %s", file.filePath().toUtf8().constData());
s_filesRejected++;
}
return;
}
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
//Emit the file now!
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
QWriteLocker lock(&s_lock);
s_filesAccepted++;
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
s_recentlyAdded.insert(file.filePath().toLower());
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
lock.unlock();
waitForPreviousThreads();
emit fileAnalyzed(file);
}
////////////////////////////////////////////////////////////
// Privtae Functions
////////////////////////////////////////////////////////////
const AudioFileModel AnalyzeTask::analyzeFile(const QString &filePath, int *type)
{
*type = fileTypeNormal;
AudioFileModel audioFile(filePath);
QReadLocker readLock(&s_lock);
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
if(s_recentlyAdded.contains(filePath.toLower()))
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
{
*type = fileTypeSkip;
return audioFile;
}
readLock.unlock();
QFile readTest(filePath);
if(!readTest.open(QIODevice::ReadOnly))
{
*type = fileTypeDenied;
return audioFile;
}
if(checkFile_CDDA(readTest))
{
*type = fileTypeCDDA;
return audioFile;
}
readTest.close();
bool skipNext = false;
unsigned int id_val[2] = {UINT_MAX, UINT_MAX};
cover_t coverType = coverNone;
QByteArray coverData;
QStringList params;
params << QString("--Inform=file://%1").arg(QDir::toNativeSeparators(m_templateFile));
params << QDir::toNativeSeparators(filePath);
QProcess process;
process.setProcessChannelMode(QProcess::MergedChannels);
process.setReadChannel(QProcess::StandardOutput);
process.start(m_mediaInfoBin, params);
if(!process.waitForStarted())
{
qWarning("MediaInfo process failed to create!");
qWarning("Error message: \"%s\"\n", process.errorString().toLatin1().constData());
process.kill();
process.waitForFinished(-1);
return audioFile;
}
while(process.state() != QProcess::NotRunning)
{
if(*m_abortFlag)
{
process.kill();
qWarning("Process was aborted on user request!");
break;
}
if(!process.waitForReadyRead())
{
if(process.state() == QProcess::Running)
{
qWarning("MediaInfo time out. Killing process and skipping file!");
process.kill();
process.waitForFinished(-1);
return audioFile;
}
}
QByteArray data;
while(process.canReadLine())
{
QString line = QString::fromUtf8(process.readLine().constData()).simplified();
if(!line.isEmpty())
{
//qDebug("Line:%s", line.toUtf8().constData());
int index = line.indexOf('=');
if(index > 0)
{
QString key = line.left(index).trimmed();
QString val = line.mid(index+1).trimmed();
if(!key.isEmpty())
{
updateInfo(audioFile, &skipNext, id_val, &coverType, &coverData, key, val);
}
}
}
}
}
if(audioFile.fileName().isEmpty())
{
QString baseName = QFileInfo(filePath).fileName();
int index = baseName.lastIndexOf(".");
if(index >= 0)
{
baseName = baseName.left(index);
}
baseName = baseName.replace("_", " ").simplified();
index = baseName.lastIndexOf(" - ");
if(index >= 0)
{
baseName = baseName.mid(index + 3).trimmed();
}
audioFile.setFileName(baseName);
}
process.waitForFinished();
if(process.state() != QProcess::NotRunning)
{
process.kill();
process.waitForFinished(-1);
}
if((coverType != coverNone) && (!coverData.isEmpty()))
{
retrieveCover(audioFile, coverType, coverData);
}
return audioFile;
}
void AnalyzeTask::updateInfo(AudioFileModel &audioFile, bool *skipNext, unsigned int *id_val, cover_t *coverType, QByteArray *coverData, const QString &key, const QString &value)
{
//qWarning("'%s' -> '%s'", key.toUtf8().constData(), value.toUtf8().constData());
/*New Stream*/
if(IS_KEY("Gen_ID") || IS_KEY("Aud_ID"))
{
if(value.isEmpty())
{
*skipNext = false;
}
else
{
//We ignore all ID's, except for the lowest one!
bool ok = false;
unsigned int id = value.toUInt(&ok);
if(ok)
{
if(IS_KEY("Gen_ID")) { id_val[0] = qMin(id_val[0], id); *skipNext = (id > id_val[0]); }
if(IS_KEY("Aud_ID")) { id_val[1] = qMin(id_val[1], id); *skipNext = (id > id_val[1]); }
}
else
{
*skipNext = true;
}
}
if(*skipNext)
{
qWarning("Skipping info for non-primary stream!");
}
return;
}
/*Skip or empty?*/
if((*skipNext) || value.isEmpty())
{
return;
}
/*Playlist file?*/
if(IS_KEY("Aud_Source"))
{
*skipNext = true;
audioFile.setFormatContainerType(QString());
audioFile.setFormatAudioType(QString());
qWarning("Skipping info for playlist file!");
return;
}
/*General Section*/
if(IS_SEC("Gen"))
{
if(IS_KEY("Gen_Format"))
{
audioFile.setFormatContainerType(value);
}
else if(IS_KEY("Gen_Format_Profile"))
{
audioFile.setFormatContainerProfile(value);
}
else if(IS_KEY("Gen_Title") || IS_KEY("Gen_Track"))
{
audioFile.setFileName(value);
}
else if(IS_KEY("Gen_Duration"))
{
unsigned int tmp = parseDuration(value);
if(tmp > 0) audioFile.setFileDuration(tmp);
}
else if(IS_KEY("Gen_Artist") || IS_KEY("Gen_Performer"))
{
audioFile.setFileArtist(value);
}
else if(IS_KEY("Gen_Album"))
{
audioFile.setFileAlbum(value);
}
else if(IS_KEY("Gen_Genre"))
{
audioFile.setFileGenre(value);
}
else if(IS_KEY("Gen_Released_Date") || IS_KEY("Gen_Recorded_Date"))
{
unsigned int tmp = parseYear(value);
if(tmp > 0) audioFile.setFileYear(tmp);
}
else if(IS_KEY("Gen_Comment"))
{
audioFile.setFileComment(value);
}
else if(IS_KEY("Gen_Track/Position"))
{
bool ok = false;
unsigned int tmp = value.toUInt(&ok);
if(ok) audioFile.setFilePosition(tmp);
}
else if(IS_KEY("Gen_Cover") || IS_KEY("Gen_Cover_Type"))
{
if(*coverType == coverNone)
{
*coverType = coverJpeg;
}
}
else if(IS_KEY("Gen_Cover_Mime"))
{
QString temp = FIRST_TOK(value);
if(!temp.compare("image/jpeg", Qt::CaseInsensitive)) *coverType = coverJpeg;
else if(!temp.compare("image/png", Qt::CaseInsensitive)) *coverType = coverPng;
else if(!temp.compare("image/gif", Qt::CaseInsensitive)) *coverType = coverGif;
}
else if(IS_KEY("Gen_Cover_Data"))
{
if(!coverData->isEmpty()) coverData->clear();
coverData->append(QByteArray::fromBase64(FIRST_TOK(value).toLatin1()));
}
else
{
qWarning("Unknown key '%s' with value '%s' found!", key.toUtf8().constData(), value.toUtf8().constData());
}
return;
}
/*Audio Section*/
if(IS_SEC("Aud"))
{
if(IS_KEY("Aud_Format"))
{
audioFile.setFormatAudioType(value);
}
else if(IS_KEY("Aud_Format_Profile"))
{
audioFile.setFormatAudioProfile(value);
}
else if(IS_KEY("Aud_Format_Version"))
{
audioFile.setFormatAudioVersion(value);
}
else if(IS_KEY("Aud_Channel(s)"))
{
bool ok = false;
unsigned int tmp = value.toUInt(&ok);
if(ok) audioFile.setFormatAudioChannels(tmp);
}
else if(IS_KEY("Aud_SamplingRate"))
{
bool ok = false;
unsigned int tmp = value.toUInt(&ok);
if(ok) audioFile.setFormatAudioSamplerate(tmp);
}
else if(IS_KEY("Aud_BitDepth"))
{
bool ok = false;
unsigned int tmp = value.toUInt(&ok);
if(ok) audioFile.setFormatAudioBitdepth(tmp);
}
else if(IS_KEY("Aud_Duration"))
{
unsigned int tmp = parseDuration(value);
if(tmp > 0) audioFile.setFileDuration(tmp);
}
else if(IS_KEY("Aud_BitRate"))
{
bool ok = false;
unsigned int tmp = value.toUInt(&ok);
if(ok) audioFile.setFormatAudioBitrate(tmp/1000);
}
else if(IS_KEY("Aud_BitRate_Mode"))
{
if(!value.compare("CBR", Qt::CaseInsensitive)) audioFile.setFormatAudioBitrateMode(AudioFileModel::BitrateModeConstant);
if(!value.compare("VBR", Qt::CaseInsensitive)) audioFile.setFormatAudioBitrateMode(AudioFileModel::BitrateModeVariable);
}
else
{
qWarning("Unknown key '%s' with value '%s' found!", key.toUtf8().constData(), value.toUtf8().constData());
}
return;
}
/*Section not recognized*/
qWarning("Unknown section: %s", key.toUtf8().constData());
}
bool AnalyzeTask::checkFile_CDDA(QFile &file)
{
file.reset();
QByteArray data = file.read(128);
int i = data.indexOf("RIFF");
int j = data.indexOf("CDDA");
int k = data.indexOf("fmt ");
return ((i >= 0) && (j >= 0) && (k >= 0) && (k > j) && (j > i));
}
void AnalyzeTask::retrieveCover(AudioFileModel &audioFile, cover_t coverType, const QByteArray &coverData)
{
qDebug("Retrieving cover!");
QString extension;
switch(coverType)
{
case coverPng:
extension = QString::fromLatin1("png");
break;
case coverGif:
extension = QString::fromLatin1("gif");
break;
default:
extension = QString::fromLatin1("jpg");
break;
}
if(!(QImage::fromData(coverData, extension.toUpper().toLatin1().constData()).isNull()))
{
QFile coverFile(QString("%1/%2.%3").arg(lamexp_temp_folder2(), lamexp_rand_str(), extension));
if(coverFile.open(QIODevice::WriteOnly))
{
coverFile.write(coverData);
coverFile.close();
audioFile.setFileCover(coverFile.fileName(), true);
}
}
else
{
qWarning("Image data seems to be invalid :-(");
}
}
Added support for Opus Audio Codec, based on Opus-Tools v0.1.3 (2012-07-10) by Xiph.org/Mozilla
2012-07-20 23:19:08 +02:00
bool AnalyzeTask::analyzeOpusFile(const QString &filePath, AudioFileModel &info)
{
QFile opusFile(filePath);
if(opusFile.open(QIODevice::ReadOnly))
{
QByteArray data = opusFile.peek(1024);
int idx1 = data.indexOf(QString::fromAscii("OggS"));
int idx2 = data.indexOf(QString::fromAscii("OpusHead"));
if((idx1 >= 0) && (idx2 > idx1))
{
info.setFormatContainerType("OGG");
info.setFormatAudioType("Opus");
opusFile.close();
return true;
}
opusFile.close();
return false;
}
return false;
}
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
bool AnalyzeTask::analyzeAvisynthFile(const QString &filePath, AudioFileModel &info)
{
QProcess process;
process.setProcessChannelMode(QProcess::MergedChannels);
process.setReadChannel(QProcess::StandardOutput);
process.start(m_avs2wavBin, QStringList() << QDir::toNativeSeparators(filePath) << "?");
if(!process.waitForStarted())
{
qWarning("AVS2WAV process failed to create!");
qWarning("Error message: \"%s\"\n", process.errorString().toLatin1().constData());
process.kill();
process.waitForFinished(-1);
return false;
}
bool bInfoHeaderFound = false;
while(process.state() != QProcess::NotRunning)
{
if(*m_abortFlag)
{
process.kill();
qWarning("Process was aborted on user request!");
break;
}
if(!process.waitForReadyRead())
{
if(process.state() == QProcess::Running)
{
qWarning("AVS2WAV time out. Killing process and skipping file!");
process.kill();
process.waitForFinished(-1);
return false;
}
}
QByteArray data;
while(process.canReadLine())
{
QString line = QString::fromUtf8(process.readLine().constData()).simplified();
if(!line.isEmpty())
{
int index = line.indexOf(':');
if(index > 0)
{
QString key = line.left(index).trimmed();
QString val = line.mid(index+1).trimmed();
if(bInfoHeaderFound && !key.isEmpty() && !val.isEmpty())
{
if(key.compare("TotalSeconds", Qt::CaseInsensitive) == 0)
{
bool ok = false;
unsigned int duration = val.toUInt(&ok);
if(ok) info.setFileDuration(duration);
}
if(key.compare("SamplesPerSec", Qt::CaseInsensitive) == 0)
{
bool ok = false;
unsigned int samplerate = val.toUInt(&ok);
if(ok) info.setFormatAudioSamplerate (samplerate);
}
if(key.compare("Channels", Qt::CaseInsensitive) == 0)
{
bool ok = false;
unsigned int channels = val.toUInt(&ok);
if(ok) info.setFormatAudioChannels(channels);
}
if(key.compare("BitsPerSample", Qt::CaseInsensitive) == 0)
{
bool ok = false;
unsigned int bitdepth = val.toUInt(&ok);
if(ok) info.setFormatAudioBitdepth(bitdepth);
}
}
}
else
{
if(line.contains("[Audio Info]", Qt::CaseInsensitive))
{
info.setFormatAudioType("Avisynth");
info.setFormatContainerType("Avisynth");
bInfoHeaderFound = true;
}
}
}
}
}
process.waitForFinished();
if(process.state() != QProcess::NotRunning)
{
process.kill();
process.waitForFinished(-1);
}
//Check exit code
switch(process.exitCode())
{
case 0:
qDebug("Avisynth script was analyzed successfully.");
return true;
break;
case -5:
qWarning("It appears that Avisynth is not installed on the system!");
return false;
break;
default:
qWarning("Failed to open the Avisynth script, bad AVS file?");
return false;
break;
}
}
unsigned int AnalyzeTask::parseYear(const QString &str)
{
if(str.startsWith("UTC", Qt::CaseInsensitive))
{
QDate date = QDate::fromString(str.mid(3).trimmed().left(10), "yyyy-MM-dd");
if(date.isValid())
{
return date.year();
}
else
{
return 0;
}
}
else
{
bool ok = false;
int year = str.toInt(&ok);
if(ok && year > 0)
{
return year;
}
else
{
return 0;
}
}
}
unsigned int AnalyzeTask::parseDuration(const QString &str)
{
bool ok = false;
unsigned int value = str.toUInt(&ok);
return ok ? (value/1000) : 0;
}
unsigned __int64 AnalyzeTask::makeThreadIdx(void)
{
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.lock();
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
unsigned int idx = s_threadIdx_next++;
s_threadIdx_running.insert(idx);
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.unlock();
return idx;
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
}
void AnalyzeTask::waitForPreviousThreads(void)
{
Some minor tweaks to the multi-threaded FileAnalyzer class.
2012-05-04 16:40:02 +02:00
//This function will block until all threads with a *lower* index have terminated.
//Required to make sure that the files will be added in the "correct" order!
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
s_waitMutex.lock();
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
int retryCount = 0;
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
forever
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
{
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
bool bWaitFlag = false;
QSet<unsigned int>::const_iterator i;
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
for(i = s_threadIdx_running.begin(); i != s_threadIdx_running.end(); ++i)
{
if(*i < m_threadIdx) { bWaitFlag = true; break; }
}
if((!bWaitFlag) || *m_abortFlag)
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
{
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.unlock();
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
return;
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
}
Now using a QWaitCondition to synchronize the FileAnalyzer threads.
2012-05-05 03:55:27 +02:00
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
if(!s_waitCond.wait(&s_waitMutex, WAITCOND_TIMEOUT))
Implemented a method to disables update signals from the FileList model. This will significantly speed-up adding a lot files, but comes with the drawback that updates won't be visible immediately.
2012-05-05 21:56:14 +02:00
{
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
if(++retryCount > MAX_RETRIES)
{
qWarning("AnalyzeTask::waitForPreviousThreads encountered timeout !!!");
s_threadIdx_running.clear();
}
Implemented a method to disables update signals from the FileList model. This will significantly speed-up adding a lot files, but comes with the drawback that updates won't be visible immediately.
2012-05-05 21:56:14 +02:00
}
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
}
}
////////////////////////////////////////////////////////////
// Public Functions
////////////////////////////////////////////////////////////
unsigned int AnalyzeTask::filesAccepted(void)
{
QReadLocker lock(&s_lock);
return s_filesAccepted;
}
unsigned int AnalyzeTask::filesRejected(void)
{
QReadLocker lock(&s_lock);
return s_filesRejected;
}
unsigned int AnalyzeTask::filesDenied(void)
{
QReadLocker lock(&s_lock);
return s_filesDenied;
}
unsigned int AnalyzeTask::filesDummyCDDA(void)
{
QReadLocker lock(&s_lock);
return s_filesDummyCDDA;
}
unsigned int AnalyzeTask::filesCueSheet(void)
{
QReadLocker lock(&s_lock);
return s_filesCueSheet;
}
Slightly improved progress indicator for file analyzer: Additional files from playlists are now taken into account earlier.
2012-05-08 01:12:26 +02:00
int AnalyzeTask::getAdditionalFiles(QStringList &fileList)
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
{
Slightly improved progress indicator for file analyzer: Additional files from playlists are now taken into account earlier.
2012-05-08 01:12:26 +02:00
QReadLocker readLock(&s_lock);
int count = s_additionalFiles.count();
readLock.unlock();
if(count > 0)
{
QWriteLocker lock(&s_lock);
count = s_additionalFiles.count();
fileList << s_additionalFiles;
s_additionalFiles.clear();
return count;
}
return 0;
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
}
Now using a separate QSemaphore to limit the number of tasks in the queue of the QThreadPool. Instead of starting exactly as many task as there are threads in the pool and adding a new task only after a running one finished, we will now keep a fixed number of tasks in the queue (currently hardcoded to 32). Since there always will be more tasks left in the queue than there are threads in the pool, we ensure that no thread will ever become idle. Might be slightly faster than the "old" approach.
2012-05-25 02:24:44 +02:00
bool AnalyzeTask::waitForFreeSlot(volatile bool *abortFlag)
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
{
bool ret = false;
Now using a separate QSemaphore to limit the number of tasks in the queue of the QThreadPool. Instead of starting exactly as many task as there are threads in the pool and adding a new task only after a running one finished, we will now keep a fixed number of tasks in the queue (currently hardcoded to 32). Since there always will be more tasks left in the queue than there are threads in the pool, we ensure that no thread will ever become idle. Might be slightly faster than the "old" approach.
2012-05-25 02:24:44 +02:00
for(int i = 0; i < MAX_RETRIES; i++)
{
ret = s_semaphore.tryAcquire(1, WAITCOND_TIMEOUT);
if(ret || (*abortFlag)) break;
}
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
return ret;
}
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
void AnalyzeTask::reset(void)
{
QWriteLocker lock(&s_lock);
s_filesAccepted = 0;
s_filesRejected = 0;
s_filesDenied = 0;
s_filesDummyCDDA = 0;
s_filesCueSheet = 0;
s_additionalFiles.clear();
s_recentlyAdded.clear();
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
lock.unlock();
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.lock();
Changed the method to synchronize the analyzer threads: We now use QSet to maintain a list of the thread id's of all threads that are still running - instead of only recording the highest thread id that has finished. This way a thread can now actually check if there still is any running thread with a lower thread id. Before thread n could only check if thread n-1 has finished yet or not. As a result we can relax the waiting now. Only threads that want to emit a file will wait for the "previous" threads to finish.
2012-05-14 00:50:16 +02:00
s_threadIdx_next = 0;
s_threadIdx_running.clear();
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
s_waitMutex.unlock();
Now using a separate QSemaphore to limit the number of tasks in the queue of the QThreadPool. Instead of starting exactly as many task as there are threads in the pool and adding a new task only after a running one finished, we will now keep a fixed number of tasks in the queue (currently hardcoded to 32). Since there always will be more tasks left in the queue than there are threads in the pool, we ensure that no thread will ever become idle. Might be slightly faster than the "old" approach.
2012-05-25 02:24:44 +02:00
int freeSlots = s_semaphore.available();
if(freeSlots < MAX_QUEUE_SLOTS)
{
s_semaphore.release(MAX_QUEUE_SLOTS - freeSlots);
}
Some code refactoring regarding the QWaitCondition/QMutex in FileAnalyzer_Task.
2012-05-09 02:54:41 +02:00
}
Fix: In order to ensure the "correct" ordering, *all* threads need to wait for the previous threads to finish - not only the threads that will emit a file. If both, thread n and n+1, want to emit a file and thus n+1 has to wait for n, we additionally must allow thread n+2 to terminate before n and n+1, even if n+2 does *not* emit a file. That's because as soon as n+2 finishes, it unblocks all threads x with x < n+2, which includes n and n+1. If that happens and n+1 is still waiting for n, we effectively allow n+1 to emit its file *before* n. By blocking all threads x with x > n+1 until n+1 has finished (regardless of whether they want to emit a file or not) this problem is resolved. As long as we assume that most threads actually *do* emit a file, this commit shouldn't hurt the performance.
2012-05-12 02:51:24 +02:00
Implemented multi-threading for file analyzer. Now multiple files can be analyzed in parallel. This should speed-up adding files quite a bit, especially when a huge number of files is added at once.
2012-05-04 04:01:10 +02:00
////////////////////////////////////////////////////////////
// EVENTS
////////////////////////////////////////////////////////////
/*NONE*/
Copy Permalink