Scrape the Web for Doom WADs and install them. More...

#include <wadseeker.h>

Inheritance diagram for Wadseeker:

Public Slots
void	abort ()
	Tell Wadseeker to stop the seek session.

Signals
void	allDone (bool bSuccess)
	Emitted when Wadseeker finishes iterating through all files passed to startSeek().

void	fileDownloadFinished (const ModFile &filename)
	Emitted when a particular file finishes downloading.

void	fileDownloadProgress (const ModFile &filename, qint64 done, qint64 total)
	Emits download progress.

void	fileDownloadStarted (const ModFile &filename, const QUrl &url)
	Emitted when a file download starts.

void	fileInstalled (const ModFile &filename)
	Emitted when a particular file is installed correctly.

void	message (const QString &msg, WadseekerLib::MessageType type)
	Emits Wadseeker's log messages.

void	seekStarted (const ModSet &filenames)
	Emitted when Wadseeker begins the seek operation, upon a successful startSeek().

void	serviceFinished (const QString &name)
	Notifies that Wadseeker is done querying a named service.

void	serviceStarted (const QString &name)
	Notifies that Wadseeker started querying a named service.

void	siteFinished (const QUrl &site)
	Emitted when a WWW site finishes downloading.

void	siteProgress (const QUrl &site, qint64 bytes, qint64 total)
	Emits download progress of a WWW site.

void	siteRedirect (const QUrl &oldUrl, const QUrl &newUrl)
	Emitted when a WWW site redirects to a different site.

void	siteStarted (const QUrl &site)
	Emitted when a download of a WWW site starts.

Public Member Functions
	Wadseeker ()
	Initializes a new instance of Wadseeker.

	~Wadseeker () override
	Deallocates an instance of Wadseeker.

bool	isDownloadingFile (const ModFile &file) const
	Check if Wadseeker is currently downloading given file.

bool	isWorking () const
	Check if Wadseeker is still working.

void	setCustomSite (const QString &url)
	Set a high-priority URL to scrape for WADs.

void	setCustomSites (const QStringList &urls)
	setCustomSite(), but more than one.

void	setIdgamesEnabled (bool bEnabled)
	Set the /idgames Archive enabled.

void	setIdgamesUrl (QString archiveURL)
	Set the URL to the /idgames Archive search page.

void	setMaximumConcurrentDownloads (unsigned max)
	Set maximum number of concurrent WAD downloads.

void	setMaximumConcurrentSeeks (unsigned max)
	Set maximum number of concurrent site seeks (scrapings).

void	setPrimarySites (const QStringList &urlList)
	Set a list of primary sites to scrape for WADs.

void	setPrimarySitesToDefault ()
	Reset the list of primary sites to the sites hardcoded into Wadseeker.

void	setTargetDirectory (const QString &dir)
	Set the WAD installation target directory.

void	setWadArchiveEnabled (bool enabled)
	Does nothing, because Wad Archive is defunct.

void	skipFileCurrentUrl (const QString &fileName)
	Abort the current URL for the specified file.

void	skipService (const QString &service)
	Abort querying a named service if such query is in progress.

void	skipSiteSeek (const QUrl &url)
	Abort scraping the site at the specified URL.

bool	startSeek (const ModSet &wads)
	Launch the WAD retrieval session.

QString	targetDirectory () const
	Target directory is a directory where all seeked files will be saved.

Static Public Member Functions
static const QString	defaultIdgamesUrl ()
	Default URL to the /idgames Archive.

static QStringList	defaultSitesListEncoded ()
	Percent-encoded Wadseeker::defaultSites.

static ModSet	filterAllowedOnlyWads (const ModSet &wads)
	List only WADs for which isForbiddenWad() returns false.

static ModSet	filterForbiddenOnlyWads (const ModSet &wads)
	List only WADs for which isForbiddenWad() returns true.

static bool	isForbiddenWad (const ModFile &wad)
	Check if the file is on the list of forbidden WADs.

Static Public Attributes
static const QString	defaultSites []
	Default sites to scrape for files.

static const QString	forbiddenWads []
	Files that are not meant to be freely downloadable.

Detailed Description

Scrape the Web for Doom WADs and install them.

The Wadseeker class provides an interface for searching for and downloading modifications for Doom (1993) engine games. Create its object, set it up with various setters methods, and then launch it with startSeek().

All setup must be done before startSeek() is called. Wadseeker will ignore any setup changes aftewards.

Wadseeker will search for mods in a list of locations provided to setPrimarySites() and setCustomSites(). It can also contact the /idgames Archive, which can be toggled with setIdgamesEnabled().

Wadseeker will ignore a predefined set of file names which it suspects to be attempts to illegally download commercial game data, so passing in any name for which isForbiddenWad() returns true will result in the file being ignored.

It's not advisable, even if technically allowed, to have more than one instance of this class running simultaneously. Some Web servers limit the amount of simultaneous connections per client, and Wadseeker organizes its scraping with that mind, but it can only do that within a single seek session. It can't synchronize between multiple instances of itself.

Definition at line 183 of file wadseeker.h.

Constructor & Destructor Documentation

Wadseeker::Wadseeker ( )

Initializes a new instance of Wadseeker.

Wadseeker::~Wadseeker ( )

override

Deallocates an instance of Wadseeker.

Member Function Documentation

void Wadseeker::abort ( )

slot

Tell Wadseeker to stop the seek session.

This is asynchronous. Wadseeker doesn't stop outright. It takes care to clean up after itself. Once everything is stopped, allDone() with false result is emitted.

void Wadseeker::allDone ( bool bSuccess )

signal

Emitted when Wadseeker finishes iterating through all files passed to startSeek().

Parameters

bSuccess True if all seeked WADs were installed. False if any single one of the WADs was not found or if abort was issued.

static const QString Wadseeker::defaultIdgamesUrl ( )

static

Default URL to the /idgames Archive.

static QStringList Wadseeker::defaultSitesListEncoded ( )

static

Percent-encoded Wadseeker::defaultSites.

Runs content of defaultSites array through QUrl::toPercentEncoding() and returns a list of so encoded strings.

void Wadseeker::fileDownloadFinished ( const ModFile & filename )

signal

Emitted when a particular file finishes downloading.

NOTE: This doesn't mean that the WAD was successfully installed. It only serves as a notification that a download has been completed.

Parameters

filename Unique filename for the affected file. Emitted previously by seekStarted() signal as an entry on the filenames list.

void Wadseeker::fileDownloadProgress	(	const ModFile &	filename,
		qint64	done,
		qint64	total
	)

signal

Emits download progress.

The progress values may be used to update a progress bar.

Parameters

filename	Unique filename for the affected file. Emitted previously by seekStarted() signal as an entry on the filenames list.
done	Bytes already downloaded.
total	Total size of the downloaded file.

void Wadseeker::fileDownloadStarted	(	const ModFile &	filename,
		const QUrl &	url
	)

signal

Emitted when a file download starts.

Parameters

filename	Name of the downloaded file.
url	URL from which the file is being downloaded.

void Wadseeker::fileInstalled ( const ModFile & filename )

signal

Emitted when a particular file is installed correctly.

Parameters

filename Unique filename for the affected file. Emitted previously by seekStarted() signal as an entry on the filenames list.

static ModSet Wadseeker::filterAllowedOnlyWads ( const ModSet & wads )

static

List only WADs for which isForbiddenWad() returns false.

static ModSet Wadseeker::filterForbiddenOnlyWads ( const ModSet & wads )

static

List only WADs for which isForbiddenWad() returns true.

bool Wadseeker::isDownloadingFile ( const ModFile & file ) const

Check if Wadseeker is currently downloading given file.

Parameters

file	Name of the file as specified by seekStarted() signal.

static bool Wadseeker::isForbiddenWad ( const ModFile & wad )

static

Check if the file is on the list of forbidden WADs.

Parameters

wad	WAD to check.

Returns: True if Wadseeker will refuse to download this WAD.

bool Wadseeker::isWorking ( ) const

Check if Wadseeker is still working.

Returns: True if any search or download is still in progress. False if all site searches are finished and there are no downloads in progress. False will also be returned if there are still WADs pending for download URLs but there are no more sites to scrape for those URLs.

void Wadseeker::message	(	const QString &	msg,
		WadseekerLib::MessageType	type
	)

signal

Emits Wadseeker's log messages.

Parameters

msg	Text of the message.
type	Log severity level.

void Wadseeker::seekStarted ( const ModSet & filenames )

signal

Emitted when Wadseeker begins the seek operation, upon a successful startSeek().

Parameters

filenames Contains unique names of all files that will be seeked. Wadseeker will continue to refer to those filenames in other signals.

void Wadseeker::serviceFinished ( const QString & name )

signal

Notifies that Wadseeker is done querying a named service.

Parameters

name	Unique name of the service.

See Also: serviceStarted()

void Wadseeker::serviceStarted ( const QString & name )

signal

Notifies that Wadseeker started querying a named service.

Currently there is only one such service: idgames.

Parameters

name	Unique name of the service. Can be used to abort querying of this service by passing it to skipService().

See Also: serviceFinished()

void Wadseeker::setCustomSite ( const QString & url )

Set a high-priority URL to scrape for WADs.

This site has priority over all other sites and will be searched first. A direct URL to the file can also be used.

This is a good place to pass in the link provided by the game server.

Parameters

url	A valid, absolute URL.

See Also: setCustomSites() for multiple sites

void Wadseeker::setCustomSites ( const QStringList & urls )

setCustomSite(), but more than one.

Because this is a "set", it overwrites any previous custom sites, including the one set by setCustomSite().

Parameters

urls	A list of valid, absolute URLs.

void Wadseeker::setIdgamesEnabled ( bool bEnabled )

Set the /idgames Archive enabled.

If true, /idgames Archive search will be performed. If false, the /idgames Archive will be skipped.

The /idgames Archive is enabled by default.

void Wadseeker::setIdgamesUrl ( QString archiveURL )

Set the URL to the /idgames Archive search page.

Wadseeker already hardcodes the URL to the /idgames Archive and uses it by default, so in normal operation there should be no need to use this method.

void Wadseeker::setMaximumConcurrentDownloads ( unsigned max )

Set maximum number of concurrent WAD downloads.

Default value: 3.

Parameters

max	Number of max. concurrent downloads. Cannot be lesser than 1.

void Wadseeker::setMaximumConcurrentSeeks ( unsigned max )

Set maximum number of concurrent site seeks (scrapings).

Default value: 3.

Parameters

max	Number of max. concurrent seeks. Cannot be less than 1.

void Wadseeker::setPrimarySites ( const QStringList & urlList )

Set a list of primary sites to scrape for WADs.

This is where Wadseeker begins its search.

Parameters

urlList List of valid, absolute URLs.

void Wadseeker::setPrimarySitesToDefault ( )

Reset the list of primary sites to the sites hardcoded into Wadseeker.

Calling this is normally not needed, because Wadseeker already initializes the primary sites to this default list. However, if you changed the list and want to go back to the default one, this is the way to do it.

See Also: defaultSites; setPrimarySites().

void Wadseeker::setTargetDirectory ( const QString & dir )

Set the WAD installation target directory.

The target directory is the directory where all the seeked files will be saved to. If this directory is not writable, the seek will fail.

Parameters

dir	Path to a writable directory.

void Wadseeker::setWadArchiveEnabled ( bool enabled )

Does nothing, because Wad Archive is defunct.

If set to true, Wadseeker would contact Wad Archive to look for WAD URLs, but the site is dead.

Wad Archive: http://www.wad-archive.com

Deprecated:: Wad Archive was shut down in October 2022. This doesn't do anything anymore.

void Wadseeker::siteFinished ( const QUrl & site )

signal

Emitted when a WWW site finishes downloading.

void Wadseeker::siteProgress	(	const QUrl &	site,
		qint64	bytes,
		qint64	total
	)

signal

Emits download progress of a WWW site.

void Wadseeker::siteRedirect	(	const QUrl &	oldUrl,
		const QUrl &	newUrl
	)

signal

Emitted when a WWW site redirects to a different site.

void Wadseeker::siteStarted ( const QUrl & site )

signal

Emitted when a download of a WWW site starts.

void Wadseeker::skipFileCurrentUrl ( const QString & fileName )

Abort the current URL for the specified file.

The current download will be aborted, but a new file URL will be immediately taken from the queue if available, and a new download will begin for this file.

This method gives the user the option to abort the current download for this file if the site is slow.

Parameters

fileName This must be a file name as previously sent by seekStarted() signal. If the file name is not valid or download for this file is not in progress, then no operation will be performed.

void Wadseeker::skipService ( const QString & service )

Abort querying a named service if such query is in progress.

The name of the service is previously emitted by serviceStarted() signal.

Parameters

service Name of the service as emitted by serviceStarted() signal.

void Wadseeker::skipSiteSeek ( const QUrl & url )

Abort scraping the site at the specified URL.

Wadseeker will immediately start scraping the next site in the queue.

Parameters

url	This must be one of the URLs that was emitted through siteStarted() or siteRedirect() signals. If an unknown URL is passed, no operation will be performed.

bool Wadseeker::startSeek ( const ModSet & wads )

Launch the WAD retrieval session.

Setup the seek first using other methods, then call this method to start it. This is where Wadseeker begins to iterate through WWW sites to find all the desired files.

One of the three things may happen after here:

If seek starts properly, seekStarted() signal is emitted and the method returns true.

If seek is about to start, but an error occurs, allDone() signal is emitted with a false result, but true is still returned from here.

If seek cannot start, false is returned and no signals are emitted.

The list of files emitted in seekStarted() may be different than the list of files passed to startSeek() in case of duplicates or if any of the files was on the isForbiddenWad() list.

Parameters

wads	List of files that will be searched for.

Returns: True if seek was started. False otherwise.

QString Wadseeker::targetDirectory ( ) const

Target directory is a directory where all seeked files will be saved.

If setTargetDirectory() is called after a seek session is started with startSeek(), this method will return the newly set directory, not the one from the running session.

Returns: Path to a directory last set by setTargetDirectory().

Member Data Documentation

const QString Wadseeker::defaultSites[]

static

Default sites to scrape for files.

See Also: setPrimarySitesToDefault()

Definition at line 193 of file wadseeker.h.

const QString Wadseeker::forbiddenWads[]

static

Files that are not meant to be freely downloadable.

List of file names that will be ignored as they are part of a commercial product. Empty string at the end of the array is required and indicates the end of the array.

See Also: isForbiddenWad()

Definition at line 204 of file wadseeker.h.

The documentation for this class was generated from the following file:

src/wadseeker/wadseeker.h

Public Slots

Signals

Public Member Functions

Static Public Member Functions

Static Public Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation