Click Train Detector help (#59)

* Plugin Help

Small change to Help system so that plugins under development can more
easily display their help

* PamModel update

Notes on how to add a plugin to the model during the plugin development
phase.

* Click Train Detector Help

* Help for backup manager
This commit is contained in:
Douglas Gillespie 2022-09-21 15:54:01 +01:00 committed by GitHub
parent 6aefbc124b
commit 290f2fbb95
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
18 changed files with 238 additions and 93 deletions

View File

@ -146,6 +146,11 @@ public class ClickTrainControl extends PamControlledUnit implements PamSettings
* Click info manager- handles information from the algorithms.
*/
private CTAlgorithmInfoManager clAlgorithmInfoManager;
/**
* Help point which can be referenced form dialogs, etc.
*/
public static final String helpPoint = "detectors.ClickTrainDetector.docs.ClickTrainDetector";
/**
* Constructor for the ClickTrainControl.

View File

@ -587,6 +587,11 @@ public class ClickTrainAlgorithmPaneFX extends SettingsPane<ClickTrainParams> {
public void paneInitialized() {
}
@Override
public String getHelpPoint() {
return ClickTrainControl.helpPoint;
}
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@ -1,2 +1,2 @@
JavaSearch 1.0
TMAP bs=2048 rt=1 fl=-1 id1=6559 id2=1
TMAP bs=2048 rt=1 fl=-1 id1=6679 id2=1

Binary file not shown.

View File

@ -442,6 +442,8 @@
<mapID target="localisation.bearingLocaliser.docs.BL_Settings" url="localisation/bearingLocaliser/docs/BL_Settings.html"/>
<mapID target="detectors.ClickTrainDetector.docs.ClickTrainDetector" url="detectors/ClickTrainDetector/docs/ClickTrainDetector.html"/>
<mapID target="sound_processing.FiltersHelp.Docs.Filters_filters" url="sound_processing/FiltersHelp/Docs/Filters_filters.html"/>
<mapID target="localisation.bearingLocaliser.docs.BL_Output" url="localisation/bearingLocaliser/docs/BL_Output.html"/>
@ -642,6 +644,8 @@
<mapID target="mapping.AirgunDisplay.docs.AirgunConfiguration" url="mapping/AirgunDisplay/docs/AirgunConfiguration.html"/>
<mapID target="utilities.backupmanager.docs.backupmanager" url="utilities/backupmanager/docs/backupmanager.html"/>
<mapID target="localisation.targetmotion.docs.targetmotion_Localising" url="localisation/targetmotion/docs/targetmotion_Localising.html"/>
<mapID target="detectors.clickDetectorHelp.docs.offline_Tools" url="detectors/clickDetectorHelp/docs/offline_Tools.html"/>

View File

@ -1,7 +1,10 @@
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<!--generated by JHelpDev Version: 0.63, 14 May 2008, see jhelpdev.sourceforge.net--><!DOCTYPE helpset PUBLIC "-//Sun Microsystems Inc.//DTD JavaHelp HelpSet Version 2.0//EN" "http://java.sun.com/products/javahelp/helpset_2_0.dtd">
<helpset version="1.0">
@ -17,7 +20,10 @@
<title>Help </title>
@ -33,7 +39,10 @@
<maps>
@ -51,7 +60,10 @@
<homeID>
top
</homeID>
@ -67,7 +79,10 @@ top
<mapref location="Map.jhm"/>
@ -83,7 +98,10 @@ top
</maps>
@ -99,7 +117,10 @@ top
<view mergetype="javax.help.UniteAppendMerge">
@ -117,7 +138,10 @@ top
<name>
TOC
</name>
@ -135,7 +159,10 @@ TOC
<label>
TOC
</label>
@ -153,7 +180,10 @@ TOC
<type>
javax.help.TOCView
</type>
@ -171,7 +201,10 @@ javax.help.TOCView
<data>
PAMGUARDTOC.xml
</data>
@ -187,7 +220,10 @@ PAMGUARDTOC.xml
</view>
@ -203,7 +239,10 @@ PAMGUARDTOC.xml
<view>
@ -221,7 +260,10 @@ PAMGUARDTOC.xml
<name>
Index
</name>
@ -239,7 +281,10 @@ Index
<label>
Index
</label>
@ -257,7 +302,10 @@ Index
<type>
javax.help.IndexView
</type>
@ -275,7 +323,10 @@ javax.help.IndexView
<data>
PAMGUARDIndex.xml
</data>
@ -291,7 +342,10 @@ PAMGUARDIndex.xml
</view>
@ -307,7 +361,10 @@ PAMGUARDIndex.xml
<view>
@ -325,7 +382,10 @@ PAMGUARDIndex.xml
<name>
Search
</name>
@ -343,7 +403,10 @@ Search
<label>
Search
</label>
@ -361,7 +424,10 @@ Search
<type>
javax.help.SearchView
</type>
@ -379,7 +445,10 @@ javax.help.SearchView
<data engine="com.sun.java.help.search.DefaultSearchEngine">
JavaHelpSearch
</data>
@ -395,7 +464,10 @@ JavaHelpSearch
</view>

View File

@ -6,7 +6,7 @@
<project>
PAMGUARD
</project>
<projectdir>C:\PamguardCode2021\PamGuard NOAA2\src\help</projectdir>
<projectdir>C:\Users\dg50\source\repos\PAMGuardDG\src\help</projectdir>
<startpage>
index.html
</startpage>

View File

@ -191,6 +191,10 @@
<tocitem text="Importing " target="utilities.quickAnnotations.docs.import" image="topic"/>
</tocitem>
<tocitem text="Backup Manager " target="utilities.backupmanager.docs.backupmanager">
<tocitem text="Overview " target="utilities.backupmanager.docs.backupmanager" image="topic"/>
</tocitem>
</tocitem>
<tocitem text="Sensors ">
<tocitem text="Analog Array Sensors ">
@ -481,6 +485,10 @@
<tocitem text="Localisation " target="detectors.gpl.docs.localisation" image="topic"/>
</tocitem>
<tocitem text="Click Train Detector " target="detectors.ClickTrainDetector.docs.ClickTrainDetector">
<tocitem text="Overview " target="detectors.ClickTrainDetector.docs.ClickTrainDetector" image="topic"/>
</tocitem>
</tocitem>
<tocitem text="Classifiers ">
<tocitem text="Whistle Classifier ">

View File

@ -6,7 +6,6 @@
</head>
<body>
<h1 id="click-train-detector">Click Train Detector</h1>
<h2 id="overview">Overview</h2>
<p>When a toothed whale, bat or other echolocator uses echolocation
@ -33,24 +32,24 @@
with multiple types of acoustic data, from CPOD detections to single
channel and multi-channel hydrophone recordings.</p>
<h2 id="how-it-works">How it works</h2>
<p>PAMGuard's click train detector utilises both a detection and
<p>PAMGuards click train detector utilises both a detection and
classification stage to extract click trains from recordings.</p>
<p>The detection stage is currently based on a multi hypothesis
tracking (MHT) algorithm. This algorithm considers all possible
combinations of transient detections creating a large hypothesis
matrix which holds potential click trains. As more clicks are added to
the hypothesis matrix it grows exponentially and so, to prevent a
computer running out memory, it is regularly <em>pruned</em> to keep only
the most likely click trains over time. The assigned likelihood of a
click train is based on number of properties which can be defined in
by the user. For example, a user might select, ICI, Amplitude and
Correlation as variables to score click trains; this would mean that
combinations of clicks with slowly changing ICI, amplitude and
waveforms would be favoured by the algorithm and stay in the
hypothesis matrix. Other properties such as bearing, click length and
peak frequency can also be selected. A graphical explanation of the
click train detection algorithm is shown in Figure 1 and a more
detailed explanation of the be found in Macaulay (2019).</p>
computer running out memory, it is regularly “pruned” to keep only the
most likely click trains over time. The assigned likelihood of a click
train is based on number of properties which can be defined in by the
user. For example, a user might select, ICI, Amplitude and Correlation
as variables to score click trains; this would mean that combinations
of clicks with slowly changing ICI, amplitude and waveforms would be
favoured by the algorithm and stay in the hypothesis matrix. Other
properties such as bearing, click length and peak frequency can also
be selected. A graphical explanation of the click train detection
algorithm is shown in Figure 1 and a more detailed explanation of the
be found in Macaulay (2019).</p>
<p align="center">
<img width="930" height="900" src="resources/mht_diagram.png">
</p>
@ -67,18 +66,18 @@
pruned. After a minimum of Npmin clicks (in this case 4) each track
hypothesis (possible click train) is assigned a &Chi;<sup>2</sup> score.
The track hypothesis with lowest score (defined by larger coloured
circles) has it's branch traced back Np (in this case 3) clicks.
Any track hypothesis which do not include the click Np steps back are
circles) has its branch traced back Np (in this case 3) clicks. Any
track hypothesis which do not include the click Np steps back are
pruned (defined by the double lines). Clicks which share no click
associations with the first track hypothesis are then pruned and the
process repeats until all clicks are part of a track or a maximum
number of tracks have been considered (in this example there are two
tracks). The algorithm then moves to the next click, adds it to the
hypothesis matrix, assigns &Chi;<sup>2</sup> scores and traces the
lowest &Chi;<sup>2</sup> branch Np steps back, pruning the hypothesis
matrix again; the process repeats until the last click. Note that
there is always a track hypothesis with no associated clicks (i.e.
the bottom-most branch where no clicks belong to a click train). If a
hypothesis matrix, assigns &Chi;<sup>2</sup> scores and traces the lowest
&Chi;<sup>2</sup> branch Np steps back, pruning the hypothesis matrix
again; the process repeats until the last click. Note that there is
always a track hypothesis with no associated clicks (i.e. the
bottom-most branch where no clicks belong to a click train). If a
track hypothesis is confirmed and thus removed from the hypothesis
matrix, then this track can be used to start another click train
</em>
@ -98,12 +97,12 @@
correlation of the average spectrum of the click train with a
predefined spectral template.</p>
<p>A click train which has been both detected and classified is
saved to PAMGuard's database and can be reclassified in PAMGuard's
saved to PAMGuards database and can be reclassified in PAMGuards
viewer mode.</p>
<h2 id="configuring-the-click-train-detector">Configuring the
click train detector</h2>
<p>The primary settings to configure can be split into MHT Kernel
and &Chi;<sup>2</sup> settings, these are all set in the primary click train
and &Chi;^2 settings these are all set in the primary click train
detector dialog as shown in Figure 2.</p>
<p align="center">
<img width="850" height="700" src="resources/detection_pane.png">
@ -119,8 +118,8 @@
important because they influence speed (a larger number of possible
click trains in memory is more processor intensive) and the quality of
the detections (the larger the number of possibilities the more likely
that <em>true</em> click trains are contained in the hypothesis matrix).
The specific settings are;</p>
that “true” click trains are contained in the hypothesis matrix). The
specific settings are;</p>
<p>
<strong><em>Prune-back</em></strong>: The hypothesis matrix needs
pruned so that it does not grow exponentially and cause memory issues.
@ -152,7 +151,7 @@
<p>
<strong><em>Max no. trains</em></strong>: This is a maximum allowed
number of trains in the hypothesis mix. Note this refers to the number
of trains which can survive pruning - the actual number of potential
of trains which can survive pruning the actual number of potential
click trains in the hypothesis mix will be much larger. Generally,
just via pruning, the hypothesis matrix will keep itself below the max
no. trains, however, in certain situations it can grow too large and
@ -165,8 +164,8 @@
</h3>
<p>
&Chi;<sup>2</sup> is a measure of the likelihood that a click train is
from a (usually) biological source. The higher the &Chi;<sup>2</sup>
value the lower the quality of a click train.
from a (usually) biological source. The higher the &Chi;<sup>2</sup> value
the lower the quality of a click train.
</p>
<p>
The &Chi;<sup>2</sup> model used in the click train detector considers
@ -198,8 +197,8 @@
influence on the calculation of &Chi;<sup>2</sup> and decreasing means
that the descriptor has a larger influence on &Chi;<sup>2</sup>. In some
cases, clicks can be so close together that the variance is tiny and
thus &Chi;<sup>2</sup> becomes huge e.g. during buzzes. A minimum
variance value (<em>qt<sub>i</sub></em>) prevents the variance <em>(max¡(q<sub>i</sub>
thus &Chi;<sup>2</sup> becomes huge e.g. during buzzes. A minimum variance
value (<em>qt<sub>i</sub></em>) prevents the variance <em>(max(q<sub>i</sub>
(t<sub>(k+1)</sub>-t<sub>k</sub> ),qt<sub>i</sub> )<sup>2</sup>)
</em> from falling below very low values.
</p>
@ -229,7 +228,7 @@
between detections.
</p>
<p>
<strong><em>Amplitude:</em></strong> the amplitude in dB re 1&mu;Pa pp.
<strong><em>Amplitude:</em></strong> the amplitude in dB re 1/muPa pp.
The algorithm looks for slowly changing amplitude values. Note that
the algorithm is comparing the change in change in amplitude so that
the click train algorithm is not biased against large but consistent
@ -312,14 +311,14 @@
<strong><em>Long track bonus:</em></strong> add a bonus factor for
longer click trains to prevent fragmentation. This is the total length
of the click train in seconds divided by the total hypothesis matrix
time in seconds L which is then multiplied so that &Chi;<sup>2</sup> =
(&Chi;<sup>2</sup>*L)<sup>LT</sup> where LT is the long track bonus.
time in seconds L which is then multiplied so that &Chi;<sup>2</sup> = (&Chi;<sup>2</sup>*L)<sup>LT</sup>
where LT is the long track bonus.
</p>
<p>
<strong><em>Coast penalty:</em></strong> add a penalty for
'coasting' i.e. when an expected click, based on ICI, is not
present in the click train. This penalty is multiplied by the number
of coasts i.e. the likely number of missed clicks based on ICI
<strong><em>Coast penalty:</em></strong> add a penalty for “coasting”
i.e. when an expected click, based on ICI, is not present in the click
train. This penalty is multiplied by the number of coasts i.e. the
likely number of missed clicks based on ICI
</p>
<p>
<strong><em>New Track Penalty:</em></strong> if a track hypothesis is
@ -361,10 +360,10 @@
&Chi;<sup>2</sup> threshold classifier
</h3>
<p>
The click train is classified if it's overall &Chi;<sup>2</sup> value
is lower than the set &Chi;<sup>2</sup> Threshold and it has more than
Min. Clicks and the time between the first and last click is greater
than Min. Time
The click train is classified if its overall &Chi;<sup>2</sup> value is
lower than the set &Chi;<sup>2</sup> Threshold and it has more than Min.
Clicks and the time between the first and last click is greater than
Min. Time
</p>
<h3 id="idi-classifier">IDI Classifier</h3>
<p>The click train is classified if the median/mean and standard
@ -374,11 +373,11 @@
<p>The click train is classified if the average spectra of the
click train has a correlation value above Spectrum Correlation
Threshold with a user defined spectral template. The template can be
set using the button on the top right of the spectrum plot – a
default spectrum can be loaded or a spectrum can be loaded from a .mat
or .csv file. A csv file should have the first row as the spectrum and
first column of the second row the sample rate. A .mat file should be
a single saved structure with sR (sample rate) and spectrum (array of
set using the button on the top right of the spectrum plot a default
spectrum can be loaded or a spectrum can be loaded from a .mat or .csv
file. A csv file should have the first row as the spectrum and first
column of the second row the sample rate. A .mat file should be a
single saved structure with sR (sample rate) and spectrum (array of
spectrum values) fields.</p>
<h3 id="bearing-classifier">Bearing Classifier</h3>
<p>The click train is classified if minimum and maximum bearing
@ -389,7 +388,7 @@
<h2 id="parametrising-the-classifier">Parametrising the classifier</h2>
<p>Each classifier has a set of metadata that are added to click
trains. This can be accessed through the tooltip or right click menus
in various displays. For example, in the Time Base Display FX hover
in various displays. For example, in the Time Base Display Fx hover
the mouse over a click train or bring the pop menu with a right click.
Parameters such as the spectral correlation value, IDI and bearing
information etc are displayed which allows users to get an idea of
@ -464,23 +463,23 @@
are localised.</em>
</p>
<p>In viewer mode, add the module and then go to Settings&gt;Click
Train Detector &gt; Reanalyse click trains.This will bring up
PAMGuard's generic data reprocessing dialog with two settings, Click
Train Detector &gt; Reanalyse click trains.This will bring up
PAMGuards generic data reprocessing dialog with two settings, Click
Train Detector or Click Train Classifier. The Click Train Detector
option will run the detection and classification algorithm again. The
Click Train Classifier will only run the classification algorithm on
existing detected click trains (much faster). Note that users can
select how much data to reprocess in the Data dropdown menu. All
Data means the entire dataset will be reprocessed, Loaded Data means
just the current data loaded in the display (all scrollable data),
Select Data allows the user to define two time limits between which
all data is reprocessed.</p>
select how much data to reprocess in the Data dropdown menu All Data
means the entire dataset will be reprocessed, Loaded Data means just
the current data loaded in the display (all scrollable data), Select
Data allows the user to define two time limits between which all data
is reprocessed.</p>
<h2 id="visualising-results">Visualising Results</h2>
<p>The results from the click train detector can be visualised in a
variety of displays in PAMGuard.</p>
<h3 id="click-bearing-time-display">Click bearing time display</h3>
<p>By default, clicks trains will be shown in the Click Detector
Module's in built bearing time display. Different click trains are
Modules in built bearing time display. Different click trains are
represented as different colours. Note that you must right click on
the display and select Colour by Click Trains</p>
<p align="center">
@ -492,31 +491,22 @@
bearing time display. Different colours correspond to different click
trains.</em>
</p>
<h3 id="time-display-fx">Time Display FX</h3>
<p>The Time Display FX is a more modern display which allows any
<h3 id="time-display-fx">Time Display Fx</h3>
<p>The Time Display Fx is a more modern display which allows any
time-based data to be plotted together on a large variety of y-axis
(e.g., frequency, bearing, amplitude etc.). Click trains will be
plotted on the time-based display by adding Click detections to the
display and then using the right</p>
<p align="center">
<img width="940" height="500" src="resources/clicktrain_TDFX.png">
<img width="940" height="500" src="resources/clicktrain_TDFx.png">
</p>
<p>
<em>Click train data displayed in the time display FX. Users can
<em>Click train data displayed in the time display Fx. Users can
right click on click trains to view average spectra and waterfall
spectrograms (shown here in top right).</em>
</p>
<p CLASS="previousLink">
<a href=""></a>
</p>
<p CLASS="nextLink">
<a href="ClickDetector_clickDetectorDisplays.html">Next: Click
Detector Displays </a>
</p>
<br>
<br>
</body>
</html>

View File

@ -0,0 +1,61 @@
<html>
<head>
<link href="../../../pamHelpStylesheet.css" rel="STYLESHEET"
type="text/css" />
<title>Backup Manager</title>
</head>
<body>
<h1>Backup Manager</h1>
<h2>Overview</h2>
<p>
The backup manager can be used to automatically backup (copy) data from PAMGuard on a daily basis, e.g. to external hard drives. For example, you might initially
write data to internal hard drives, then want copies of the data made to external hard drives prior to deleting the data data on the internal drives. </p>
<p>Note that this module is really intended to be used with PAMGuard installations running continuously, and is probably of very little use if you're using it
on a more ad-hoc or daily basis from a boat with an operator present</p>
<p>The three main outputs from PAMGuard are the <a href="../../BinaryStore/docs/binarystore_overview.html">Binary Store</a>,
the <a href="../../generalDatabaseHelp/docs/database_database.html">Database</a>, and the
<a href="../../../sound_processing/soundRecorderHelp/docs/RecorderOverview.html">Sound Recorder</a>. Data for each of these can be automatically copied
each day and then, if you wish, also deleted from the main hard drive. <p>
<h2>Configuration</h2>
<p>To use the backup manager, add a Backup Manager module from the utilities group menu: <b>File>Add Module>Utilities>Backup Manager</b></p>
<p>You then add (or use an existing) <a href="../../../displays/userDisplayHelp/docs/userDisplayPanel.html">User Display</a> and add a
Backup Manager Display from the User Display menu. </p>
<center><img src="./images/backupdisplay1.png">
<br><em>Figure 1. Backup manager display panel prior to configuration.</em></center>
<br>
<p>Until correctly configured, the backup manager won't do anything. For each type of data you should add one or more <em>actions</em>. Possible actions for
the binary store and the sound recorder are "Copy Files", "Move Files", "Delete Files". The actions will take place in order, and the Delete action will not run
if a preceding copy action has failed (e.g. due to an external hard disk being full).</p>
<p>Add the actions you want, then click on the configure button (small wheel) to set the destination folder.
<center><img src="./images/backupdisplay2.png">
<br><em>Figure 1. Backup manager display panel prior to configuration.</em></center>
<br>
<p>Actions for the database are slightly different: "Copy database" and "Clone database". "Copy"" performs a straight forward
file copy of the database file. "Clone" creates a new database, then runs a series of SQL commands to copy data from the old to
the new database. The latter is slightly safer since the copied database can become corrupted if the database it updated while
it is copying.</p>
<h3>Other types of data</h3>
<p>If you also want to back up other types of data being collected on the same computer, but not by PAMGuard, then you can use the menu
<b>Settings>Backup Manager>Add Stream</b> and then configure the new data stream on the main backup manager display to take files from within
a designated folder and Copy / Move / Delete them as for the binary files and recordings above. As an example, we are using this to automatically
backup data being collected using Tritech Genesis software which acquires data from multibeam sonars.<p>
<h2>Running and Pausing</h2>
<p>There is currently only one available backup schedule whereby the backup is run each day at at 01:00 UTC. Other possible schedules allowing
more user control may be added in the future.</p>
<p>By default, the backup will run every day. You can pause the backup, e.g. pausing for a few days while you wait for someone to change over external
hard drives. </p>
<h2>Database Output</h2>
<p>If a database is present on the system, then a record of all backup actions for each file will be recorded in the database.</p>
</body>
</html>

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.2 KiB